[OpenSER-Devel] [ openser-Patches-1803180 ] New Sleepcat DB Module

Fri Sep 28 07:04:32 CEST 2007

Patches item #1803180, was opened at 2007-09-26 21:43
Message generated for change (Comment added) made by will_quan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=743022&aid=1803180&group_id=139143

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: modules
Group: ver 1.2.x
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Will Quan (will_quan)
Assigned to: Henning Westerholt (henningw)
Summary: New Sleepcat DB Module

Initial Comment:
All,

Please consider adding the following module to the Openser codebase.

The attached tarball contains the source and docs to a new module named 'sleepycat'. This is a DB module that incorporates the Berkeley DB in Openser. Berkeley DB is not a relational DB, but rather a high-performance embedded DB kernel.

The Berkeley DB offers a couple of architectures:
1. Concurrent Data Store (CDS)
2. Transaction Data Store

This module uses the CDS model. One of the main differences between these architectures is DB recovery and robustness (Please refer to the Berkeley DB for full details). In a nutshell, the onus of recovery is on the application (e.g. openser). This modules contains a recovery program utils/sleepycat/sc_recover that fulfills this need.

Like other DB modules, the DB must be created before Openser can be started. There is a script scripts/sleepycat.sh that can be used to create the DB once the Berkeley DB is installed.

**Important** Berkeley DB v4.6.19 contains a bug that will most certainly result in deadlock. There is a patch available for Berkeley DB 4.6.19, and the fix is to be sourced in the next release of 4.6. For now I recommend 4.5 (older version should work too).

Special thanks to the authors of dbtext and postgesql modules. Some of this code may look familiar to you all .

dbtext - Daniel-Constantin Mierla

postgres - Greg Fausak, Andy Fullford, Norman Brandinger

Regards,
William Quan
Cisco Systems

----------------------------------------------------------------------

>Comment By: Will Quan (will_quan)
Date: 2007-09-28 00:04

Message:
Logged In: YES 
user_id=1850067
Originator: YES

Thanks Henning, I'll look into it. -w

----------------------------------------------------------------------

Comment By: Henning Westerholt (henningw)
Date: 2007-09-27 12:29

Message:
Logged In: YES 
user_id=337916
Originator: NO

Hello William,

i was able to did a short review of the code already today. Some notes:

1. Makefile

You use hard coded library paths in the Makefile, this should be changed
to 
use the $(LOCALBASE) variable. See the mysql Makefile in 1.2 for an
example.

2. code duplication

In sc_val.c you implement some datatype conversation (e.g. double to
string).
You should use the common implementation of this function in db/db_ut.h.

For the db_free_rows and db_free_row there exists also a implementation in
db/db_row.h and
db/db_res.h. Please consider using them, as would like to avoid unecessary
code duplication.

3. Warnings

If you compile the code in the trunk, there are several warnings. These
should be
fixed. (One result from the changed module interface, just add a "0" after
the
/* exported pseudo-variables */ field.)

4. Documentation
Have you tried to create the documentation with "make modules-readme"? On
my system
i get some sgml errors. And how do i generate the _design.sgml document?

5. Utils
The sc_recover (appart from some warnings) uses some non portable
functions (e.g. strlcat).
These are not available on Linux, the program don't compile for me.
You should consider using the universal available strcat, despite the
security concerns.

6. Setup scripts
There exist some duplication of functionality in the script too, and this
uses the old
database creation structure. This was updated in the trunk some month
ago.

I don't think i'll be able to do this changes myself, as the remaining
time until
the code freeze is rather short. I can probably do some integration work
for the setup
scripts, but have no time for the code related changes. 

If you provide an updated version, please create a patch against the svn
trunk.

Thank you again for your contribution, its really appreciated
notwithstanding 
my comments. :-)

Henning

----------------------------------------------------------------------

Comment By: Will Quan (will_quan)
Date: 2007-09-27 08:56

Message:
Logged In: YES 
user_id=1850067
Originator: YES

klaus,
i do not mind changing the name if it makes more sense to people.

henning,
I was working from the openser 1.2 branch.
90% of dev and testing was on FreeBSD (and MacOS). Limited desktop testing
on Fedora 6.
As for the DB versions, I used 4.6.19 and 4.5.20. 

This thread has a necessary patch to DB v4.6.19, if you want to to that
route.
http://forums.oracle.com/forums/thread.jspa?threadID=551806&tstart=15

-will

----------------------------------------------------------------------

Comment By: Klaus Darilion (klaus_darilion)
Date: 2007-09-27 04:41

Message:
Logged In: YES 
user_id=1318360
Originator: NO

hi! shouldn't we name the module "berkeley_db" instead of sleepycat? I
think berkeley DB is more known than sleepycat.

----------------------------------------------------------------------

Comment By: Henning Westerholt (henningw)
Date: 2007-09-27 02:37

Message:
Logged In: YES 
user_id=337916
Originator: NO

Hello Will,

that module sounds really interesting, thank you for the contribution!
>From what version of openser did you use some code, 1.2 branch or trunk?
And what version was tested?

I'll review it tomorow and probably integrate it into the trunk, if there
are no issues.

Cheers,

Henning

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=743022&aid=1803180&group_id=139143