[OpenSER-Devel] [ openser-Patches-1803180 ] New Sleepcat DB Module
SourceForge.net
noreply at sourceforge.net
Thu Oct 4 13:16:19 CEST 2007
Patches item #1803180, was opened at 2007-09-27 02:43
Message generated for change (Comment added) made by henningw
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=743022&aid=1803180&group_id=139143
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: modules
Group: ver devel
>Status: Closed
Resolution: Accepted
Priority: 9
Private: No
Submitted By: Will Quan (will_quan)
Assigned to: Henning Westerholt (henningw)
Summary: New Sleepcat DB Module
Initial Comment:
All,
Please consider adding the following module to the Openser codebase.
The attached tarball contains the source and docs to a new module named 'sleepycat'. This is a DB module that incorporates the Berkeley DB in Openser. Berkeley DB is not a relational DB, but rather a high-performance embedded DB kernel.
The Berkeley DB offers a couple of architectures:
1. Concurrent Data Store (CDS)
2. Transaction Data Store
This module uses the CDS model. One of the main differences between these architectures is DB recovery and robustness (Please refer to the Berkeley DB for full details). In a nutshell, the onus of recovery is on the application (e.g. openser). This modules contains a recovery program utils/sleepycat/sc_recover that fulfills this need.
Like other DB modules, the DB must be created before Openser can be started. There is a script scripts/sleepycat.sh that can be used to create the DB once the Berkeley DB is installed.
**Important** Berkeley DB v4.6.19 contains a bug that will most certainly result in deadlock. There is a patch available for Berkeley DB 4.6.19, and the fix is to be sourced in the next release of 4.6. For now I recommend 4.5 (older version should work too).
Special thanks to the authors of dbtext and postgesql modules. Some of this code may look familiar to you all .
dbtext - Daniel-Constantin Mierla
postgres - Greg Fausak, Andy Fullford, Norman Brandinger
Regards,
William Quan
Cisco Systems
----------------------------------------------------------------------
>Comment By: Henning Westerholt (henningw)
Date: 2007-10-04 11:16
Message:
Logged In: YES
user_id=337916
Originator: NO
Commited in revision 2844, discussion moved to mailing list.
----------------------------------------------------------------------
Comment By: Henning Westerholt (henningw)
Date: 2007-10-04 09:31
Message:
Logged In: YES
user_id=337916
Originator: NO
OK, i'll do the upload later. Will, do you plan to further extend the
module? Or are
you satisfied with the current functionality?
Regards the naming: I agree that there is a need for a better organisation
of the
modules. Perhaps we could also add a few directories, but for this type of
changes
its now a little bit to late.
For now db_berkeley is fine for me.
----------------------------------------------------------------------
Comment By: Klaus Darilion (klaus_darilion)
Date: 2007-10-04 09:26
Message:
Logged In: YES
user_id=1318360
Originator: NO
I second the db_... naming. We could change all the module names already
for 1.3.
----------------------------------------------------------------------
Comment By: Bogdan (bogdan_iancu)
Date: 2007-10-04 09:00
Message:
Logged In: YES
user_id=1275325
Originator: NO
Hi Henning, Hi William,
The remaining items are not critical, so I guess we can proceed with the
upload (we will sort them out later). Henning should you do the upload? or
we should grant Will SVN access for the module?
Regarding the naming: I have a third opinion :D - I think we should use a
prefix for all the db module, like we have "mi_" for all modules
implementing the MI interface. So maybe we should do "db_" for all the
modules implementing the DB API -> db_berkley and we will move (in the
future) the rest of the modules (postgres, mysql, flatstore, dbtext) to
this format...Just an idea :)
Regards,
Bogdan
PS: also I can do the upload just not to bother any of you with changing
the name again L:)
----------------------------------------------------------------------
Comment By: Henning Westerholt (henningw)
Date: 2007-10-04 08:38
Message:
Logged In: YES
user_id=337916
Originator: NO
Hello William,
thank you, good work! Your patch looks resonable and is integrated into
the build process too.
Sorry for the delay, i've been on holidays as well..
Some work still remains:
- build berkley_db as own debian package
- add berkley_db to the xml database creation process to generate the
content in
scripts/berkley_db/openser like the dbtext stuff
- evaluate if its possible to use the db_free_row and db_free_rows
functions for this module too (and
perhaps for postgresql?)
I'll take care of these points after the upload to trunk.
Regards the naming: Is berkley_db fine, or should we go to berkleydb? I'm
not a fan of to much
underscores in names.. ;-) But this of course not really an issue for me.
Bogdan, no objections from my side anymore to include this module into
trunk. The remaining issue could be sorted out in the next week, just as
you said.
Cheers,
Henning
----------------------------------------------------------------------
Comment By: Bogdan (bogdan_iancu)
Date: 2007-10-03 14:28
Message:
Logged In: YES
user_id=1275325
Originator: NO
Hi Will,
The module will definitly find its way in the 1.3 release. Thanks for your
work and also thanks to Henning for the overview - I will take a look asap
and if there are no major stopers, I will make the upload on the SVN. For
smaller polishes and tunnings there is time during the testing phase.
Thanks and regards,
Bogdan
----------------------------------------------------------------------
Comment By: Will Quan (will_quan)
Date: 2007-10-02 18:49
Message:
Logged In: YES
user_id=1850067
Originator: YES
Henning,
I am including a new patch for the trunk. Could you review this when you
get a chance?
-will
File Added: berkeleydb-trunk.patch
----------------------------------------------------------------------
Comment By: Will Quan (will_quan)
Date: 2007-09-29 00:49
Message:
Logged In: YES
user_id=1850067
Originator: YES
Great comments and thank you for your support!
I have finished the first 5 of Henning's requests and have also renamed
everything to 'berkeley_db'.
I am now working on the Setup scripts (item 6). I plan to upload a new
patch (from trunk) tomorrow.
Sorry for these delays; I've been on holiday this week.
Regards,
-will
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2007-09-28 11:32
Message:
Logged In: NO
Will,
Great job, thank you !!
Also took a look at your module and agree with Henning's comments. I'll
try to assist in integrating the module into the database structures
that OpenSER currently uses. Just can't lead the effort right now, so
if someone takes control of leading the effort, I'll lend a hand.
Regards,
Norm
----------------------------------------------------------------------
Comment By: Will Quan (will_quan)
Date: 2007-09-28 05:04
Message:
Logged In: YES
user_id=1850067
Originator: YES
Thanks Henning, I'll look into it. -w
----------------------------------------------------------------------
Comment By: Henning Westerholt (henningw)
Date: 2007-09-27 17:29
Message:
Logged In: YES
user_id=337916
Originator: NO
Hello William,
i was able to did a short review of the code already today. Some notes:
1. Makefile
You use hard coded library paths in the Makefile, this should be changed
to
use the $(LOCALBASE) variable. See the mysql Makefile in 1.2 for an
example.
2. code duplication
In sc_val.c you implement some datatype conversation (e.g. double to
string).
You should use the common implementation of this function in db/db_ut.h.
For the db_free_rows and db_free_row there exists also a implementation in
db/db_row.h and
db/db_res.h. Please consider using them, as would like to avoid unecessary
code duplication.
3. Warnings
If you compile the code in the trunk, there are several warnings. These
should be
fixed. (One result from the changed module interface, just add a "0" after
the
/* exported pseudo-variables */ field.)
4. Documentation
Have you tried to create the documentation with "make modules-readme"? On
my system
i get some sgml errors. And how do i generate the _design.sgml document?
5. Utils
The sc_recover (appart from some warnings) uses some non portable
functions (e.g. strlcat).
These are not available on Linux, the program don't compile for me.
You should consider using the universal available strcat, despite the
security concerns.
6. Setup scripts
There exist some duplication of functionality in the script too, and this
uses the old
database creation structure. This was updated in the trunk some month
ago.
I don't think i'll be able to do this changes myself, as the remaining
time until
the code freeze is rather short. I can probably do some integration work
for the setup
scripts, but have no time for the code related changes.
If you provide an updated version, please create a patch against the svn
trunk.
Thank you again for your contribution, its really appreciated
notwithstanding
my comments. :-)
Henning
----------------------------------------------------------------------
Comment By: Will Quan (will_quan)
Date: 2007-09-27 13:56
Message:
Logged In: YES
user_id=1850067
Originator: YES
klaus,
i do not mind changing the name if it makes more sense to people.
henning,
I was working from the openser 1.2 branch.
90% of dev and testing was on FreeBSD (and MacOS). Limited desktop testing
on Fedora 6.
As for the DB versions, I used 4.6.19 and 4.5.20.
This thread has a necessary patch to DB v4.6.19, if you want to to that
route.
http://forums.oracle.com/forums/thread.jspa?threadID=551806&tstart=15
-will
----------------------------------------------------------------------
Comment By: Klaus Darilion (klaus_darilion)
Date: 2007-09-27 09:41
Message:
Logged In: YES
user_id=1318360
Originator: NO
hi! shouldn't we name the module "berkeley_db" instead of sleepycat? I
think berkeley DB is more known than sleepycat.
----------------------------------------------------------------------
Comment By: Henning Westerholt (henningw)
Date: 2007-09-27 07:37
Message:
Logged In: YES
user_id=337916
Originator: NO
Hello Will,
that module sounds really interesting, thank you for the contribution!
>From what version of openser did you use some code, 1.2 branch or trunk?
And what version was tested?
I'll review it tomorow and probably integrate it into the trunk, if there
are no issues.
Cheers,
Henning
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=743022&aid=1803180&group_id=139143
More information about the Devel
mailing list