[OpenSER-Devel] RFC: memory management in database modules

[Bogdan-Andrei Iancu]

Hi Henning,

I agree with you. I see here two arguments for this:
    1) if mysql is not duplicating the date returned from driver, it 
means all the modules using DB are already safe from this point of view 
- they do their one copy and they do not count on the persistence of 
data returned from DB.
    2) copying data too many times may have a performance impact, but 
not because of alloc/copy/free ops, but mainly because of memory 
fragmentation - the size of data operated with DB vary a lot (like size 
of chucks), so the impact may be huge.

But I have here a note: it may not be possible in all case to pass the 
pointer returned by driver to the upper layer (module) as the data 
returned by driver may need some pre-processing. Like the postgres 
module does for string and blobs (if I'm not wrong) to do escape and 
unescape. So, the DB module may hide (totally transparent) that certain 
fields are re-allocated due some pre-processing. This extra mem must be 
also freed (also transparent) by the DB module when the result is freed.
This will not break the the overall behaviour, but I just mentioned 
because the no suppositions should made on the data returned by the DB 
module -  it may be allocated by underlaying driver, may be in openser 
pkg mem or in heap. If a module needs the data, it must make a copy!

Regards,
Bogdan

Henning Westerholt wrote:
> Hi,
>
> i've a question about the 'correct' way to do the memory management for 
> results in the database modules. 
>
> At the moment there existing two different styles:
>
> mysql, unixodbc don't allocate new memory and just assign the string pointer 
> of the result set to the internal OpenSER representation. The modules that 
> needs to use this results needs to copy them, because there are not available 
> after a call to the DB specific result free function.
>
> postgres, db_berkeley allocate new memory and copy all string values from the 
> database to the internal representation. Modules that uses this driver don't 
> need to copy there values, even after the freeing of the result set. 
>
> As mysql is the most used database, (i assume) that every module copy the 
> values from the result set after the query execution. This is unnecessary for 
> the postgres DB, and further prevents me from using only one memory 
> management function for the internal representation of the DB structures.
>
> As we have many more modules than database connectors in the code, i think it 
> would make more sense to change the postgres and db_berkeley module to match 
> the behaviour of the mysql module.
>
> Any comments? 
>
> Henning
>
> _______________________________________________
> Devel mailing list
> Devel at lists.openser.org
> http://lists.openser.org/cgi-bin/mailman/listinfo/devel
>
>