[SR-Users] No memory left in send_subscribe (PUA) from update_pua while building the tm dlg_t structure

Daniel-Constantin Mierla miconda at gmail.com
Fri Dec 23 12:36:31 CET 2011


Hello,

looks like a leak in a module that is storing record-routes and use them 
later, perhaps pua module, I will check it soon.

Cheers,
Daniel

On 12/23/11 11:31 AM, laura testi wrote:
> Hi Daniel,
> I just follow the instruction in the link you sent
> (http://www.asipto.com/pub/kamailio-devel-guide/#c04troubleshooting)
> to use gdb to print PKG fragments. When I got the error in PUA:
>
> Dec 23 11:10:53 /.../sbin/kamailio[23276]: ERROR: pua
> [send_subscribe.c:158]: No memory left for size:439
> Dec 23 11:10:53 /.../sbin/kamailio[23276]: ERROR: pua [pua.c:747]:
> while building tm dlg_t structure
> Dec 23 11:10:53 /.../sbin/kamailio[23276]: ERROR: pua [pua.c:652]:
> while updating record
>
>
> The I run the command
> gdb /.../sbin/kamailio 23276
> and write the following commands in the gdb:
>
> set $i=0
> set $a = mem_block->first_frag
> while($i<10000)
>    if($i>2000)
>      if($a->u.is_free==0)
>        p *$a
>      end
>    end
>    set $a = ((struct qm_frag*)((char*)($a)+sizeof(struct
> qm_frag)+((struct qm_frag*)$a)->size+sizeof(struct qm_frag_end)))
>    set $i = $i + 1
> end
> ...
>
> after a while I got a lot of prints on the screen like these:
>
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1348 = {size = 72, u = {nxt_free = 0x0, is_free = 0},
>    file = 0x5d76c9 "<core>: parser/parse_rr.c",
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1349 = {size = 72, u = {nxt_free = 0x0, is_free = 0},
>    file = 0x5d76c9 "<core>: parser/parse_rr.c",
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1350 = {size = 72, u = {nxt_free = 0x0, is_free = 0},
>    file = 0x5d76c9 "<core>: parser/parse_rr.c",
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1351 = {size = 104, u = {nxt_free = 0x0, is_free = 0},
>    file = 0x5d76c9 "<core>: parser/parse_rr.c",
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1352 = {size = 104, u = {nxt_free = 0x0, is_free = 0},
>    file = 0x5d76c9 "<core>: parser/parse_rr.c",
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1353 = {size = 72, u = {nxt_free = 0x0, is_free = 0},
>    file = 0x5d76c9 "<core>: parser/parse_rr.c",
>    func = 0x5d7b00 "do_parse_rr_body", line = 74, check = 4042322160}
> $1354 = {size = 72, u = {nxt_free = 0x0, is_free = 0},
> ---Type<return>  to continue, or q<return>  to quit---
>
> ...
>
>
> But I don't understand if these are normal or something goes wrong....
>
> Can you help
>
>
> Best Regards,
> Laura
>
> On Wed, Dec 21, 2011 at 12:18 PM, Daniel-Constantin Mierla
> <miconda at gmail.com>  wrote:
>> Hello,
>>
>> pkg.stats was added in 3.2.0, iirc. For 3.1, you can walk the packets in
>> memory with gdb -- 3.1 has memory debug on, so you don't need to recompile
>> (unless you turned it off).
>>
>> Just attach to the pid of a sip worker  (gdb /path/to/kamailio _pid_value_)
>> and run the gdb script.
>>
>> Cheers,
>> Daniel
>>
>>
>> On 12/21/11 11:58 AM, laura testi wrote:
>>> Hi Daniel,
>>> I try the sercmd for pkg memory but it return 500 error:
>>>
>>>
>>> # sercmd
>>> sercmd 0.2
>>> Copyright 2006 iptelorg GmbH
>>> This is free software with ABSOLUTELY NO WARRANTY.
>>> For details type `warranty'.
>>> sercmd>    pkg.stats
>>> error: 500 - command pkg.stats not found
>>> sercmd>
>>>
>>>
>>>
>>> Is it available only for 3.2.x and master branch? Because we are using
>>> 3.1.5. But take the PUA module from master branch for the fetch_row
>>> parameter you have patched ;-)
>>>
>>>
>>> core.shmmem is ok.
>>>
>>>
>>>
>>> Thanks a lot
>>>
>>> Laura
>>>
>>> On Wed, Dec 21, 2011 at 10:58 AM, Daniel-Constantin Mierla
>>> <miconda at gmail.com>    wrote:
>>>> Hello,
>>>>
>>>> you can see the available pkg via sercmd, sending command pkg.stats
>>>> (match
>>>> the entry for the pid printing the error). If there is no free memory,
>>>> then
>>>> might be a leak.
>>>>
>>>> You can attach with gdb to the pid printing these errors and walk to pkg,
>>>> you see the commands for gdb at:
>>>>
>>>> http://www.asipto.com/pub/kamailio-devel-guide/#c04troubleshooting
>>>>
>>>> See if you have lot of allocated chunks from same place in source code
>>>> (ignore those at the beginning, mainly related to cfg parsing) and send
>>>> the
>>>> details here.
>>>>
>>>> Cheers,
>>>> Daniel
>>>>
>>>>
>>>> On 12/21/11 10:44 AM, laura testi wrote:
>>>>
>>>> Hi,
>>>> we are using the PUA_XMPP and PUA modules from the master branch. When
>>>> the modules are started, everything are ok, the presence events from
>>>> XMPP are sent to kamailio SIP servers (PUBLISH/SUBSCRIBE) and cached
>>>> in the hash. But when there are several thousands records in the hash
>>>> tabel, the update_pua function called in the hashT_clean gives a lot
>>>> of "No memory left" error when the hashT_clean is waked up from the
>>>> time:
>>>>
>>>> ERROR: pua [send_subscribe.c]: No memory left
>>>> ERROR: pua [pua.c]: while building tm dlg_t structure
>>>>
>>>> The failed call is:
>>>>         td = (dlg_t*)pkg_malloc(size);
>>>>         if(td == NULL)
>>>>         {
>>>>                 LM_ERR("No memory left\n");
>>>>                 return NULL;
>>>>         }
>>>>
>>>> in dlg_t* pua_build_dlg_t(ua_pres_t* presentity) function in
>>>> send_subscribe.c. The size is about 400 and something... It's
>>>> strange.....
>>>>
>>>> Is it the memory leak in the PUA module?
>>>>
>>>>
>>>>
>>>> I also try to increase the pkg_memory from 4MB default to 16MB, but it
>>>> doesn't help.
>>>>
>>>>
>>>> Any Idea?
>>>>
>>>> Thanks in advanced
>>>>
>>>> Laura
>>>>
>>>> _______________________________________________
>>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>>> sr-users at lists.sip-router.org
>>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>>>
>>>>
>>>> --
>>>> Daniel-Constantin Mierla -- http://www.asipto.com
>>>> http://linkedin.com/in/miconda -- http://twitter.com/miconda
>>> _______________________________________________
>>> SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>> sr-users at lists.sip-router.org
>>> http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>
>> --
>> Daniel-Constantin Mierla -- http://www.asipto.com
>> http://linkedin.com/in/miconda -- http://twitter.com/miconda
>>

-- 
Daniel-Constantin Mierla -- http://www.asipto.com
http://linkedin.com/in/miconda -- http://twitter.com/miconda




More information about the sr-users mailing list