[sr-dev] Compile modules with debug symbols
Carlos Ruiz Díaz
carlos.ruizdiaz at gmail.com
Tue Sep 11 05:27:59 CEST 2012
What I meant to say is, why is rounded to 28 instead of 32 for values that
are always equal to 27 in length?
On Mon, Sep 10, 2012 at 11:22 PM, Carlos Ruiz Díaz <
carlos.ruizdiaz at gmail.com> wrote:
> I took a look on mem/q_malloc.c and I saw:
>
> /*size must be a multiple of 8*/
> size=ROUNDUP(size);
>
> that pretty much answered most of my questions regarding the size of the
> memory chunks, but unfortunately not the reason of the crashing after 40k
> calls. MEMDBG doesn't report leaks on my module after finalizing Kamailio.
>
> I don't see why the size should be changed since the values that I'm
> passing are always equal to 27 in length. Why is this?
>
> Regards.
>
> Carlos.
>
>
> On Mon, Sep 10, 2012 at 8:24 PM, Carlos Ruiz Díaz <
> carlos.ruizdiaz at gmail.com> wrote:
>
>> Thanks for your help Daniel.
>>
>> After trying memory dumping with sercmd and killall, I ended up
>> using pkg_sums() before and after every block of dynamic memory allocation.
>> What I found out (and scared me) is that my module finishes with no leak
>> according to the memory summary but, somehow, the memory consumption grows
>> in, what appears to be, a linear basis.
>>
>> Below the outputs per stage after the entry point function is called:
>>
>> *Stage 1:*
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 10 size= 256 bytes from cnx3a: cnx3a_parser.c: parse_body(122)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 40 bytes from cnx3a: cnx3a_parser.c: parse_header(170)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 40 bytes from cnx3a: cnx3a_mod.c: cnx3a_invite(1032)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 10 size= 88 bytes from cnx3a: cnx3a_parser.c: parse_body(88)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 5000 bytes from cnx3a: cnx3a_mod.c: cnx3a_invite(972)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 5000 bytes from cnx3a: cnx3a_mod.c: cnx3a_invite(971)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 10 size= 92 bytes from cnx3a: cnx3a_parser.c: parse_header(182)
>>
>> *In the final stage, the module is supposed to store a SIPURI and
>> return. That SIPURI will later be consulted by the script using a PV,
>> reason why, stage 2 ends with 28 bytes of memory usage.*
>>
>> pkg_str_dup() is called.
>>
>> *Stage2:*
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 28 bytes from cnx3a: /usr/include/kamailio/parser/../ut.h:
>> pkg_str_dup(720)
>>
>> *Afterwards, the module returns with 1.*
>> *
>> *
>> Later when the script tries to write to a PV, the function in charge is
>> called and if the value is assigned (str.s != NULL) it frees the internal
>> variable first and then does another pkg_str_dup().
>>
>> *1st assignment. Before trying to free: *
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 28 bytes from cnx3a: /usr/include/kamailio/parser/../ut.h:
>> pkg_str_dup(720)
>>
>> *1st assignment. **After trying to free:*
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 28 bytes from cnx3a: /usr/include/kamailio/parser/../ut.h:
>> pkg_str_dup(720)
>>
>> *Same output since the var is null*
>>
>> *1st assignment. **After pkg_str_dup():*
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: <core>
>> [mem/q_malloc.c:369]: qm_malloc(0xb71fd008, 27) called from cnx3a:
>> /usr/include/kamailio/parser/../ut.h: pkg_str_dup(720)
>> Sep 10 19:58:29 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 2 size= 56 bytes from cnx3a: /usr/include/kamailio/parser/../ut.h:
>> pkg_str_dup(720)
>>
>> -------------------------------------------------------------
>>
>> *2nd assignment. B**efore trying to free:*
>> Sep 10 19:58:33 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 2 size= 56 bytes from cnx3a: /usr/include/kamailio/parser/../ut.h:
>> pkg_str_dup(720)
>>
>> *2nd assignment. **After trying to free**: *
>> Sep 10 19:58:33 carlosrdcnx-laptop kamailio: NOTICE: <core>
>> [mem/q_malloc.c:437]: qm_free(0xb71fd008, 0xb7275dec), called from cnx3a:
>> cnx3a_mod.c: pv_set_routes(2041)
>>
>> Sep 10 19:58:33 carlosrdcnx-laptop kamailio: NOTICE: <core>
>> [mem/q_malloc.c:461]: qm_free: freeing frag. 0xb7275dd4 alloc'ed from
>> cnx3a: /usr/include/kamailio/parser/../ut.h: pkg_str_dup(720)
>>
>> Sep 10 19:58:33 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 1 size= 28 bytes from cnx3a: /usr/include/kamailio/parser/../ut.h:
>> pkg_str_dup(720)
>>
>> *free() worked since var != NULL. We ended up with 28 bytes, which is
>> correct.*
>>
>> *2nd assignment. **After pkg_str_dup():*
>> *
>> *
>> Sep 10 19:58:33 carlosrdcnx-laptop kamailio: NOTICE: <core>
>> [mem/q_malloc.c:369]: qm_malloc(0xb71fd008, 27) called from cnx3a:
>> /usr/include/kamailio/parser/../ut.h: pkg_str_dup(720)
>> Sep 10 19:58:33 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: count=
>> 2 size= * 56* bytes from cnx3a:
>> /usr/include/kamailio/parser/../ut.h: pkg_str_dup(720)
>>
>> 56 is correct.
>>
>> Now, when the whole circuit is repeated for the second time, in *Stage2* happens
>> the following:
>>
>> Sep 10 19:58:39 carlosrdcnx-laptop kamailio: NOTICE: <core>
>> [mem/q_malloc.c:369]: qm_malloc(0xb71fd008, *27*) called from cnx3a:
>> /usr/include/kamailio/parser/../ut.h: pkg_str_dup(720)
>>
>> Sep 10 19:58:39 carlosrdcnx-laptop kamailio: NOTICE: qm_sums: *count=
>> 1* size= *32 *bytes from cnx3a:
>> /usr/include/kamailio/parser/../ut.h: pkg_str_dup(720)
>>
>> -------------------------------------------------------------
>>
>> Why pkg_malloc always gives 28 bytes instead of the 27 requested? Why the
>> returned size is equal to 32 instead of just 27 in the last example? Can
>> this be a memory fragmentation problem?
>>
>> This happened after a few calls. Can this be the cause for a crash after
>> running 40k calls?
>>
>> I'm using kamailio 3.2.3.
>>
>> Regards and thanks in advance.
>>
>> Carlos
>>
>> On Mon, Sep 10, 2012 at 12:18 PM, Daniel-Constantin Mierla <
>> miconda at gmail.com> wrote:
>>
>>> Hello,
>>>
>>>
>>> On 9/10/12 2:33 PM, Carlos Ruiz Díaz wrote:
>>>
>>> Hi Daniel,
>>>
>>> no, it's just a plain make && make install.
>>>
>>> I spent a few hours on debugging and I figured out that it was an off
>>> by one error on a memcpy() on certain exceptional conditions. I fixed that
>>> but then I found a memory leak that's causing the crash after 40k calls but
>>> I'm having a hard time interpreting the dump provided after sending the
>>> SIGUSR1 to the process. if I'm not mistaken, there are two pkg_str_dup()
>>> leaks per call but I'm still searching where I'm missing the pkg_free().
>>>
>>> Is there a way to know where in the code the pkg_str_dup was called?
>>>
>>> I tried with valgrind too, but after the macro expansion of the memory
>>> management functions, is very hard to find where it was called.
>>>
>>>
>>> here are some hints about troubleshooting memory:
>>> http://www.asipto.com/pub/kamailio-devel-guide/#c04troubleshooting
>>>
>>> if you set memdbg and memlog lower than debug global parameter, then you
>>> get all memory operations in the syslog (be sure you compile with
>>> MEMDBG=1). Also look at mem_summary:
>>> http://www.kamailio.org/wiki/cookbooks/3.3.x/core#mem_summary
>>>
>>> and cfgutils module readme for some functions that can be used in config
>>> to dump the memory usage.
>>>
>>> Cheers,
>>> Daniel
>>>
>>>
>>>
>>>
>>> Regards.
>>>
>>> Carlos.
>>>
>>>
>>>
>>> On Mon, Sep 10, 2012 at 4:02 AM, Daniel-Constantin Mierla <
>>> miconda at gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> yes, by default is with debug symbols on. Do you provide special flags
>>>> or parameters to the make command?
>>>>
>>>> Cheers,
>>>> Daniel
>>>>
>>>>
>>>> On 9/8/12 9:39 PM, Carlos Ruiz Díaz wrote:
>>>>
>>>> Hello list,
>>>>
>>>> I'm trying to debug my module after it crashed by SIGSEGV, but the
>>>> problem is that I'm unable to see the relevant data in the core file that's
>>>> being dumped after the crash.
>>>>
>>>> For example, if I do *bt full, *I get:
>>>>
>>>> 0 0x0ef0f494 in append_string () from
>>>> /opt/install/kamailio-3.2.3/lib/kamailio/modules_k/cnx3a.so
>>>> No symbol table info available.
>>>> #1 0x0eef9658 in build_message () from
>>>> /opt/install/kamailio-3.2.3/lib/kamailio/modules_k/cnx3a.so
>>>> No symbol table info available.
>>>> Backtrace stopped: frame did not save the PC
>>>>
>>>> Only the functions that were called but no parameter values.
>>>>
>>>> AFAIK, if I compile kamailio from sources, it is by default compiled
>>>> with the -g option, so, it looks like that I'm missing something else.
>>>>
>>>> Regards.
>>>>
>>>> Carlos.
>>>>
>>>>
>>>> _______________________________________________
>>>> sr-dev mailing listsr-dev at lists.sip-router.orghttp://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
>>>>
>>>>
>>>> --
>>>> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>>>> Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat
>>>> Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
>>>>
>>>>
>>>
>>> --
>>> Daniel-Constantin Mierla - http://www.asipto.comhttp://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
>>> Kamailio Advanced Training, Berlin, Nov 5-8, 2012 - http://asipto.com/u/kat
>>> Kamailio Advanced Training, Miami, USA, Nov 12-14, 2012 - http://asipto.com/u/katu
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-dev/attachments/20120910/d25942ba/attachment.htm>
More information about the sr-dev
mailing list