[Kamailio-Devel] OpenSer 1.3.0 crash

[Henning Westerholt]

On Wednesday 17 September 2008, Robin Vleij wrote:
> [..]

Hi Robin.

> > The h_addr_list should normally contain a list of hosts, it seems that
> > this variable gets somehow corrupt. You could investigate in previous
> > frames why this happens, just change with e.g. "f 1" [1] to the first
> > frame, and examine the variables there, and so on. You can also print the
> > actual source code of the function with "list" [2].
>
> Right. Interesting, I've been reading the pages you linked about gdb a
> bit and have been throwing commands at the prompt now. I see lots of
> things, but since I'm not a developer I don't get much wiser.
> I understand so far, I think. The problem lies in the following function:
> [..]
> and print p->host gives:
>
> $25 = {h_name = 0x6e05e8 "sip-corporate2.tele2.se", h_aliases =
> 0x6aee60, h_addrtype = 2, h_length = 4, h_addr_list = 0x38}
>
> There's the h_addr_list and the invalid pointer there.
>
> One step up again I find the following code:
>
> 217             /* ACKs do not establish a transaction and are fwd-ed
> statelessly */
> 218             if ( p_msg->REQ_METHOD==METHOD_ACK) {
> 219                     LM_DBG("forwarding ACK\n");
> 220                     /* send it out */
> 221                     if (proxy==0) {
> 222                             uri = GET_RURI(p_msg);
> 223                             proxy=uri2proxy(GET_NEXT_HOP(p_msg),
> PROTO_NONE);
> 224                             if (proxy==0) {
> 225                                             ret=E_BAD_ADDRESS;
> 226                                             goto done;
> 227                             }
> 228                             ret=forward_request( p_msg , proxy);
> 229                             if (ret>=0) ret=1;
> 230                             free_proxy( proxy );

The proxy is probably assigned from the uri2proxy function (as aparently 
w_t_relay is called without a proxy parameter). uri2proxy create a new proxy 
with mk_proxy, would be interesting if the invalid h_addr_list is created 
there. But a strange thing is that in the original bt, the t_relay_to 
functions has a proxy parameter, and the uri2proxy/ mk_proxy also not show up 
in the bt at all. This don't fit together, perhaps something wrongly 
displayed from the gdb.

#2  0x00002adec6dc01ea in t_relay_to (p_msg=0x6de4a0, proxy=0x6e3fc0,
flags=<value optimized out>) at t_funcs.c:230
#3  0x00002adec6dd1fcd in w_t_relay (p_msg=0x6de4a0, proxy=0x0,
flags=0x0) at tm.c:929

> Not sure how this is supposed to work and if I'm at all on the right
> way. I stop here, lost it. :)

Yes, its a little bit difficult to follow. :-)

> Now we get to core2 that we got after a normal invite to a test system.
> This core was related to the following bt:
>
> #0  free_lump_list (l=0x636d20) at data_lump.c:412
> #1  0x000000000048ed02 in free_sip_msg (msg=0x6df7b8) at
> parser/msg_parser.c:661
> #2  0x000000000044b4b9 in receive_msg (
>     buf=0x625ca0 "INVITE sip:blah at domain;transport=UDP SIP/2.0\r\nFrom:
> <sip:blah2 at test_server:5060>;tag=pstn20080913184322\r\nTo:
> <sip:blah2 at domain:5060>\r\nCall-ID: pstn20080913184322-1-1"..., len=728,
> rcv_info=0x7fffee85b310) at receive.c:206
> #3  0x0000000000488154 in udp_rcv_loop () at udp_server.c:438
> #4  0x0000000000425081 in main (argc=9, argv=0x7fffee85b518) at main.c:834
>
> Here I go to f 2 and the code there is:
> [..]
> print msg->add_rm gives $3 = (struct lump *) 0x707d98.
>
> Looks all OK to me, but there is something that goes terribly wrong in
> this free_lump_list. Sounds like some memory problem or so, why on earth
> would such a basic thing go wrong?

Could be that a previous operation invalidate some memory, which causes the 
crash after some time.. Hard to tell.

> The only thing different in the config there was that I was fiddling
> with the avp_fr_timer_avp values, which I reverted later on. It gave
> some "invalid type assignment" error, but that seems unrelated to this.
>
> Any ideas in which direction I have to search further now? I'd like to
> upgrade to the latest 1.3, but I can't do that now.

You could try to investigate the first case a little bit more. Upgrading to 
1.3.3/ the 1.3 stable branch would be definitly a good thing.

Cheers,

Henning