<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hello,<br>
<br>
On 1/28/10 3:34 PM, Aymeric Moizard wrote:
<blockquote cite="mid:Pine.LNX.4.63.1001281519461.10244@sip2"
type="cite"><br>
some other answer below:
<br>
<br>
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
<br>
<br>
<blockquote type="cite">I am cc-ing sr-dev, since tcp code is from
ser and Andrei may have more insights...
<br>
<br>
<br>
On 1/28/10 2:41 PM, Aymeric Moizard wrote:
<br>
<blockquote type="cite"><br>
<br>
On Thu, 28 Jan 2010, Henning Westerholt wrote:
<br>
<br>
<blockquote type="cite">On Thursday 28 January 2010, Aymeric
Moizard wrote:
<br>
<blockquote type="cite">here is the backtrace I have.
unfortunatly without debug symbol!
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br>
<br>
can you recompile with debug symbols? Do you have it installed from
package or sources? It will give more hints about the place in the
function...<br>
<br>
I will try to reproduce, but now I do not have the proper environment
for testing...<br>
<br>
Thanks,<br>
Daniel<br>
<br>
<br>
<br>
<br>
<blockquote cite="mid:Pine.LNX.4.63.1001281519461.10244@sip2"
type="cite">
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">
<blockquote type="cite">I found the same for many of the
kamailio process. "sched_yield"
<br>
is pending for ever. My system is a debian/etch.
<br>
<br>
#0 0xffffe424 in __kernel_vsyscall ()
<br>
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
<br>
#2 0x080a93fd in tcp_send ()
<br>
#3 0xb7975679 in send_pr_buffer () from
/usr/lib/kamailio/modules/tm.so
<br>
#4 0xb79789ac in t_forward_nonack () from
/usr/lib/kamailio/modules/tm.so
<br>
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
<br>
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
<br>
#7 0x081cf810 in mem_pool ()
<br>
#8 0x00000000 in ?? ()
<br>
<br>
I guess most t_relay operation towards my "mobipouce.com" domain
<br>
with one IP being down breaks each kamailio process one after the
<br>
other... I'm not sure every such t_relay operation is always breaking
<br>
exactly one thread each time.
<br>
<br>
I went through the lock/unlock of tcp_main.c but it seems every
<br>
lock has an unlock at least...
<br>
</blockquote>
<br>
Hi Aymeric,
<br>
<br>
i remember that we observed this "sched_yield" problems on one old 0.9
system
<br>
after some time (like weeks or month). We did not found the solution in
this
<br>
case, after a restart it was gone again..
<br>
<br>
You mentioned in an earlier mail that you see this related to UDP
traffic, but
<br>
in the log file and also in your investigations you think its related
to TPC?
<br>
</blockquote>
<br>
This is the exact case:
<br>
1-> SUBSCRIBE sent to/received by over UDP to kamailio.
<br>
2-> kamailio does a SRV record lookup for "mobipouce.com"
<br>
3-> kamailio try sip2.mobipouce.com (91.199.234.47) over TCP first
<br>
4-> connection failed with logs:
<br>
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
<br>
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
<br>
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
<br>
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_send:
connect failed
<br>
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:msg_send:
tcp_send failed
<br>
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:tm:t_forward_nonack: sending request failed
<br>
5-> I guess kamailio is supposed to try other SRV record value:
<br>
sip2.mobipouce.com (91.199.234.46) but it doesn't
<br>
<br>
Thus, I'm guessing the issue is related to SRV record with failover OR
just tcp failure. Not related to UDP at all.
<br>
</blockquote>
<br>
so TCP connect failed, the tcp worker returned as it prints the message
and, to be sure I got it right, the UDP worker (the one that received)
got blocked?
<br>
</blockquote>
<br>
1-> TCP connect failed
<br>
2-> second SRV is used: TCP connect succeed, but lock in tcp_send
<br>
<br>
That's what I understand.
<br>
<br>
I have tested a TCP connection to my server: It seems to be still
<br>
working.
<br>
<br>
<blockquote type="cite">
<blockquote type="cite">It's definitly possible to reproduce the
issue now!
<br>
<br>
I guess anyone can try your version of kamailio and t_relay message
<br>
to "mobipouce.com" and you'll fall in that case! Sending plenty of
<br>
those messages will finally lock all kamailio process.
<br>
</blockquote>
<br>
All? tcp and udp?
<br>
</blockquote>
<br>
Only udp!
<br>
Aymeric
<br>
<br>
<blockquote type="cite">Cheers,
<br>
Daniel
<br>
<br>
<blockquote type="cite"><br>
Regards,
<br>
Aymeric MOIZARD / ANTISIP
<br>
amsip - <a class="moz-txt-link-freetext" href="http://www.antisip.com">http://www.antisip.com</a>
<br>
osip2 - <a class="moz-txt-link-freetext" href="http://www.osip.org">http://www.osip.org</a>
<br>
eXosip2 - <a class="moz-txt-link-freetext" href="http://savannah.nongnu.org/projects/exosip/">http://savannah.nongnu.org/projects/exosip/</a>
<br>
<br>
<br>
<blockquote type="cite">Regards,
<br>
<br>
Henning
<br>
<br>
Viele Grüße,
<br>
<br>
Henning
<br>
<br>
</blockquote>
<br>
_______________________________________________
<br>
Kamailio (OpenSER) - Users mailing list
<br>
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.kamailio.org">Users@lists.kamailio.org</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.kamailio.org/cgi-bin/mailman/listinfo/users">http://lists.kamailio.org/cgi-bin/mailman/listinfo/users</a>
<br>
<a class="moz-txt-link-freetext" href="http://lists.openser-project.org/cgi-bin/mailman/listinfo/users">http://lists.openser-project.org/cgi-bin/mailman/listinfo/users</a>
<br>
</blockquote>
<br>
-- <br>
Daniel-Constantin Mierla
<br>
* <a class="moz-txt-link-freetext" href="http://www.asipto.com/">http://www.asipto.com/</a>
<br>
<br>
<br>
</blockquote>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
Kamailio (OpenSER) - Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.kamailio.org">Users@lists.kamailio.org</a>
<a class="moz-txt-link-freetext" href="http://lists.kamailio.org/cgi-bin/mailman/listinfo/users">http://lists.kamailio.org/cgi-bin/mailman/listinfo/users</a>
<a class="moz-txt-link-freetext" href="http://lists.openser-project.org/cgi-bin/mailman/listinfo/users">http://lists.openser-project.org/cgi-bin/mailman/listinfo/users</a></pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Daniel-Constantin Mierla
* <a class="moz-txt-link-freetext" href="http://www.asipto.com/">http://www.asipto.com/</a>
</pre>
</body>
</html>