15 Jan
2010
15 Jan
'10
10:33 p.m.
I have an issue where my sip endpoint is sending ISUP information in the invite. IE:
sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060 and when I run the enum query I am
getting:
0(24410) enum_query(): uri user is not an E164 number
This is most likely due to the fact that $rU=+17794294010;npdi=yes;cic=+1-0110
Does openser not read the user properly? Is there a way to correct this?
I am running openser 1.2.2 notls
Thanks for any help!
-Eric
_________________________________________________________________
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
http://clk.atdmt.com/GBL/go/196390709/direct/01/
16 Jan
16 Jan
10:51 a.m.
Hi ERic!
The ENUM lookup is done with the complete userpart. In your case you
have to parse the E164 number manually.
You can do this for example with string transformations: e.g.
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind…
Then use enum_pv_function() with the pseudo variable (pv) holding the
parsed number.
http://kamailio.org/docs/modules/1.5.x/enum#id2467740
regards
klaus
Eric Hiller wrote:
I have an issue where my sip endpoint is sending ISUP
information in the
invite. IE: sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060 and when I
run the enum query I am getting:
0(24410) enum_query(): uri user is not an E164 number
This is most likely due to the fact that
$rU=+17794294010;npdi=yes;cic=+1-0110
Does openser not read the user properly? Is there a way to correct this?
I am running openser 1.2.2 notls
Thanks for any help!
-Eric
------------------------------------------------------------------------
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up
now. <http://clk.atdmt.com/GBL/go/196390709/direct/01/>
------------------------------------------------------------------------
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
18 Jan
18 Jan
3:51 p.m.
On Saturday 16 January 2010, Klaus Darilion wrote:
The ENUM lookup is done with the complete userpart. In
your case you
have to parse the E164 number manually.
You can do this for example with string transformations: e.g.
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind
ex_separator
Then use enum_pv_function() with the pseudo variable (pv) holding the
parsed number.
http://kamailio.org/docs/modules/1.5.x/enum#id2467740
> I have an issue where my sip endpoint is sending ISUP information in the
> invite. IE: sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060 and when I
> run the enum query I am getting:
> 0(24410) enum_query(): uri user is not an E164 number
>
> This is most likely due to the fact that
> $rU=+17794294010;npdi=yes;cic=+1-0110
>
> Does openser not read the user properly? Is there a way to correct this?
>
> I am running openser 1.2.2 notls
Hi Eric,
the functionality that Klaus mentioned (string transformations) are not
available in the version you use. Generelly i'd suggest to update to at least
kamailio 1.5.3 or kamailio 3.0 which was released a few days ago, if you can.
Cheers,
Henning
5:47 p.m.
Hello,
On 1/18/10 3:51 PM, Henning Westerholt wrote:
On Saturday 16 January 2010, Klaus Darilion wrote:
I recommend as well an upgrade, however, for that version should try a
solution with subst_user() from textops module.
Cheers,
Daniel
--
Daniel-Constantin Mierla
* http://www.asipto.com/
The ENUM lookup is done with the complete
userpart. In your case you
have to parse the E164 number manually.
You can do this for example with string transformations: e.g.
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind
ex_separator
Then use enum_pv_function() with the pseudo variable (pv) holding the
parsed number.
http://kamailio.org/docs/modules/1.5.x/enum#id2467740
> I have an issue where my sip endpoint is sending ISUP information in the
> invite. IE: sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060 and when I
> run the enum query I am getting:
> 0(24410) enum_query(): uri user is not an E164 number
>
> This is most likely due to the fact that
> $rU=+17794294010;npdi=yes;cic=+1-0110
>
> Does openser not read the user properly? Is there a way to correct this?
>
> I am running openser 1.2.2 notls
>
Hi Eric,
the functionality that Klaus mentioned (string transformations) are not
available in the version you use. Generelly i'd suggest to update to at least
kamailio 1.5.3 or kamailio 3.0 which was released a few days ago, if you can.
27 Jan
27 Jan
7:50 p.m.
Klaus,
Thanks for the help, I am working on getting 1.5
installed now as this functionality isn't available in 1.2 it would
appear. In 1.5 I am getting an error for my code:
$ruri.domain = "216.82.224.202";
forward();
Which worked fine in 1.2, but does not in 1.5. What would be the equivalent config for
1.5?
For the enum side I would do:
enum_pv_query("{s.select,$ruri,;}");
If I did textops then it would be:
subst_user('/;(.*)$//');
Correct?
Thanks again!
-Eric
Date: Sat, 16 Jan 2010 10:51:59 +0100
From: klaus.mailinglists(a)pernau.at
To: mrraptor98(a)hotmail.com
CC: users(a)lists.kamailio.org
Subject: Re: [Kamailio-Users] isup in invite
Hi ERic!
The ENUM lookup is done with the complete userpart. In your case you
have to parse the E164 number manually.
You can do this for example with string transformations: e.g.
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind…
Then use enum_pv_function() with the pseudo variable (pv) holding the
parsed number.
http://kamailio.org/docs/modules/1.5.x/enum#id2467740
regards
klaus
Eric Hiller wrote:
_________________________________________________________________
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
http://clk.atdmt.com/GBL/go/196390709/direct/01/
I have an issue where my sip endpoint is sending
ISUP information in the
invite. IE: sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060 and when I
run the enum query I am getting:
0(24410) enum_query(): uri user is not an E164 number
This is most likely due to the fact that
$rU=+17794294010;npdi=yes;cic=+1-0110
Does openser not read the user properly? Is there a way to correct this?
I am running openser 1.2.2 notls
Thanks for any help!
-Eric
------------------------------------------------------------------------
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up
now. <http://clk.atdmt.com/GBL/go/196390709/direct/01/>
------------------------------------------------------------------------
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
9:36 p.m.
Hello,
On 1/27/10 7:50 PM, Eric Hiller wrote:
Klaus,
Thanks for the help, I am working on getting 1.5 installed now as this
functionality isn't available in 1.2 it would appear. In 1.5 I am
getting an error for my code:
$ruri.domain = "216.82.224.202";
forward();
Which worked fine in 1.2, but does not in 1.5. What would be the
equivalent config for 1.5?
have you loaded the pv.so module? In 1.5.0 and greater most of
pseudo-variables are implemented in pv module. instead of $ruri.domain
you can use the shorter version $rd.
Cheers,
Daniel
For the enum side I would do:
enum_pv_query("{s.select,$ruri,;}");
If I did textops then it would be:
subst_user('/;(.*)$//');
Correct?
Thanks again!
-Eric
------------------------------------------------------------------------
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up
now. <http://clk.atdmt.com/GBL/go/196390709/direct/01/>
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Date: Sat, 16 Jan 2010 10:51:59 +0100
From: klaus.mailinglists(a)pernau.at
To: mrraptor98(a)hotmail.com
CC: users(a)lists.kamailio.org
Subject: Re: [Kamailio-Users] isup in invite
Hi ERic!
The ENUM lookup is done with the complete userpart. In your case you
have to parse the E164 number manually.
You can do this for example with string transformations: e.g.
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind…
Then use enum_pv_function() with the pseudo variable (pv) holding the
parsed number.
http://kamailio.org/docs/modules/1.5.x/enum#id2467740
regards
klaus
Eric Hiller wrote:
> I have an issue where my sip endpoint is sending ISUP information
in the
> invite. IE:
sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060 and
when I
> run the enum query I am getting:
> 0(24410) enum_query(): uri user is not an E164 number
>
> This is most likely due to the fact that
> $rU=+17794294010;npdi=yes;cic=+1-0110
>
> Does openser not read the user properly? Is there a way to correct
this?
>
> I am running openser 1.2.2 notls
>
> Thanks for any help!
> -Eric
>
>
------------------------------------------------------------------------
> Your E-mail and More On-the-Go. Get Windows
Live Hotmail Free.
Sign up
> now.
<http://clk.atdmt.com/GBL/go/196390709/direct/01/>
>
>
>
------------------------------------------------------------------------
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
28 Jan
28 Jan
6:44 a.m.
Thanks much! Got it all running once I loaded pv.so
Just in case it helps anyone else out; here is my final config:## switch.5+diag# same
script, bug with diagnostic printouts; run with:# /usr/local/sbin/openser -f
/usr/local/etc/openser/openser.cfg.diag## ----------- global configuration parameters
------------------------
debug=3#fork=yes#log_stderror=nochildren=4
# Uncomment these lines to enter debugging
mode#debug=10fork=nolog_stderror=yeslisten=xxx#
port=5060
# ------------------ module loading ----------------------------------
#set module pathmpath="/usr/local/lib/kamailio/modules/"
loadmodule "sl.so"loadmodule "maxfwd.so"loadmodule
"enum.so"loadmodule "pv.so"loadmodule "textops.so"
#for diagnostic purposesloadmodule "mi_fifo.so"loadmodule "xlog.so"
# ----------------- setting module-specific parameters ---------------
#for diagnostic purposesmodparam("mi_fifo", "fifo_name",
"/tmp/openser_fifo")
modparam("enum", "domain_suffix", "xxx")
# ------------------------- request routing logic -------------------
# main routing logic# ------------------route{ # initial checks if
(!mf_process_maxfwd_header("10")) {
sl_send_reply("483","Too Many Hops"); exit; };
if (msg:len >= 2048 ) { sl_send_reply("513",
"Message too big"); exit; };
xlog("valid received to=<$ru> from=<$fu>\n");
# ENUM :: makes uri E164; then checks enum subst_user('/;(.*)$//');
if(enum_query()){ xlog("ENUM to <$ru>\n");
forward(); exit; } # GATEWAY :: forwards all valid
unknowns to the gateway if(is_from_user_enum()){ xlog("SENT TO
GATEWAY\n"); $ruri.domain = "xxx"; forward();
} else { xlog("FROM USER NOT IN ENUM\n"); }}
Date: Wed, 27 Jan 2010 21:36:00 +0100
From: miconda(a)gmail.com
To: mrraptor98(a)hotmail.com
CC: klaus.mailinglists(a)pernau.at; users(a)lists.kamailio.org
Subject: Re: [Kamailio-Users] isup in invite
Hello,
On 1/27/10 7:50 PM, Eric Hiller wrote:
Klaus,
Thanks for the help, I am working on getting 1.5
installed now as this functionality isn't available in 1.2 it would
appear. In 1.5 I am getting an error for my code:
$ruri.domain = "216.82.224.202";
forward();
Which worked fine in 1.2, but does not in 1.5. What would be the
equivalent config for 1.5?
have you loaded the pv.so module? In 1.5.0 and greater most of
pseudo-variables are implemented in pv module. instead of $ruri.domain
you can use the shorter version $rd.
Cheers,
Daniel
For the enum side I would do:
enum_pv_query("{s.select,$ruri,;}");
If I did textops then it would be:
subst_user('/;(.*)$//');
Correct?
Thanks again!
-Eric
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind…
>
------------------------------------------------------------------------
>
>
>
------------------------------------------------------------------------
>
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Date: Sat, 16 Jan 2010 10:51:59 +0100
From: klaus.mailinglists(a)pernau.at
To: mrraptor98(a)hotmail.com
CC: users(a)lists.kamailio.org
Subject: Re: [Kamailio-Users] isup in invite
Hi ERic!
The ENUM lookup is done with the complete userpart. In
your case
you
have to parse the E164 number manually.
You can do this for example with string
transformations: e.g.
Then use enum_pv_function() with the pseudo variable
(pv) holding
the
parsed number.
regards
klaus
Eric Hiller wrote:
> I have an issue where my sip endpoint is sending
ISUP
information in the
> invite. IE:
sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060
and when I
> run the enum query I am getting:
> 0(24410) enum_query(): uri user is not an E164
number
> > This is most likely due to the fact that
> $rU=+17794294010;npdi=yes;cic=+1-0110
> > Does openser not read the user properly? Is there
a way to
correct this?
> > I am running openser 1.2.2 notls
> > Thanks for any help!
> -Eric
> > Your E-mail and More On-the-Go. Get Windows Live
Hotmail
Free. Sign up
> > now. <http://clk.atdmt.com/GBL/go/196390709/direct/01/ > _______________________________________________
> Kamailio (OpenSER) - Users mailing list
> Users(a)lists.kamailio.org
> _______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign
up now.
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
--
Daniel-Constantin Mierla
* http://www.asipto.com/
_________________________________________________________________
Hotmail: Free, trusted and rich email service.
http://clk.atdmt.com/GBL/go/196390708/direct/01/
11:50 a.m.
On 1/28/10 6:44 AM, Eric Hiller wrote:
Thanks much! Got it all running once I loaded pv.so
welcome! You can create a new wiki page explaining what this config
offers and posting it there:
http://www.kamailio.org/dokuwiki/doku.php/cfg-scripts-bank:main-index
It is easy to reference later to others.
Cheers,
Daniel
Just in case it helps anyone else out; here is my final config:
#
# switch.5+diag
# same script, bug with diagnostic printouts; run with:
# /usr/local/sbin/openser -f /usr/local/etc/openser/openser.cfg.diag
#
# ----------- global configuration parameters ------------------------
debug=3
#fork=yes
#log_stderror=no
children=4
# Uncomment these lines to enter debugging mode
#debug=10
fork=no
log_stderror=yes
listen=xxx
#
port=5060
# ------------------ module loading ----------------------------------
#set module path
mpath="/usr/local/lib/kamailio/modules/"
loadmodule "sl.so"
loadmodule "maxfwd.so"
loadmodule "enum.so"
loadmodule "pv.so"
loadmodule "textops.so"
#for diagnostic purposes
loadmodule "mi_fifo.so"
loadmodule "xlog.so"
# ----------------- setting module-specific parameters ---------------
#for diagnostic purposes
modparam("mi_fifo", "fifo_name", "/tmp/openser_fifo")
modparam("enum", "domain_suffix", "xxx")
# ------------------------- request routing logic -------------------
# main routing logic
# ------------------
route{
# initial checks
if (!mf_process_maxfwd_header("10")) {
sl_send_reply("483","Too Many Hops");
exit;
};
if (msg:len >= 2048 ) {
sl_send_reply("513", "Message too big");
exit;
};
xlog("valid received to=<$ru> from=<$fu>\n");
# ENUM :: makes uri E164; then checks enum
subst_user('/;(.*)$//');
if(enum_query()){
xlog("ENUM to <$ru>\n");
forward();
exit;
}
# GATEWAY :: forwards all valid unknowns to the gateway
if(is_from_user_enum()){
xlog("SENT TO GATEWAY\n");
$ruri.domain = "xxx";
forward();
} else {
xlog("FROM USER NOT IN ENUM\n");
}
}
------------------------------------------------------------------------
Date: Wed, 27 Jan 2010 21:36:00 +0100
From: miconda(a)gmail.com
To: mrraptor98(a)hotmail.com
CC: klaus.mailinglists(a)pernau.at; users(a)lists.kamailio.org
Subject: Re: [Kamailio-Users] isup in invite
Hello,
On 1/27/10 7:50 PM, Eric Hiller wrote:
Klaus,
Thanks for the help, I am working on getting 1.5 installed now as
this functionality isn't available in 1.2 it would appear. In 1.5
I am getting an error for my code:
$ruri.domain = "216.82.224.202";
forward();
Which worked fine in 1.2, but does not in 1.5. What would be the
equivalent config for 1.5?
have you loaded the pv.so module? In 1.5.0 and greater most of
pseudo-variables are implemented in pv module. instead of $ruri.domain
you can use the shorter version $rd.
Cheers,
Daniel
For the enum side I would do:
enum_pv_query("{s.select,$ruri,;}");
If I did textops then it would be:
subst_user('/;(.*)$//');
Correct?
Thanks again!
-Eric
------------------------------------------------------------------------
Your E-mail and More On-the-Go. Get Windows Live Hotmail Free.
Sign up now. <http://clk.atdmt.com/GBL/go/196390709/direct/01/>
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org <mailto:Users@lists.kamailio.org>
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
--
Daniel-Constantin Mierla
*http://www.asipto.com/
------------------------------------------------------------------------
Hotmail: Free, trusted and rich email service. Get it now.
<http://clk.atdmt.com/GBL/go/196390708/direct/01/>
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Date: Sat, 16 Jan 2010 10:51:59 +0100
From: klaus.mailinglists(a)pernau.at
<mailto:klaus.mailinglists@pernau.at>
To: mrraptor98(a)hotmail.com
<mailto:mrraptor98@hotmail.com>
CC: users(a)lists.kamailio.org <mailto:users@lists.kamailio.org>
Subject: Re: [Kamailio-Users] isup in invite
Hi ERic!
The ENUM lookup is done with the complete userpart. In your case
you
have to parse the E164 number manually.
You can do this for example with string transformations: e.g.
http://www.kamailio.org/dokuwiki/doku.php/transformations:1.5.x#sselect_ind…
Then use enum_pv_function() with the pseudo variable (pv)
holding the
parsed number.
http://kamailio.org/docs/modules/1.5.x/enum#id2467740
regards
klaus
Eric Hiller wrote:
> I have an issue where my sip endpoint is sending ISUP
information in the
> invite. IE:
sip:+1xxxxxxxxxx;npdi=yes;cic=+1-0110@xxxx:5060
and when I
> run the enum query I am getting:
> 0(24410) enum_query(): uri user is not an E164 number
>
> This is most likely due to the fact that
> $rU=+17794294010;npdi=yes;cic=+1-0110
>
> Does openser not read the user properly? Is there a way to
correct this?
>
> I am running openser 1.2.2 notls
>
> Thanks for any help!
> -Eric
>
>
------------------------------------------------------------------------
> Your E-mail and More On-the-Go. Get Windows
Live Hotmail Free.
Sign up
> now.
<http://clk.atdmt.com/GBL/go/196390709/direct/01/>
>
>
>
------------------------------------------------------------------------
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org <mailto:Users@lists.kamailio.org>
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org <mailto:Users@lists.kamailio.org>
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users 
10:46 a.m.
New subject: [Kamailio-Users] kamailio / deadlock3
Hi again people!
I'm currently having some trouble with my sip.antisip.com server.
Within the previous 2 or 3 days, kamailio sometimes fall into
some kind of dead lock.
I've been checking my logs while the dead lock happen, and it
seems (although I'm not sure with only the logs) that only UDP
support is broken: I can see some TLS and TCP registrations but
do not see the usual udp traffic (keep alive for example)
Any idea?
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
11:18 a.m.
New subject: [Kamailio-Users] kamailio / deadlock3
Got some more info:
The UDP deadlock always seems to happen after a SUBSCRIBE
is sent (in UDP) to mobipouce.com:
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:core:tcp_blocking_connect: poll
error: flags 18
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:core:tcp_blocking_connect: failed
to retrieve SO_ERROR (111) Connection refused
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:core:tcpconn_connect:
tcp_blocking_connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:core:tcp_send: connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:tm:msg_send: tcp_send failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:tm:t_forward_nonack: sending
request failed
This logs happens each time I got a SUSCRIBE being relayed to another
server: mobipouce.com. But the deadlock doesn't appear each time.
mobipouce.com is an existing & running server where I can connect with UDP
and TCP. However, the SRV record returns 2 host where one host is down.
(and I never got reply for the SUBSCRIBE: either if it fall into deadlock
cas or not)
In case I can reproduce what step could I take to get more information
about the issue? Any kmctl command?
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
On Thu, 28 Jan 2010, Aymeric Moizard wrote:
Hi again people!
I'm currently having some trouble with my sip.antisip.com server.
Within the previous 2 or 3 days, kamailio sometimes fall into
some kind of dead lock.
I've been checking my logs while the dead lock happen, and it
seems (although I'm not sure with only the logs) that only UDP
support is broken: I can see some TLS and TCP registrations but
do not see the usual udp traffic (keep alive for example)
Any idea?
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
11:23 a.m.
New subject: [Kamailio-Users] kamailio / deadlock3
Hello,
On 1/28/10 11:18 AM, Aymeric Moizard wrote:
Got some more info:
The UDP deadlock always seems to happen after a SUBSCRIBE
is sent (in UDP) to mobipouce.com:
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_send: connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:tm:msg_send:
tcp_send failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:tm:t_forward_nonack: sending request failed
This logs happens each time I got a SUSCRIBE being relayed to another
server: mobipouce.com. But the deadlock doesn't appear each time.
mobipouce.com is an existing & running server where I can connect with
UDP and TCP. However, the SRV record returns 2 host where one host is
down.
(and I never got reply for the SUBSCRIBE: either if it fall into
deadlock cas or not)
In case I can reproduce what step could I take to get more information
about the issue? Any kmctl command?
is it recovering itself or you have to restart? How much cpu usage you get?
I if one or many eating lot of cpu, then use gdb to attach to the pid of
process using lot of cpu and get the back trace:
gdb /path/to/kamailio pid
Cheers,
Daniel
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
On Thu, 28 Jan 2010, Aymeric Moizard wrote:
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Hi again people!
I'm currently having some trouble with my sip.antisip.com server.
Within the previous 2 or 3 days, kamailio sometimes fall into
some kind of dead lock.
I've been checking my logs while the dead lock happen, and it
seems (although I'm not sure with only the logs) that only UDP
support is broken: I can see some TLS and TCP registrations but
do not see the usual udp traffic (keep alive for example)
Any idea?
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
12:19 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
Hello,
On 1/28/10 11:18 AM, Aymeric Moizard wrote:
Not noticed any CPU issue: I'll check exactly next time. (but traffic is
growing up as kamailio don't answer any more.
Got some more info:
The UDP deadlock always seems to happen after a SUBSCRIBE
is sent (in UDP) to mobipouce.com:
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:core:tcp_send:
connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:tm:msg_send:
tcp_send failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:tm:t_forward_nonack: sending request failed
This logs happens each time I got a SUSCRIBE being relayed to another
server: mobipouce.com. But the deadlock doesn't appear each time.
mobipouce.com is an existing & running server where I can connect with UDP
and TCP. However, the SRV record returns 2 host where one host is down.
(and I never got reply for the SUBSCRIBE: either if it fall into deadlock
cas or not)
In case I can reproduce what step could I take to get more information
about the issue? Any kmctl command?
is it recovering itself or you have to restart? How much cpu usage you get? I if one or many eating lot of cpu, then use gdb to
attach to the pid of
process using lot of cpu and get the back trace:
gdb /path/to/kamailio pid
I think I can reproduce now. So I'll take a try.
It's definitly after the SRV check: the server choose the
sip2.mobipouce.com server where no sip server is running
and failed to connect. Then the network capture shows that
kamailio is still sending a few SIP packets (like NOTIFY)
but no SIP answers is coming out of kamailio.
I will do more testing, but I guess one can reproduce
by relaying to mobipouce.com!
Aymeric
Cheers,
Daniel
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
On Thu, 28 Jan 2010, Aymeric Moizard wrote:
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Hi again people!
I'm currently having some trouble with my sip.antisip.com server.
Within the previous 2 or 3 days, kamailio sometimes fall into
some kind of dead lock.
I've been checking my logs while the dead lock happen, and it
seems (although I'm not sure with only the logs) that only UDP
support is broken: I can see some TLS and TCP registrations but
do not see the usual udp traffic (keep alive for example)
Any idea?
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
1:59 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
Hi again,
here is the backtrace I have. unfortunatly without debug symbol!
I found the same for many of the kamailio process. "sched_yield"
is pending for ever. My system is a debian/etch.
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send ()
#3 0xb7975679 in send_pr_buffer () from /usr/lib/kamailio/modules/tm.so
#4 0xb79789ac in t_forward_nonack () from /usr/lib/kamailio/modules/tm.so
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
#7 0x081cf810 in mem_pool ()
#8 0x00000000 in ?? ()
I guess most t_relay operation towards my "mobipouce.com" domain
with one IP being down breaks each kamailio process one after the
other... I'm not sure every such t_relay operation is always breaking
exactly one thread each time.
I went through the lock/unlock of tcp_main.c but it seems every
lock has an unlock at least...
Tks,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
On Thu, 28 Jan 2010, Aymeric Moizard wrote:
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
Hello,
On 1/28/10 11:18 AM, Aymeric Moizard wrote:
Not noticed any CPU issue: I'll check exactly next time. (but traffic is
growing up as kamailio don't answer any more.
Got some more info:
The UDP deadlock always seems to happen after a SUBSCRIBE
is sent (in UDP) to mobipouce.com:
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:core:tcp_send:
connect failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]: ERROR:tm:msg_send:
tcp_send failed
Jan 28 11:00:40 ns26829 /usr/sbin/kamailio[13363]:
ERROR:tm:t_forward_nonack: sending request failed
This logs happens each time I got a SUSCRIBE being relayed to another
server: mobipouce.com. But the deadlock doesn't appear each time.
mobipouce.com is an existing & running server where I can connect with UDP
and TCP. However, the SRV record returns 2 host where one host is down.
(and I never got reply for the SUBSCRIBE: either if it fall into deadlock
cas or not)
In case I can reproduce what step could I take to get more information
about the issue? Any kmctl command?
is it recovering itself or you have to restart? How much cpu usage you get? I if one or many eating lot of cpu, then use gdb
to attach to the pid of
process using lot of cpu and get the back trace:
gdb /path/to/kamailio pid
I think I can reproduce now. So I'll take a try.
It's definitly after the SRV check: the server choose the
sip2.mobipouce.com server where no sip server is running
and failed to connect. Then the network capture shows that
kamailio is still sending a few SIP packets (like NOTIFY)
but no SIP answers is coming out of kamailio.
I will do more testing, but I guess one can reproduce
by relaying to mobipouce.com!
Aymeric
Cheers,
Daniel
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
On Thu, 28 Jan 2010, Aymeric Moizard wrote:
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Hi again people!
I'm currently having some trouble with my sip.antisip.com server.
Within the previous 2 or 3 days, kamailio sometimes fall into
some kind of dead lock.
I've been checking my logs while the dead lock happen, and it
seems (although I'm not sure with only the logs) that only UDP
support is broken: I can see some TLS and TCP registrations but
do not see the usual udp traffic (keep alive for example)
Any idea?
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
2:06 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
On Thursday 28 January 2010, Aymeric Moizard wrote:
here is the backtrace I have. unfortunatly without
debug symbol!
I found the same for many of the kamailio process. "sched_yield"
is pending for ever. My system is a debian/etch.
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send ()
#3 0xb7975679 in send_pr_buffer () from /usr/lib/kamailio/modules/tm.so
#4 0xb79789ac in t_forward_nonack () from /usr/lib/kamailio/modules/tm.so
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
#7 0x081cf810 in mem_pool ()
#8 0x00000000 in ?? ()
I guess most t_relay operation towards my "mobipouce.com" domain
with one IP being down breaks each kamailio process one after the
other... I'm not sure every such t_relay operation is always breaking
exactly one thread each time.
I went through the lock/unlock of tcp_main.c but it seems every
lock has an unlock at least...
Hi Aymeric,
i remember that we observed this "sched_yield" problems on one old 0.9 system
after some time (like weeks or month). We did not found the solution in this
case, after a restart it was gone again..
You mentioned in an earlier mail that you see this related to UDP traffic, but
in the log file and also in your investigations you think its related to TPC?
Regards,
Henning
Viele Grüße,
Henning
2:41 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
On Thu, 28 Jan 2010, Henning Westerholt wrote:
On Thursday 28 January 2010, Aymeric Moizard wrote:
This is the exact case:
1-> SUBSCRIBE sent to/received by over UDP to kamailio.
2-> kamailio does a SRV record lookup for "mobipouce.com"
3-> kamailio try sip2.mobipouce.com (91.199.234.47) over TCP first
4-> connection failed with logs:
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_blocking_connect: poll
error: flags 18
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_blocking_connect: failed
to retrieve SO_ERROR (111) Connection refused
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcpconn_connect:
tcp_blocking_connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_send: connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:msg_send: tcp_send failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:t_forward_nonack: sending
request failed
5-> I guess kamailio is supposed to try other SRV record value:
sip2.mobipouce.com (91.199.234.46) but it doesn't
Thus, I'm guessing the issue is related to SRV record with failover OR
just tcp failure. Not related to UDP at all.
It's definitly possible to reproduce the issue now!
I guess anyone can try your version of kamailio and t_relay message
to "mobipouce.com" and you'll fall in that case! Sending plenty of
those messages will finally lock all kamailio process.
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
here is the backtrace I have. unfortunatly
without debug symbol!
I found the same for many of the kamailio process. "sched_yield"
is pending for ever. My system is a debian/etch.
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send ()
#3 0xb7975679 in send_pr_buffer () from /usr/lib/kamailio/modules/tm.so
#4 0xb79789ac in t_forward_nonack () from /usr/lib/kamailio/modules/tm.so
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
#7 0x081cf810 in mem_pool ()
#8 0x00000000 in ?? ()
I guess most t_relay operation towards my "mobipouce.com" domain
with one IP being down breaks each kamailio process one after the
other... I'm not sure every such t_relay operation is always breaking
exactly one thread each time.
I went through the lock/unlock of tcp_main.c but it seems every
lock has an unlock at least...
Hi Aymeric,
i remember that we observed this "sched_yield" problems on one old 0.9 system
after some time (like weeks or month). We did not found the solution in this
case, after a restart it was gone again..
You mentioned in an earlier mail that you see this related to UDP traffic, but
in the log file and also in your investigations you think its related to TPC? Regards,
Henning
Viele Grüße,
Henning
2:56 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
I am cc-ing sr-dev, since tcp code is from ser and Andrei may have more
insights...
On 1/28/10 2:41 PM, Aymeric Moizard wrote:
On Thu, 28 Jan 2010, Henning Westerholt wrote:
so TCP connect failed, the tcp worker returned as it prints the message
and, to be sure I got it right, the UDP worker (the one that received)
got blocked?
On Thursday 28 January 2010, Aymeric Moizard
wrote:
This is the exact case:
1-> SUBSCRIBE sent to/received by over UDP to kamailio.
2-> kamailio does a SRV record lookup for "mobipouce.com"
3-> kamailio try sip2.mobipouce.com (91.199.234.47) over TCP first
4-> connection failed with logs:
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_send:
connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:msg_send:
tcp_send failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:tm:t_forward_nonack: sending request failed
5-> I guess kamailio is supposed to try other SRV record value:
sip2.mobipouce.com (91.199.234.46) but it doesn't
Thus, I'm guessing the issue is related to SRV record with failover OR
just tcp failure. Not related to UDP at all. here is the backtrace I have. unfortunatly
without debug symbol!
I found the same for many of the kamailio process. "sched_yield"
is pending for ever. My system is a debian/etch.
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send ()
#3 0xb7975679 in send_pr_buffer () from
/usr/lib/kamailio/modules/tm.so
#4 0xb79789ac in t_forward_nonack () from
/usr/lib/kamailio/modules/tm.so
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
#7 0x081cf810 in mem_pool ()
#8 0x00000000 in ?? ()
I guess most t_relay operation towards my "mobipouce.com" domain
with one IP being down breaks each kamailio process one after the
other... I'm not sure every such t_relay operation is always breaking
exactly one thread each time.
I went through the lock/unlock of tcp_main.c but it seems every
lock has an unlock at least...
Hi Aymeric,
i remember that we observed this "sched_yield" problems on one old
0.9 system
after some time (like weeks or month). We did not found the solution
in this
case, after a restart it was gone again..
You mentioned in an earlier mail that you see this related to UDP
traffic, but
in the log file and also in your investigations you think its related
to TPC?
It's definitly possible to reproduce the issue now!
I guess anyone can try your version of kamailio and t_relay message
to "mobipouce.com" and you'll fall in that case! Sending plenty of
those messages will finally lock all kamailio process.
All? tcp and udp?
Cheers,
Daniel
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Regards,
Henning
Viele Grüße,
Henning
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
3:34 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
some other answer below:
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
Only udp!
Aymeric
I am cc-ing sr-dev, since tcp code is from ser and
Andrei may have more
insights...
On 1/28/10 2:41 PM, Aymeric Moizard wrote:
1-> TCP connect failed
2-> second SRV is used: TCP connect succeed, but lock in tcp_send
That's what I understand.
I have tested a TCP connection to my server: It seems to be still
working.
On Thu, 28 Jan 2010, Henning Westerholt wrote:
so TCP connect failed, the tcp worker returned as it prints the message and,
to be sure I got it right, the UDP worker (the one that received) got
blocked? On Thursday 28 January 2010, Aymeric Moizard
wrote:
This is the exact case:
1-> SUBSCRIBE sent to/received by over UDP to kamailio.
2-> kamailio does a SRV record lookup for "mobipouce.com"
3-> kamailio try sip2.mobipouce.com (91.199.234.47) over TCP first
4-> connection failed with logs:
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_send:
connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:msg_send:
tcp_send failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:tm:t_forward_nonack: sending request failed
5-> I guess kamailio is supposed to try other SRV record value:
sip2.mobipouce.com (91.199.234.46) but it doesn't
Thus, I'm guessing the issue is related to SRV record with failover OR just
tcp failure. Not related to UDP at all. here is the backtrace I have. unfortunatly
without debug symbol!
I found the same for many of the kamailio process. "sched_yield"
is pending for ever. My system is a debian/etch.
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send ()
#3 0xb7975679 in send_pr_buffer () from /usr/lib/kamailio/modules/tm.so
#4 0xb79789ac in t_forward_nonack () from
/usr/lib/kamailio/modules/tm.so
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
#7 0x081cf810 in mem_pool ()
#8 0x00000000 in ?? ()
I guess most t_relay operation towards my "mobipouce.com" domain
with one IP being down breaks each kamailio process one after the
other... I'm not sure every such t_relay operation is always breaking
exactly one thread each time.
I went through the lock/unlock of tcp_main.c but it seems every
lock has an unlock at least...
Hi Aymeric,
i remember that we observed this "sched_yield" problems on one old 0.9
system
after some time (like weeks or month). We did not found the solution in
this
case, after a restart it was gone again..
You mentioned in an earlier mail that you see this related to UDP traffic,
but
in the log file and also in your investigations you think its related to
TPC? It's
definitly possible to reproduce the issue now!
I guess anyone can try your version of kamailio and t_relay message
to "mobipouce.com" and you'll fall in that case! Sending plenty of
those messages will finally lock all kamailio process.
All? tcp and udp? Cheers,
Daniel
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Regards,
Henning
Viele Grüße,
Henning
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
4:23 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
Hello,
On 1/28/10 3:34 PM, Aymeric Moizard wrote:
so TCP connect failed, the tcp worker returned as it prints the
message and, to be sure I got it right, the UDP worker (the one that
received) got blocked?
1-> TCP connect failed
2-> second SRV is used: TCP connect succeed, but lock in tcp_send
That's what I understand.
I have tested a TCP connection to my server: It seems to be still
working.
Only udp!
Aymeric
--
Daniel-Constantin Mierla
* http://www.asipto.com/
some other answer below:
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
> I am cc-ing sr-dev, since tcp code is from ser and Andrei may have
> more insights...
>
>
> On 1/28/10 2:41 PM, Aymeric Moizard wrote:
>>
>>
>> On Thu, 28 Jan 2010, Henning Westerholt wrote:
>>
>>> On Thursday 28 January 2010, Aymeric Moizard wrote:
>>>> here is the backtrace I have. unfortunatly without debug symbol!
can you recompile with debug symbols? Do you have it installed from
package or sources? It will give more hints about the place in the
function...
I will try to reproduce, but now I do not have the proper environment
for testing...
Thanks,
Daniel
> I found
the same for many of the kamailio process. "sched_yield"
> is pending for ever. My system is a debian/etch.
>
> #0 0xffffe424 in __kernel_vsyscall ()
> #1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
> #2 0x080a93fd in tcp_send ()
> #3 0xb7975679 in send_pr_buffer () from
> /usr/lib/kamailio/modules/tm.so
> #4 0xb79789ac in t_forward_nonack () from
> /usr/lib/kamailio/modules/tm.so
> #5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
> #6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
> #7 0x081cf810 in mem_pool ()
> #8 0x00000000 in ?? ()
>
> I guess most t_relay operation towards my "mobipouce.com" domain
> with one IP being down breaks each kamailio process one after the
> other... I'm not sure every such t_relay operation is always breaking
> exactly one thread each time.
>
> I went through the lock/unlock of tcp_main.c but it seems every
> lock has an unlock at least...
Hi Aymeric,
i remember that we observed this "sched_yield" problems on one old
0.9 system
after some time (like weeks or month). We did not found the
solution in this
case, after a restart it was gone again..
You mentioned in an earlier mail that you see this related to UDP
traffic, but
in the log file and also in your investigations you think its
related to TPC?
This is the exact case:
1-> SUBSCRIBE sent to/received by over UDP to kamailio.
2-> kamailio does a SRV record lookup for "mobipouce.com"
3-> kamailio try sip2.mobipouce.com (91.199.234.47) over TCP first
4-> connection failed with logs:
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_send: connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:msg_send:
tcp_send failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:tm:t_forward_nonack: sending request failed
5-> I guess kamailio is supposed to try other SRV record value:
sip2.mobipouce.com (91.199.234.46) but it doesn't
Thus, I'm guessing the issue is related to SRV record with failover
OR just tcp failure. Not related to UDP at all. It's
definitly possible to reproduce the issue now!
I guess anyone can try your version of kamailio and t_relay message
to "mobipouce.com" and you'll fall in that case! Sending plenty of
those messages will finally lock all kamailio process.
All? tcp and udp? Cheers,
Daniel
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Regards,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
--
Daniel-Constantin Mierla
* http://www.asipto.com/
Regards,
Henning
Viele Grüße,
Henning
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users 
6:27 p.m.
New subject: [Kamailio-Users] [sr-dev] kamailio / deadlock3
On Jan 28, 2010 at 14:56, Daniel-Constantin Mierla <miconda(a)gmail.com> wrote:
so TCP connect failed, the tcp worker returned as it prints the message
and, to be sure I got it right, the UDP worker (the one that received)
got blocked?
My guess (assuming kamailio 1.5) is that the problem is in tm somewhere
and not in the TCP code. There are hardly any TCP locks used in this
case (connect failure). Most likely the backtrace is bogus.
Andrei
I am cc-ing sr-dev, since tcp code is from ser and
Andrei may have more
insights...
Is this kamailio 1.5 or kamailio 3.0 (looks like <3.0 to me)?
On 1/28/10 2:41 PM, Aymeric Moizard wrote:
>
>
>On Thu, 28 Jan 2010, Henning Westerholt wrote:
>
>>On Thursday 28 January 2010, Aymeric Moizard wrote:
>>>here is the backtrace I have. unfortunatly without debug symbol!
>>>I found the same for many of the kamailio process. "sched_yield"
>>>is pending for ever. My system is a debian/etch.
>>>
>>>#0 0xffffe424 in __kernel_vsyscall ()
>>>#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
>>>#2 0x080a93fd in tcp_send ()
>>>#3 0xb7975679 in send_pr_buffer () from
>>>/usr/lib/kamailio/modules/tm.so
>>>#4 0xb79789ac in t_forward_nonack () from
>>>/usr/lib/kamailio/modules/tm.so
>>>#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
>>>#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
>>>#7 0x081cf810 in mem_pool ()
>>>#8 0x00000000 in ?? ()
The backtrace looks strange (mem_pool() and load_tm() for example).
It would help greatly to have it compiled with debug symbols from
source (or have around the exact source used for the compiled code).
>>>
>>>I guess most t_relay operation towards my "mobipouce.com" domain
>>>with one IP being down breaks each kamailio process one after the
>>>other... I'm not sure every such t_relay operation is always breaking
>>>exactly one thread each time.
>>>
>>>I went through the lock/unlock of tcp_main.c but it seems every
>>>lock has an unlock at least...
>>
>>Hi Aymeric,
>>
>>i remember that we observed this "sched_yield" problems on one old
>>0.9 system
>>after some time (like weeks or month). We did not found the solution
>>in this
>>case, after a restart it was gone again..
If it's kamailio 1.5 (or any version < 3.0) then the sched_yield() means
spinning on a lock. However I'm not sure we can trust the backtrace.
You mentioned in an earlier mail that you see this related to UDP
traffic, but
in the log file and also in your investigations you think its related
to TPC?
This is the exact case:
1-> SUBSCRIBE sent to/received by over UDP to kamailio.
2-> kamailio does a SRV record lookup for "mobipouce.com"
3-> kamailio try sip2.mobipouce.com (91.199.234.47) over TCP first
4-> connection failed with logs:
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: poll error: flags 18
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcp_blocking_connect: failed to retrieve SO_ERROR (111)
Connection refused
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:core:tcpconn_connect: tcp_blocking_connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:core:tcp_send:
connect failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]: ERROR:tm:msg_send:
tcp_send failed
Jan 27 12:56:38 ns26829 /usr/sbin/kamailio[9763]:
ERROR:tm:t_forward_nonack: sending request failed
5-> I guess kamailio is supposed to try other SRV record value:
sip2.mobipouce.com (91.199.234.46) but it doesn't
Thus, I'm guessing the issue is related to SRV record with failover OR
just tcp failure. Not related to UDP at all.
It's definitly possible to reproduce the issue now!
I guess anyone can try your version of kamailio and t_relay message
to "mobipouce.com" and you'll fall in that case! Sending plenty of
those messages will finally lock all kamailio process.
All? tcp and udp?
8:40 p.m.
New subject: [Kamailio-Users] [sr-dev] kamailio / deadlock3
On Thu, 28 Jan 2010, Andrei Pelinescu-Onciul wrote:
On Jan 28, 2010 at 14:56, Daniel-Constantin Mierla
<miconda(a)gmail.com> wrote:
This is branches/1.5
With svn version 5949.
Here is the debug backtrace: with kamailio-dbg_1.5.0_i386.deb installed:
(gdb) bt
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7d694ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send (send_sock=0x8159d60, type=3,
buf=0xb3992908 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nRecord-Route:
<sip:91.121.81.212:5061;transport=tls;r2=on;lr=on>\r\nRecord-Route:
<sip:91.121.81.212;r2=on;lr=on>\r\nVia: SIP/2.0/TLS
91.121.81.212:5061;branc"..., len=645, to=0xb392f494, id=0) at
fastlock.h:182
#3 0xb79ef679 in send_pr_buffer (rb=0xb392f480, buf=0xb3992908, len=645)
at ../../forward.h:127
#4 0xb79f29ac in t_forward_nonack (t=0xb392f368, p_msg=0x81d02d8,
proxy=0x0) at t_fwd.c:691
#5 0xb79ee784 in t_relay_to (p_msg=0x81d02d8, proxy=0x0, flags=<value
optimized out>) at t_funcs.c:264
#6 0xb79fda11 in w_t_relay (p_msg=0x81d02d8, proxy=0x0, flags=0x0) at
tm.c:1002
#7 0x080551ef in do_action (a=0x8172100, msg=0x81d02d8) at action.c:874
#8 0x080577df in run_action_list (a=0x8172100, msg=0x81d02d8) at
action.c:145
#9 0x0808f11b in eval_expr (e=0x8172168, msg=0x81d02d8, val=0x0) at
route.c:1171
#10 0x0808ebb0 in eval_expr (e=0x8172190, msg=0x81d02d8, val=0x0) at
route.c:1488
#11 0x0808eb3f in eval_expr (e=0x81721b8, msg=0x81d02d8, val=0x0) at
route.c:1493
#12 0x08055005 in do_action (a=0x81722d0, msg=0x81d02d8) at action.c:729
#13 0x080577df in run_action_list (a=0x8171928, msg=0x81d02d8) at
action.c:145
#14 0x08055e49 in do_action (a=0x816ba50, msg=0x81d02d8) at action.c:120
#15 0x080577df in run_action_list (a=0x816ba50, msg=0x81d02d8) at
action.c:145
#16 0x08056d0f in do_action (a=0x816bab8, msg=0x81d02d8) at action.c:746
#17 0x080577df in run_action_list (a=0x81618c0, msg=0x81d02d8) at
action.c:145
#18 0x08057b93 in run_top_route (a=0x81618c0, msg=0x81d02d8) at
action.c:120
#19 0x08083a0d in receive_msg (
buf=0x81341c0 "SUBSCRIBE sip:aymeric2@mobipouce.com SIP/2.0\r\nVia:
SIP/2.0/UDP 192.168.2.3:6010;rport;branch=z9hG4bK972183375\r\nFrom:
\"aymeric\" <sip:antisip@sip.antisip.com>;tag=286101806\r\nTo:
<sip:aymeric2@mobipouce."..., len=692, rcv_info=0xbfc9ad54) at
receive.c:175
#20 0x080b3943 in udp_rcv_loop () at udp_server.c:460
#21 0x0806b294 in main (argc=-1211358212, argv=0xb7f61590) at main.c:774
One thing that didn't came up before is that it seems the message
is containing TLS, not TCP. I don't have time to analyse it now
deeper, but I may try to change the SRV to see how it differ.
Tks,
Aymeric
I am cc-ing sr-dev, since tcp code is from ser
and Andrei may have more
insights...
Is this kamailio 1.5 or kamailio 3.0 (looks like <3.0 to me)?
8:52 p.m.
New subject: [Kamailio-Users] [sr-dev] kamailio / deadlock3
On 1/28/10 8:40 PM, Aymeric Moizard wrote:
On Thu, 28 Jan 2010, Andrei Pelinescu-Onciul wrote:
I thought it is 3.0.0, as all your other emails were related
to this
version. On another hand, if you run 1.x is better to use the last one,
1.5.3.
Please include the version when you report a problem, otherwise we can
hunt in difference places.
Cheers,
Daniel
On Jan 28, 2010 at 14:56, Daniel-Constantin
Mierla
<miconda(a)gmail.com> wrote:
This is branches/1.5
With svn version 5949. I am cc-ing sr-dev, since tcp code is from ser
and Andrei may have more
insights...
Is this kamailio 1.5 or kamailio 3.0 (looks like <3.0 to me)?
Here is the debug backtrace: with kamailio-dbg_1.5.0_i386.deb installed:
(gdb) bt
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7d694ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send (send_sock=0x8159d60, type=3,
buf=0xb3992908 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nRecord-Route:
<sip:91.121.81.212:5061;transport=tls;r2=on;lr=on>\r\nRecord-Route:
<sip:91.121.81.212;r2=on;lr=on>\r\nVia: SIP/2.0/TLS
91.121.81.212:5061;branc"..., len=645, to=0xb392f494, id=0) at
fastlock.h:182
#3 0xb79ef679 in send_pr_buffer (rb=0xb392f480, buf=0xb3992908,
len=645) at ../../forward.h:127
#4 0xb79f29ac in t_forward_nonack (t=0xb392f368, p_msg=0x81d02d8,
proxy=0x0) at t_fwd.c:691
#5 0xb79ee784 in t_relay_to (p_msg=0x81d02d8, proxy=0x0, flags=<value
optimized out>) at t_funcs.c:264
#6 0xb79fda11 in w_t_relay (p_msg=0x81d02d8, proxy=0x0, flags=0x0) at
tm.c:1002
#7 0x080551ef in do_action (a=0x8172100, msg=0x81d02d8) at action.c:874
#8 0x080577df in run_action_list (a=0x8172100, msg=0x81d02d8) at
action.c:145
#9 0x0808f11b in eval_expr (e=0x8172168, msg=0x81d02d8, val=0x0) at
route.c:1171
#10 0x0808ebb0 in eval_expr (e=0x8172190, msg=0x81d02d8, val=0x0) at
route.c:1488
#11 0x0808eb3f in eval_expr (e=0x81721b8, msg=0x81d02d8, val=0x0) at
route.c:1493
#12 0x08055005 in do_action (a=0x81722d0, msg=0x81d02d8) at action.c:729
#13 0x080577df in run_action_list (a=0x8171928, msg=0x81d02d8) at
action.c:145
#14 0x08055e49 in do_action (a=0x816ba50, msg=0x81d02d8) at action.c:120
#15 0x080577df in run_action_list (a=0x816ba50, msg=0x81d02d8) at
action.c:145
#16 0x08056d0f in do_action (a=0x816bab8, msg=0x81d02d8) at action.c:746
#17 0x080577df in run_action_list (a=0x81618c0, msg=0x81d02d8) at
action.c:145
#18 0x08057b93 in run_top_route (a=0x81618c0, msg=0x81d02d8) at
action.c:120
#19 0x08083a0d in receive_msg (
buf=0x81341c0 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nVia: SIP/2.0/UDP
192.168.2.3:6010;rport;branch=z9hG4bK972183375\r\nFrom: \"aymeric\"
<sip:antisip@sip.antisip.com>;tag=286101806\r\nTo:
<sip:aymeric2@mobipouce."..., len=692, rcv_info=0xbfc9ad54) at
receive.c:175
#20 0x080b3943 in udp_rcv_loop () at udp_server.c:460
#21 0x0806b294 in main (argc=-1211358212, argv=0xb7f61590) at main.c:774
One thing that didn't came up before is that it seems the message
is containing TLS, not TCP. I don't have time to analyse it now
deeper, but I may try to change the SRV to see how it differ.
Tks,
Aymeric
--
Daniel-Constantin Mierla
* http://www.asipto.com/
9:17 p.m.
New subject: [Kamailio-Users] [sr-dev] kamailio / deadlock3
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
On 1/28/10 8:40 PM, Aymeric Moizard wrote:
Sorry, I though I did mentioned it in my initial mail (sent on kamailio
mailing list) however, it waz not the case.
I would have asked on ser-users if it was 3.0 ;)
On Thu, 28 Jan 2010, Andrei Pelinescu-Onciul wrote:
I thought it is 3.0.0, as all your other emails were
related to this version. On Jan 28, 2010 at 14:56, Daniel-Constantin
Mierla <miconda(a)gmail.com>
wrote:
This is branches/1.5
With svn version 5949. I am cc-ing sr-dev, since tcp code is from ser
and Andrei may have more
insights...
Is this kamailio 1.5 or kamailio 3.0 (looks like <3.0 to me)? On another hand, if you run 1.x is better to use the
last one, 1.5.3.
I'm pretty sure it's 1.5.3: changelog starts with:
===================== 2009-10-XX Kamailio v1.5.3 released =====================
Regards,
Aymeric
Please include the version when you report a problem,
otherwise we can hunt
in difference places.
Cheers,
Daniel
Here is the debug backtrace: with kamailio-dbg_1.5.0_i386.deb installed:
(gdb) bt
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7d694ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send (send_sock=0x8159d60, type=3,
buf=0xb3992908 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nRecord-Route:
<sip:91.121.81.212:5061;transport=tls;r2=on;lr=on>\r\nRecord-Route:
<sip:91.121.81.212;r2=on;lr=on>\r\nVia: SIP/2.0/TLS
91.121.81.212:5061;branc"..., len=645, to=0xb392f494, id=0) at
fastlock.h:182
#3 0xb79ef679 in send_pr_buffer (rb=0xb392f480, buf=0xb3992908, len=645)
at ../../forward.h:127
#4 0xb79f29ac in t_forward_nonack (t=0xb392f368, p_msg=0x81d02d8,
proxy=0x0) at t_fwd.c:691
#5 0xb79ee784 in t_relay_to (p_msg=0x81d02d8, proxy=0x0, flags=<value
optimized out>) at t_funcs.c:264
#6 0xb79fda11 in w_t_relay (p_msg=0x81d02d8, proxy=0x0, flags=0x0) at
tm.c:1002
#7 0x080551ef in do_action (a=0x8172100, msg=0x81d02d8) at action.c:874
#8 0x080577df in run_action_list (a=0x8172100, msg=0x81d02d8) at
action.c:145
#9 0x0808f11b in eval_expr (e=0x8172168, msg=0x81d02d8, val=0x0) at
route.c:1171
#10 0x0808ebb0 in eval_expr (e=0x8172190, msg=0x81d02d8, val=0x0) at
route.c:1488
#11 0x0808eb3f in eval_expr (e=0x81721b8, msg=0x81d02d8, val=0x0) at
route.c:1493
#12 0x08055005 in do_action (a=0x81722d0, msg=0x81d02d8) at action.c:729
#13 0x080577df in run_action_list (a=0x8171928, msg=0x81d02d8) at
action.c:145
#14 0x08055e49 in do_action (a=0x816ba50, msg=0x81d02d8) at action.c:120
#15 0x080577df in run_action_list (a=0x816ba50, msg=0x81d02d8) at
action.c:145
#16 0x08056d0f in do_action (a=0x816bab8, msg=0x81d02d8) at action.c:746
#17 0x080577df in run_action_list (a=0x81618c0, msg=0x81d02d8) at
action.c:145
#18 0x08057b93 in run_top_route (a=0x81618c0, msg=0x81d02d8) at
action.c:120
#19 0x08083a0d in receive_msg (
buf=0x81341c0 "SUBSCRIBE sip:aymeric2@mobipouce.com SIP/2.0\r\nVia:
SIP/2.0/UDP 192.168.2.3:6010;rport;branch=z9hG4bK972183375\r\nFrom:
\"aymeric\" <sip:antisip@sip.antisip.com>;tag=286101806\r\nTo:
<sip:aymeric2@mobipouce."..., len=692, rcv_info=0xbfc9ad54) at
receive.c:175
#20 0x080b3943 in udp_rcv_loop () at udp_server.c:460
#21 0x0806b294 in main (argc=-1211358212, argv=0xb7f61590) at main.c:774
One thing that didn't came up before is that it seems the message
is containing TLS, not TCP. I don't have time to analyse it now
deeper, but I may try to change the SRV to see how it differ.
Tks,
Aymeric
--
Daniel-Constantin Mierla
* http://www.asipto.com/
9:37 p.m.
New subject: [Kamailio-Users] [sr-dev] kamailio / deadlock3
On 1/28/10 9:17 PM, Aymeric Moizard wrote:
--
Daniel-Constantin Mierla
* http://www.asipto.com/
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
kamailio 3.0.0 is on this mailing list, being latest stable version of
kamailio. sr-users is mainly for mainstream sip-router users. serusers
is for ser project users.
i do not care if kamailio and sr-users get mixed, though, i prefer
kamailio to be discussed here, there are different default setting for
it than in other 3.0 branches.
Cheers,
Daniel
On 1/28/10 8:40 PM, Aymeric Moizard wrote:
Sorry, I though I did mentioned it in my initial mail (sent on
kamailio mailing list) however, it waz not the case.
I would have asked on ser-users if it was 3.0 ;)
On Thu, 28 Jan 2010, Andrei Pelinescu-Onciul wrote:
I thought it is 3.0.0, as all your other emails were
related to this
version. On Jan 28, 2010 at 14:56, Daniel-Constantin
Mierla
<miconda(a)gmail.com> wrote:
> I am cc-ing sr-dev, since tcp code is from ser and Andrei may have
> more
> insights...
Is this kamailio 1.5 or kamailio 3.0 (looks like <3.0 to me)?
This is branches/1.5
With svn version 5949. On another hand, if you run 1.x is better to use
the last one, 1.5.3.
I'm pretty sure it's 1.5.3: changelog starts with:
===================== 2009-10-XX Kamailio v1.5.3 released
=====================
Regards,
Aymeric
Please include the version when you report a
problem, otherwise we
can hunt in difference places.
Cheers,
Daniel
_______________________________________________
sr-dev mailing list
sr-dev(a)lists.sip-router.org
http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-dev
Here is the debug backtrace: with kamailio-dbg_1.5.0_i386.deb
installed:
(gdb) bt
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7d694ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send (send_sock=0x8159d60, type=3,
buf=0xb3992908 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nRecord-Route:
<sip:91.121.81.212:5061;transport=tls;r2=on;lr=on>\r\nRecord-Route:
<sip:91.121.81.212;r2=on;lr=on>\r\nVia: SIP/2.0/TLS
91.121.81.212:5061;branc"..., len=645, to=0xb392f494, id=0) at
fastlock.h:182
#3 0xb79ef679 in send_pr_buffer (rb=0xb392f480, buf=0xb3992908,
len=645) at ../../forward.h:127
#4 0xb79f29ac in t_forward_nonack (t=0xb392f368, p_msg=0x81d02d8,
proxy=0x0) at t_fwd.c:691
#5 0xb79ee784 in t_relay_to (p_msg=0x81d02d8, proxy=0x0,
flags=<value optimized out>) at t_funcs.c:264
#6 0xb79fda11 in w_t_relay (p_msg=0x81d02d8, proxy=0x0, flags=0x0)
at tm.c:1002
#7 0x080551ef in do_action (a=0x8172100, msg=0x81d02d8) at
action.c:874
#8 0x080577df in run_action_list (a=0x8172100, msg=0x81d02d8) at
action.c:145
#9 0x0808f11b in eval_expr (e=0x8172168, msg=0x81d02d8, val=0x0) at
route.c:1171
#10 0x0808ebb0 in eval_expr (e=0x8172190, msg=0x81d02d8, val=0x0) at
route.c:1488
#11 0x0808eb3f in eval_expr (e=0x81721b8, msg=0x81d02d8, val=0x0) at
route.c:1493
#12 0x08055005 in do_action (a=0x81722d0, msg=0x81d02d8) at
action.c:729
#13 0x080577df in run_action_list (a=0x8171928, msg=0x81d02d8) at
action.c:145
#14 0x08055e49 in do_action (a=0x816ba50, msg=0x81d02d8) at
action.c:120
#15 0x080577df in run_action_list (a=0x816ba50, msg=0x81d02d8) at
action.c:145
#16 0x08056d0f in do_action (a=0x816bab8, msg=0x81d02d8) at
action.c:746
#17 0x080577df in run_action_list (a=0x81618c0, msg=0x81d02d8) at
action.c:145
#18 0x08057b93 in run_top_route (a=0x81618c0, msg=0x81d02d8) at
action.c:120
#19 0x08083a0d in receive_msg (
buf=0x81341c0 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nVia: SIP/2.0/UDP
192.168.2.3:6010;rport;branch=z9hG4bK972183375\r\nFrom: \"aymeric\"
<sip:antisip@sip.antisip.com>;tag=286101806\r\nTo:
<sip:aymeric2@mobipouce."..., len=692, rcv_info=0xbfc9ad54) at
receive.c:175
#20 0x080b3943 in udp_rcv_loop () at udp_server.c:460
#21 0x0806b294 in main (argc=-1211358212, argv=0xb7f61590) at
main.c:774
One thing that didn't came up before is that it seems the message
is containing TLS, not TCP. I don't have time to analyse it now
deeper, but I may try to change the SRV to see how it differ.
Tks,
Aymeric
--
Daniel-Constantin Mierla
* http://www.asipto.com/
2 Feb
2 Feb
12:19 p.m.
New subject: [Kamailio-Users] [sr-dev] kamailio / deadlock3
Hello Aymeric,
I was forwarding some registrations to your domain and didn't get the
deadlock. I did get tcp_send failed many times and then used second dns
record. I have two udp workers.
How fast did you get to lock?
I am using latest 1.5.3 here, what is your output of kamailio -V?
Thanks,
Daniel
On 1/28/10 9:17 PM, Aymeric Moizard wrote:
On Thu, 28 Jan 2010, Daniel-Constantin Mierla wrote:
--
Daniel-Constantin Mierla
eLearning class for Kamailio 3.0.0
Starting Feb 8, 2010
* http://www.asipto.com/
On 1/28/10 8:40 PM, Aymeric Moizard wrote:
Sorry, I though I did mentioned it in my initial mail (sent on
kamailio mailing list) however, it waz not the case.
I would have asked on ser-users if it was 3.0 ;)
On Thu, 28 Jan 2010, Andrei Pelinescu-Onciul wrote:
I thought it is 3.0.0, as all your other emails were
related to this
version. On Jan 28, 2010 at 14:56, Daniel-Constantin
Mierla
<miconda(a)gmail.com> wrote:
> I am cc-ing sr-dev, since tcp code is from ser and Andrei may have
> more
> insights...
Is this kamailio 1.5 or kamailio 3.0 (looks like <3.0 to me)?
This is branches/1.5
With svn version 5949. On another hand, if you run 1.x is better to use
the last one, 1.5.3.
I'm pretty sure it's 1.5.3: changelog starts with:
===================== 2009-10-XX Kamailio v1.5.3 released
=====================
Regards,
Aymeric
Please include the version when you report a
problem, otherwise we
can hunt in difference places.
Cheers,
Daniel
_______________________________________________
Kamailio (OpenSER) - Users mailing list
Users(a)lists.kamailio.org
http://lists.kamailio.org/cgi-bin/mailman/listinfo/users
http://lists.openser-project.org/cgi-bin/mailman/listinfo/users
Here is the debug backtrace: with kamailio-dbg_1.5.0_i386.deb
installed:
(gdb) bt
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7d694ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send (send_sock=0x8159d60, type=3,
buf=0xb3992908 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nRecord-Route:
<sip:91.121.81.212:5061;transport=tls;r2=on;lr=on>\r\nRecord-Route:
<sip:91.121.81.212;r2=on;lr=on>\r\nVia: SIP/2.0/TLS
91.121.81.212:5061;branc"..., len=645, to=0xb392f494, id=0) at
fastlock.h:182
#3 0xb79ef679 in send_pr_buffer (rb=0xb392f480, buf=0xb3992908,
len=645) at ../../forward.h:127
#4 0xb79f29ac in t_forward_nonack (t=0xb392f368, p_msg=0x81d02d8,
proxy=0x0) at t_fwd.c:691
#5 0xb79ee784 in t_relay_to (p_msg=0x81d02d8, proxy=0x0,
flags=<value optimized out>) at t_funcs.c:264
#6 0xb79fda11 in w_t_relay (p_msg=0x81d02d8, proxy=0x0, flags=0x0)
at tm.c:1002
#7 0x080551ef in do_action (a=0x8172100, msg=0x81d02d8) at
action.c:874
#8 0x080577df in run_action_list (a=0x8172100, msg=0x81d02d8) at
action.c:145
#9 0x0808f11b in eval_expr (e=0x8172168, msg=0x81d02d8, val=0x0) at
route.c:1171
#10 0x0808ebb0 in eval_expr (e=0x8172190, msg=0x81d02d8, val=0x0) at
route.c:1488
#11 0x0808eb3f in eval_expr (e=0x81721b8, msg=0x81d02d8, val=0x0) at
route.c:1493
#12 0x08055005 in do_action (a=0x81722d0, msg=0x81d02d8) at
action.c:729
#13 0x080577df in run_action_list (a=0x8171928, msg=0x81d02d8) at
action.c:145
#14 0x08055e49 in do_action (a=0x816ba50, msg=0x81d02d8) at
action.c:120
#15 0x080577df in run_action_list (a=0x816ba50, msg=0x81d02d8) at
action.c:145
#16 0x08056d0f in do_action (a=0x816bab8, msg=0x81d02d8) at
action.c:746
#17 0x080577df in run_action_list (a=0x81618c0, msg=0x81d02d8) at
action.c:145
#18 0x08057b93 in run_top_route (a=0x81618c0, msg=0x81d02d8) at
action.c:120
#19 0x08083a0d in receive_msg (
buf=0x81341c0 "SUBSCRIBE sip:aymeric2@mobipouce.com
SIP/2.0\r\nVia: SIP/2.0/UDP
192.168.2.3:6010;rport;branch=z9hG4bK972183375\r\nFrom: \"aymeric\"
<sip:antisip@sip.antisip.com>;tag=286101806\r\nTo:
<sip:aymeric2@mobipouce."..., len=692, rcv_info=0xbfc9ad54) at
receive.c:175
#20 0x080b3943 in udp_rcv_loop () at udp_server.c:460
#21 0x0806b294 in main (argc=-1211358212, argv=0xb7f61590) at
main.c:774
One thing that didn't came up before is that it seems the message
is containing TLS, not TCP. I don't have time to analyse it now
deeper, but I may try to change the SRV to see how it differ.
Tks,
Aymeric
--
Daniel-Constantin Mierla
* http://www.asipto.com/
28 Jan
28 Jan
3:13 p.m.
New subject: [Kamailio-Users] kamailio / deadlock3
Again additionnal information:
Doing new capture: after the failure, I can see that a TCP
connection is made with second SRV record: sip.mobipouce.com
(91.199.234.46)
I got:
SYN ACK -> sip.mobipouce.com
ACK <- sip.mobipouce.com
PSH, ACK <- sip.mobipouce.com
ACK -> sip.mobipouce.com
I'm guessing that this is where the stack trace is dead locked because
no SUBSCRIBE is sent then... -> #2 0x080a93fd in tcp_send ()
strangly in this "tcp_send" method, there is no
TCPCONN_LOCK/TCPCONN_UNLOCK: instead, there is
a
lock_get(&c->write_lock);
...
lock_release(&c->write_lock);
May be still correct anyway...
Tks,
Aymeric MOIZARD / ANTISIP
amsip - http://www.antisip.com
osip2 - http://www.osip.org
eXosip2 - http://savannah.nongnu.org/projects/exosip/
On Thu, 28 Jan 2010, Henning Westerholt wrote:
On Thursday 28 January 2010, Aymeric Moizard wrote:
here is the backtrace I have. unfortunatly
without debug symbol!
I found the same for many of the kamailio process. "sched_yield"
is pending for ever. My system is a debian/etch.
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7cef4ac in sched_yield () from /lib/tls/i686/cmov/libc.so.6
#2 0x080a93fd in tcp_send ()
#3 0xb7975679 in send_pr_buffer () from /usr/lib/kamailio/modules/tm.so
#4 0xb79789ac in t_forward_nonack () from /usr/lib/kamailio/modules/tm.so
#5 0xb7974784 in t_relay_to () from /usr/lib/kamailio/modules/tm.so
#6 0xb7983a11 in load_tm () from /usr/lib/kamailio/modules/tm.so
#7 0x081cf810 in mem_pool ()
#8 0x00000000 in ?? ()
I guess most t_relay operation towards my "mobipouce.com" domain
with one IP being down breaks each kamailio process one after the
other... I'm not sure every such t_relay operation is always breaking
exactly one thread each time.
I went through the lock/unlock of tcp_main.c but it seems every
lock has an unlock at least...
Hi Aymeric,
i remember that we observed this "sched_yield" problems on one old 0.9 system
after some time (like weeks or month). We did not found the solution in this
case, after a restart it was gone again..
You mentioned in an earlier mail that you see this related to UDP traffic, but
in the log file and also in your investigations you think its related to TPC?
Regards,
Henning
Viele Grüße,
Henning
5200
days inactive
5218
days old
24 comments
6 participants
participants (6)
-
Andrei Pelinescu-Onciul -
Aymeric Moizard -
Daniel-Constantin Mierla -
Eric Hiller -
Henning Westerholt -
Klaus Darilion