Re: [Serusers] Serious Problems with SER - Not responding anything.

Hello list. I'm experiencing problems with my SER. As i mentioned in a past post (http://lists.iptel.org/pipermail/serusers/2005-April/018324.html), from time to time (it is getting pretty often) my SER not answering any request, when a REGISTER reach the server it is not challenged, this happens too with the INVITE's messages. About two hours ago this happened again. I asked in my last post how to LOG the internal errors or messages from SER to see if i can get some idea about what's going on, but i have no answer about it. This is the infornmation i was able to capture when this was happening : the serctl moni command shows a lot of "failures". [cycle #: 11; if constant make sure server lives and fifo is on] Server: Sip EXpress router (0.8.14-3 (i386/linux)) Now: Wed Apr 20 12:56:29 2005 Up Since: Thu Apr 14 21:32:18 2005 Up time: 487451 [sec] Transaction Statistics Current: 6 (2612 waiting) Total: 38194 (0 local) Replied localy: 80752 Completion status 6xx: 4643, 5xx: 768, 4xx: 10664, 3xx: 0,2xx: 22552 Stateless Server Statistics 200: 5975004 202: 0 2xx: 0 300: 0 301: 0 302: 0 3xx: 0 400: 0 401: 205980 403: 0 404: 0 407: 22918 408: 0 483: 0 4xx: 19 500: 0 5xx: 0 6xx: 0 xxx: 0 failures: 24180 UsrLoc Stats Domain Registered Expired 'location' 219 4552 -------------------------------------------------------------- [cycle #: 5; if constant make sure server lives and fifo is on] Server: Sip EXpress router (0.8.14-3 (i386/linux)) Now: Wed Apr 20 12:56:47 2005 Up Since: Thu Apr 14 21:32:18 2005 Up time: 487469 [sec] Transaction Statistics Current: 4 (2619 waiting) Total: 38219 (0 local) Replied localy: 80762 Completion status 6xx: 4643, 5xx: 770, 4xx: 10667, 3xx: 0,2xx: 22552 Stateless Server Statistics 200: 5975030 202: 0 2xx: 0 300: 0 301: 0 302: 0 3xx: 0 400: 0 401: 205995 403: 0 404: 0 407: 22922 408: 0 483: 0 4xx: 19 500: 0 5xx: 0 6xx: 0 xxx: 0 failures: 24380 UsrLoc Stats Domain Registered Expired 'location' 220 4552 ----------------------------------------------------------------- [cycle #: 2; if constant make sure server lives and fifo is on] Server: Sip EXpress router (0.8.14-3 (i386/linux)) Now: Wed Apr 20 12:56:52 2005 Up Since: Thu Apr 14 21:32:18 2005 Up time: 487474 [sec] Transaction Statistics Current: 4 (2620 waiting) Total: 38224 (0 local) Replied localy: 80762 Completion status 6xx: 4643, 5xx: 770, 4xx: 10667, 3xx: 0,2xx: 22552 Stateless Server Statistics 200: 5975034 202: 0 2xx: 0 300: 0 301: 0 302: 0 3xx: 0 400: 0 401: 206000 403: 0 404: 0 407: 22923 408: 0 483: 0 4xx: 19 500: 0 5xx: 0 6xx: 0 xxx: 0 failures: 24445 UsrLoc Stats Domain Registered Expired 'location' 215 4557 What is happening to cause the "failures" counter grows that much?. The free command shows this : [root@root]# free total used free shared buffers cached Mem: 1030888 626464 404424 0 86692 369848 -/+ buffers/cache: 169924 860964 Swap: 2040244 4864 2035380 The top command shows not much use of CPU. The proccess were running (the SER and mediaproxy) Despite of all this, the SER was not answering anything. Could this be a bug? or maybe a problem with my ser.cfg file?. I'm attaching my ser.cfg file too to see if someone can tell me if there is something wrong about it. I really hope that someone could help me, this is getting really complicated every time it happens. Thanks Ricardo Martinez.- ************************************************** SER.CFG ************************************************** #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++ # ----------- global configuration parameters ------------------------ debug=0 # debug level (cmd line: -dddddddddd) fork=yes log_stderror=yes # (cmd line: -E) #Uncomment these lines to enter debugging mode #fork=no #log_stderror=yes listen=ip.of.my.ser alias=sip.mydomain.com alias=sip2.mydomain.net alias=ip.of.my.ser check_via=no # (cmd. line: -v) dns=no # (cmd. line: -r) rev_dns=no # (cmd. line: -R) port=5060 #children=4 fifo="/tmp/ser_fifo" fifo_mode=0666 # ------------------ module loading ---------------------------------- # Uncomment this if you want to use SQL database #loadmodule "/usr/local//lib/ser/modules/mysql.so" loadmodule "/usr/local//lib/ser/modules/sl.so" loadmodule "/usr/local//lib/ser/modules/tm.so" loadmodule "/usr/local//lib/ser/modules/rr.so" loadmodule "/usr/local//lib/ser/modules/maxfwd.so" loadmodule "/usr/local//lib/ser/modules/usrloc.so" loadmodule "/usr/local//lib/ser/modules/registrar.so" loadmodule "/usr/local//lib/ser/modules/textops.so" loadmodule "/usr/local//lib/ser/modules/exec.so" # ++++++++++ Para trabajar con NAT loadmodule "/usr/local//lib/ser/modules/mediaproxy.so" loadmodule "/usr/local//lib/ser/modules/domain.so" loadmodule "/usr/local//lib/ser/modules/dbtext.so" loadmodule "/usr/local//lib/ser/modules/nathelper.so" # Uncomment this if you want digest authentication # mysql.so must be loaded ! loadmodule "/usr/local/lib/ser/modules/auth.so" # ++++++++++ Modulos de Accounting y Autorizacion via Radius loadmodule "/usr/local/lib/ser/modules/auth_radius.so" loadmodule "/usr/local/lib/ser/modules/group_radius.so" loadmodule "/usr/local/lib/ser/modules/acc.so" # ----------------- setting module-specific parameters --------------- # -- rr params -- # add value to ;lr param to make some broken UAs happy modparam("rr", "enable_full_lr", 1) # +++++ module authorization modparam("auth_radius","radius_config","/usr/local/etc/radiusclient/radiuscl ient.conf") modparam("auth_radius","service_type",15) # +++++ module accounting modparam("acc","radius_config","/usr/local/etc/radiusclient/radiusclient.con f") modparam("acc","log_level",1) modparam("acc", "service_type", 15) modparam("acc", "radius_flag", 1) modparam("acc", "radius_missed_flag", 3) modparam("acc", "failed_transactions", 1) modparam("acc", "report_ack", 0) # +++++ group radius modparam("group_radius","radius_config","/usr/local/etc/radiusclient/radiusc lient.conf") modparam("group_radius", "use_domain", 1) # +++++ module mediaproxy modparam("mediaproxy", "natping_interval", 60) modparam("mediaproxy", "mediaproxy_socket", "/var/run/mediaproxy.sock") #modparam("mediaproxy", "sip_asymmetrics", "/usr/local/etc/ser/sip-asymmetrics-clients") #modparam("mediaproxy", "rtp_asymmetrics", "/usr/local/etc/ser/rtp-asymmetrics-clients") modparam("registrar", "nat_flag", 5) # +++++ module domain modparam("domain", "db_url", "/usr/local/etc/ser/domaintables") modparam("domain", "domain_table", "domain") modparam("domain", "domain_col", "domain") # +++++ module tm modparam("tm", "fr_timer", 15) modparam("tm", "fr_inv_timer", 22) modparam("tm", "wt_timer", 5) # +++++ module registrar (Con estos parametro haces que el location lookup priorize solo los mas recientes contacts) modparam("registrar", "append_branches", 0) modparam("registrar", "desc_time_order", 1) # ------------------------- request routing logic ------------------- # main routing logic route { #++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Llamada a Rutina Externa "translate" para cambio de numero #++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if (method=="INVITE" || method=="CANCEL") { exec_dset("/usr/local/etc/ser/translate"); }; #++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Initial sanity checks -- messages with # max_forwards==0, or excessively long requests #++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if (!mf_process_maxfwd_header("10")) { sl_send_reply("483","Too Many Hops"); break; }; if ( msg:len > max_len ) { sl_send_reply("513", "Message too big"); break; }; #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Marcamos todos los mensajes para que pasen por nuestro servidor # a traves del record route. #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ record_route(); #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Permitimos el uso del campo Route en el ruteo de la llamada # #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if (method=="BYE" || method=="CANCEL") { log(1, "NAT: BYE o CANCEL recibido --> terminando la sesion de media\n"); end_media_session(); setflag(1); }; if (loose_route()) { t_relay(); break; }; #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + # Manejo de los mensajes NOTIFY para Keep Alive de NAT #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + if ((method=="NOTIFY") && search("^Event: keep-alive")) { sl_send_reply("200", "OK"); break; }; #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + # Marcamos las llamadas para accounting #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + setflag(1); #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + # Test para clientes NATeados # # 1 : Chequea si el cliente tiene IP privada en el campo Contact. # 2 : Chequea si el cliente se contacto con una IP distinta a la del campo VIA. # 3 : Chequea si el cliente tiene IP privada en el ultimo campo VIA. # #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + if (client_nat_test("3")) { log(1, "NAT: Requerimiento de IP privada --> fixed contact (en rutina principal)\n"); setflag(5); force_rport(); if (method=="REGISTER") { fix_nated_contact(); } else { fix_contact(); }; append_hf("P-hint: fixed NAT contact for request\r\n"); }; if (uri==myself) { #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Chequeo de REGISTRO para equipos del dominio 1 # Dominio Voiss : sip.mydomain.com #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if (method=="REGISTER" && uri=~"^sip:.*sip.mydomain.com") { # Challenge/Response if ( !radius_www_authorize("sip.mydomain.com")) { www_challenge("sip.mydomain.com", "1"); break; }; # Mandamos un mensaje de Log si es que un cliente NATEADO se registro. if (isflagset(5)) { log(1, "NAT: Cliente NAT'eado registrado (dentro del chequeo de REGISTER)\n"); }; save("location"); break; }; # Fin chequeo de Registro para Dominio : sip.mydomain.com #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ # Chequeo de REGISTRO para equipos del dominio 2 # Dominio Mayoristas : sip2.mydomain.net #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ if (method=="REGISTER" && uri=~"^sip:.*sip2.mydomain.net") { # Challenge/Response if ( !radius_www_authorize("sip2.mydomain.net")) { www_challenge("sip2.mydomain.net", "1"); break; }; # Mandamos un mensaje de Log si es que un cliente NATEADO se registro. if (isflagset(5)) { log(1, "NAT: Cliente NAT'eado registrado (dentro del chequeo de REGISTER)\n"); }; save("location"); break; }; # Fin chequeo de Registro para Dominio : sip2.mydomain.net #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++ # CHEQUEO DEL INVITE PARA TODAS LAS LLAMADAS #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++ if (method=="INVITE" && src_ip!=xx.xx.xx.xx) { if ( !radius_proxy_authorize("")) { proxy_challenge("", "1"); log (1, "LOG: Llamada no AUTORIZADA\n"); break; }; }; #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++ # Llamadas hacia dominio H.323 #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++ if ( (uri=~"^sip:717.*@.*") | (uri=~"^sip:707.*@.*") | (uri=~"^sip:777.*@.*") | (uri=~"^sip:333.*@.*") | (uri=~"^sip :0.*@.*") ){ log(1,"FORWARDING : Llamada hacia plataforma H.323\n"); route(1); break; }; #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++ # Llamadas SIP locales #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++ if (!lookup("location")) { #Usuarios que no estan en "location" database. route(4); break; }; setflag(3); #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++ # Manejo para llamadas NAT'das #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++ if (method=="INVITE") { log(1, "ROUTING: Reply processing (online user) enabled to handle NAT'd resonses\n"); t_on_reply("1"); if ( (isflagset(5)) && (method=="INVITE" || method=="ACK") ) { log(1, "NAT: Invite received --> enabling media proxy (else del lookup 2do if)\n"); use_media_proxy(); append_hf("P-hint: request forced to media proxy\r\n"); }; }; # Do it ? (Ricardo Martinez) }; append_hf("P-hint: USRLOC\r\n"); #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++ # Efectuamos la llamada #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++ if (!t_relay()) { sl_reply_error(); break; }; } /* end of initial routing logic */ #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++ # COMIENZO DE LA SECCION DE RUTAS #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++ #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ++++++++++++++++++ # Route 1 : Llamadas con destino plataforma H.323#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++++++++ route[1] { # send it out now; use stateful forwarding as it works reliably # even for UDP2TCP # Esta NATEADO? if (isflagset(5)) { log(1, "NAT: At least one participant NAT'd --> enable reply processing (hacia GW)\n"); t_on_reply("1"); if (method=="INVITE") { log(1, "NAT: Invite received --> enabling proxied media session (hacia GW)\n"); setflag(1); use_media_proxy(); }; }; rewritehostport("mygw.mydomain.com:5060"); append_hf("P-hint: GATEWAY\r\n"); if (!t_relay()) { sl_reply_error(); break; }; } #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++ # Route 4 : Llamadas a usuarios Off-line #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++ route[4] { if (!t_newtran()) { sl_reply_error(); }; if (!t_reply("404", "Not Found IT!")) { sl_reply_error(); }; break; } #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++ # On Reply Route 1 : Equipos NATeados!! #+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ +++++++++++++++++ onreply_route[1] { if ((isflagset(5) || client_nat_test("3")) && (status=~"(180)|(183)|2[0-9][0-9]")) { if (!search("^Content-Length:\ 0")) { use_media_proxy(); }; }; if (client_nat_test("1")) { fix_nated_contact(); }; } Ricardo Martinez.- _______________________________________________ Serusers mailing list serusers(a)lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers