[SR-Users] segmentation fault when mongodb & tls enabled. ( kamailio 4.3.2)

Daniel-Constantin Mierla miconda at gmail.com
Tue Nov 3 16:04:09 CET 2015


Hello,

I tried to reproduce here and all seems fine -- I enabled tls and
registered a user using mongodb as backend.

Can you give the parameters you set for usrloc module?

Cheers,
Daniel

On 06/10/15 19:03, Jijo wrote:
> Thanks, I have opened the issue on github tracker.
>
> regards
> jijo
>
> On Tue, Oct 6, 2015 at 9:53 AM, Daniel-Constantin Mierla
> <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>
>     Hello,
>
>     unfortunately not yet -- this requires compiling the libs and
>     currently I am traveling to USA to attend few VoIP conferences,
>     not having a good devel environment with me, access to internet is
>     limited, so connecting to a server is not easy option as well. Can
>     you open an issue on github tracker just not to forget about it?
>
>     Cheers,
>     Daniel
>
>
>     On 06/10/15 15:49, realjijo at gmail.com <mailto:realjijo at gmail.com>
>     wrote:
>>     Hi Daniel 
>>
>>     Thanks for your help .
>>
>>     Did you get a chance to look at this problem. 
>>     Please let me know how did you build  the mongo-c-driver ?
>>
>>
>>     Regards,
>>     Jijo
>>
>>     On Oct 2, 2015, at 7:21 AM, Jijo <realjijo at gmail.com
>>     <mailto:realjijo at gmail.com>> wrote:
>>
>>>     I have rebuild the official version(4.3.2) again with the
>>>     default scripts and now i'm getting diffrent core. 
>>>
>>>     This happens only when i enable TLS module in kamailio. So i
>>>     believe there is some corruption causing when TLS module in
>>>     Kamailio  and SSL in Mongo Driver  is enabled. 
>>>
>>>     I'm running with the offical kamailio script by just enabling
>>>     mongo modules. It happens for NDB as well.
>>>
>>>     How did you build the mongo driver when you implemented it? May
>>>     be i can try the same.
>>>
>>>     Please find the scripts used for testing.
>>>
>>>     Core was generated by `/usr/sbin/kamailio -P
>>>     /var/run/kamailio.pid -m 64 -M 8'.
>>>
>>>     Program terminated with signal 11, Segmentation fault.
>>>
>>>     #0  0x00007f85a21da075 in lock_udomain (_d=0x7f859c7b5f08,
>>>     _aor=0x7fff6de7ef70) at udomain.c:1017
>>>
>>>     1017lock_get(_d->table[sl].lock);
>>>
>>>     Missing separate debuginfos, use: debuginfo-install
>>>     cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
>>>     cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64
>>>     db4-4.7.25-19.el6_6.x86_64 glibc-2.12-1.166.el6_7.1.x86_64
>>>     keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-42.el6.x86_64
>>>     libcom_err-1.41.12-22.el6.x86_64
>>>     libselinux-2.0.94-5.8.el6.x86_64
>>>     nss-softokn-freebl-3.14.3-22.el6_6.x86_64
>>>     openssl-1.0.1e-42.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>>>
>>>     (gdb) bt
>>>
>>>     #0  0x00007f85a21da075 in lock_udomain (_d=0x7f859c7b5f08,
>>>     _aor=0x7fff6de7ef70) at udomain.c:1017
>>>
>>>     #1  0x00007f85a1d7069b in add_contacts (_m=0x7f85a3f054b8,
>>>     _d=0x7f859c7b5f08, _a=0x7fff6de7ef70, _mode=0, _use_regid=1) at
>>>     save.c:831
>>>
>>>     #2  0x00007f85a1d72602 in save (_m=0x7f85a3f054b8,
>>>     _d=0x7f859c7b5f08, _cflags=0, _uri=0x0) at save.c:986
>>>
>>>     #3  0x00007f85a1d5975a in w_save2 (_m=0x7f85a3f054b8,
>>>     _d=0x7f859c7b5f08 "h^{\234\205\177", _cflags=0x0) at reg_mod.c:414
>>>
>>>     #4  0x000000000041decb in do_action (h=0x7fff6de7f720,
>>>     a=0x7f85a3edd830, msg=0x7f85a3f054b8) at action.c:1059
>>>
>>>     #5  0x000000000042a553 in run_actions (h=0x7fff6de7f720,
>>>     a=0x7f85a3edd830, msg=0x7f85a3f054b8) at action.c:1548
>>>
>>>     #6  0x000000000042abb8 in run_actions_safe (h=0x7fff6de80a10,
>>>     a=0x7f85a3edd830, msg=0x7f85a3f054b8) at action.c:1613
>>>
>>>     #7  0x0000000000543d50 in rval_get_int (h=0x7fff6de80a10,
>>>     msg=0x7f85a3f054b8, i=0x7fff6de7fbf8, rv=0x7f85a3edf478,
>>>     cache=0x0) at rvalue.c:912
>>>
>>>     #8  0x0000000000547f88 in rval_expr_eval_int (h=0x7fff6de80a10,
>>>     msg=0x7f85a3f054b8, res=0x7fff6de7fbf8, rve=0x7f85a3edf470) at
>>>     rvalue.c:1906
>>>
>>>     #9  0x000000000054837e in rval_expr_eval_int (h=0x7fff6de80a10,
>>>     msg=0x7f85a3f054b8, res=0x7fff6de80080, rve=0x7f85a3edfb70) at
>>>     rvalue.c:1914
>>>
>>>     #10 0x000000000041d927 in do_action (h=0x7fff6de80a10,
>>>     a=0x7f85a3ee0270, msg=0x7f85a3f054b8) at action.c:1029
>>>
>>>     #11 0x000000000042a553 in run_actions (h=0x7fff6de80a10,
>>>     a=0x7f85a3edc990, msg=0x7f85a3f054b8) at action.c:1548
>>>
>>>     #12 0x000000000041a8c3 in do_action (h=0x7fff6de80a10,
>>>     a=0x7f85a3ebdd98, msg=0x7f85a3f054b8) at action.c:677
>>>
>>>     #13 0x000000000042a553 in run_actions (h=0x7fff6de80a10,
>>>     a=0x7f85a3ebdb08, msg=0x7f85a3f054b8) at action.c:1548
>>>
>>>     #14 0x000000000042ac80 in run_top_route (a=0x7f85a3ebdb08,
>>>     msg=0x7f85a3f054b8, c=0x0) at action.c:1634
>>>
>>>     #15 0x000000000050a9f4 in receive_msg (
>>>
>>>         buf=0xa70b00 "REGISTER sip:192.168.2.142 SIP/2.0\r\nVia:
>>>     SIP/2.0/UDP
>>>     192.168.2.119:60887;branch=z9hG4bK-524287-1---d670bd2004732b4a;rport\r\nMax-Forwards:
>>>     70\r\nContact:
>>>     <sip:usera at 192.168.2.119:60887;rinstance=d9f6274d7"..., len=534,
>>>     rcv_info=0x7fff6de80d00) at receive.c:196
>>>
>>>     #16 0x000000000060a4a6 in udp_rcv_loop () at udp_server.c:495
>>>
>>>     #17 0x00000000004a7fb3 in main_loop () at main.c:1573
>>>
>>>     #18 0x00000000004ae38e in main (argc=7, argv=0x7fff6de81138) at
>>>     main.c:2533
>>>
>>>     (gdb) quit
>>>
>>>     [root at localhost /
>>>
>>>
>>>
>>>     On Thu, Oct 1, 2015 at 12:29 PM, Daniel-Constantin Mierla
>>>     <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>>>
>>>         If you recompiled with different souce code or different
>>>         flags then the binary is no longer matching the corefile
>>>         properly.
>>>
>>>         Cheers,
>>>         Daniel
>>>
>>>
>>>         On 01/10/15 17:00, Jijo wrote:
>>>>         Why the backtrace doesn't show the timer function pointer?
>>>>         Am i missing something?
>>>>
>>>>         Im building the mongo-c-driver as below
>>>>         get the tar file
>>>>         from https://github.com/mongodb/mongo-c-driver/releases/download/1.1.11/mongo-c-driver-1.1.11.tar.gz
>>>>         run the rpm build using the spec file attached. 
>>>>         To disable the ssl, i have modified  '--enable-ssl=no' in
>>>>         the configure  command in the spec file
>>>>         %configure --disable-static --disable-silent-rules
>>>>         --enable-debug-symbols --enable-man-pages --enable-ssl=yes
>>>>         --enable-sasl --with-libbson=bundled --enable-optimizations
>>>>
>>>>         In kamailio im using db_mongodb. I'm getting same error
>>>>          for ndb_mongodb as well.
>>>>
>>>>         Please find the kamailio config scripts.
>>>>
>>>>         On Thu, Oct 1, 2015 at 10:43 AM, Daniel-Constantin Mierla
>>>>         <miconda at gmail.com <mailto:miconda at gmail.com>> wrote:
>>>>
>>>>             Hello,
>>>>
>>>>             the bracktrace is not useful.
>>>>
>>>>             How did you compile the mongo-c library before and were
>>>>             there any special parameters you set to the modules in
>>>>             kamailio config? Are you using db_mongodb or ndb_mongodb?
>>>>
>>>>             Cheers,
>>>>             Daniel
>>>>
>>>>
>>>>
>>>>             On 01/10/15 16:25, Jijo wrote:
>>>>>             Hi Daniel,
>>>>>
>>>>>             Thanks.. As a sidenote, If i disable ssl from
>>>>>             mongo-c-driver library then i don't have any crash..
>>>>>
>>>>>             Something changed in my environment. Im not able to
>>>>>             get the full backtrace. This is what i'm getting now.
>>>>>
>>>>>
>>>>>
>>>>>             aded symbols for /lib64/liblber-2.4.so.2
>>>>>             Reading symbols from
>>>>>             /lib64/libnss_dns-2.12.so...Reading symbols from
>>>>>             /usr/lib/debug/lib64/libnss_dns-2.12.so.debug...done.
>>>>>             done.
>>>>>             Loaded symbols for /lib64/libnss_dns-2.12.so
>>>>>             <http://libnss_dns-2.12.so>
>>>>>             Core was generated by
>>>>>             `/usr/local/kamailio_proxy/sbin/kamailio -f
>>>>>             /usr/local/kamailio_proxy/etc/kamail'.
>>>>>             Program terminated with signal 11, Segmentation fault.
>>>>>             #0  0x00007f69ffffffff in ?? ()
>>>>>             Missing separate debuginfos, use: debuginfo-install
>>>>>             cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
>>>>>             cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64
>>>>>             db4-4.7.25-19.el6_6.x86_64
>>>>>             keyutils-libs-1.4-5.el6.x86_64
>>>>>             krb5-libs-1.10.3-42.el6.x86_64
>>>>>             libcom_err-1.41.12-22.el6.x86_64
>>>>>             libcurl-7.19.7-46.el6.x86_64 libidn-1.18-2.el6.x86_64
>>>>>             libselinux-2.0.94-5.8.el6.x86_64
>>>>>             libssh2-1.4.2-1.el6_6.1.x86_64
>>>>>             libunistring-0.9.3-5.el6.x86_64
>>>>>             libxml2-2.7.6-20.el6.x86_64
>>>>>             mysql-libs-5.1.73-5.el6_6.x86_64
>>>>>             nspr-4.10.8-1.el6_6.x86_64 nss-3.18.0-5.3.el6_6.x86_64
>>>>>             nss-util-3.18.0-1.el6_6.x86_64
>>>>>             openldap-2.4.40-5.el6.x86_64
>>>>>             openssl-1.0.1e-42.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>>>>>             (gdb) bt
>>>>>             #0  0x00007f69ffffffff in ?? ()
>>>>>             #1  0x00000000005fef8e in compat_old_handler
>>>>>             (ti=69660042, tl=0x7f693e736d50, data=0x7f693e736d50)
>>>>>             at timer.c:996
>>>>>             #2  0x00000000005ff94c in slow_timer_main () at
>>>>>             timer.c:1130
>>>>>             #3  0x00000000004a8676 in main_loop () at main.c:1628
>>>>>             #4  0x00000000004ae38e in main (argc=15,
>>>>>             argv=0x7ffdd0dc8848) at main.c:2533
>>>>>             (gdb) 
>>>>>
>>>>>
>>>>>
>>>>>             On Thu, Oct 1, 2015 at 2:37 AM, Daniel-Constantin
>>>>>             Mierla <miconda at gmail.com <mailto:miconda at gmail.com>>
>>>>>             wrote:
>>>>>
>>>>>                 The first backtrace is related to pike. Can you
>>>>>                 get from gdb of fisrt trace:
>>>>>
>>>>>                 frame 2
>>>>>                 p root->entries[b]
>>>>>
>>>>>                 Cheers,
>>>>>                 Daniel
>>>>>
>>>>>
>>>>>                 On 30/09/15 16:19, Jijo wrote:
>>>>>>                 I'm observing a segmentation fault when mongodb &
>>>>>>                 tls enabled. It doesn't happen when one of the
>>>>>>                 module is disabled.
>>>>>>                 OS: centos 6.7   kamailio 4.3.2  mongo-c-driver
>>>>>>                 version 1.1.10.
>>>>>>
>>>>>>                 It happens immediately after i register a
>>>>>>                 subscriber. The location table is updated with
>>>>>>                 the new data and in parallel core is generated as
>>>>>>                 well.
>>>>>>
>>>>>>
>>>>>>
>>>>>>                 Loaded symbols for /lib64/libnss_dns-2.12.so
>>>>>>                 <http://libnss_dns-2.12.so>
>>>>>>                 Core was generated by
>>>>>>                 `/usr/local/kamailio_proxy/sbin/kamailio -f
>>>>>>                 /usr/local/kamailio_proxy/etc/kamail'.
>>>>>>                 Program terminated with signal 11, Segmentation
>>>>>>                 fault.
>>>>>>                 #0  0x00007fd3adf65e69 in atomic_cmpxchg_int
>>>>>>                 (var=0x56444e3055445030, old=0, new_v=1) at
>>>>>>                 ../../atomic/atomic_x86.h:233
>>>>>>                 233ATOMIC_FUNC_CMPXCHG(cmpxchg, "cmpxchgl %2,
>>>>>>                 %1", int , int)
>>>>>>                 Missing separate debuginfos, use:
>>>>>>                 debuginfo-install
>>>>>>                 cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
>>>>>>                 cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64
>>>>>>                 db4-4.7.25-19.el6_6.x86_64
>>>>>>                 keyutils-libs-1.4-5.el6.x86_64
>>>>>>                 krb5-libs-1.10.3-42.el6.x86_64
>>>>>>                 libcom_err-1.41.12-22.el6.x86_64
>>>>>>                 libcurl-7.19.7-46.el6.x86_64
>>>>>>                 libidn-1.18-2.el6.x86_64
>>>>>>                 libselinux-2.0.94-5.8.el6.x86_64
>>>>>>                 libssh2-1.4.2-1.el6_6.1.x86_64
>>>>>>                 libunistring-0.9.3-5.el6.x86_64
>>>>>>                 libxml2-2.7.6-20.el6.x86_64
>>>>>>                 mysql-libs-5.1.73-5.el6_6.x86_64
>>>>>>                 nspr-4.10.8-1.el6_6.x86_64
>>>>>>                 nss-3.18.0-5.3.el6_6.x86_64
>>>>>>                 nss-util-3.18.0-1.el6_6.x86_64
>>>>>>                 openldap-2.4.40-5.el6.x86_64
>>>>>>                 openssl-1.0.1e-42.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>>>>>>                 (gdb) bt
>>>>>>                 #0  0x00007fd3adf65e69 in atomic_cmpxchg_int
>>>>>>                 (var=0x56444e3055445030, old=0, new_v=1) at
>>>>>>                 ../../atomic/atomic_x86.h:233
>>>>>>                 #1  0x00007fd3adf65eb6 in futex_get
>>>>>>                 (lock=0x56444e3055445030) at ../../futexlock.h:99
>>>>>>                 #2  0x00007fd3adf66468 in prv_lock_tree_branch
>>>>>>                 (b=192 '\300') at ip_tree.c:47
>>>>>>                 #3  0x00007fd3adf664e5 in lock_tree_branch (b=192
>>>>>>                 '\300') at ip_tree.c:66
>>>>>>                 #4  0x00007fd3adf6dda6 in swap_routine
>>>>>>                 (ticks=123206032, param=0x0) at pike_funcs.c:293
>>>>>>                 #5  0x00000000005fef8e in compat_old_handler
>>>>>>                 (ti=1971296512, tl=0x7fd32ccea430,
>>>>>>                 data=0x7fd32ccea430) at timer.c:996
>>>>>>                 #6  0x00000000005ff94c in slow_timer_main () at
>>>>>>                 timer.c:1130
>>>>>>                 #7  0x00000000004a8676 in main_loop () at main.c:1628
>>>>>>                 #8  0x00000000004ae38e in main (argc=11,
>>>>>>                 argv=0x7fff1d97ec28) at main.c:2533
>>>>>>                 (gdb) 
>>>>>>
>>>>>>
>>>>>>                 On Clean up after the core, mongo db does
>>>>>>                 segmentation fault as well
>>>>>>
>>>>>>                 Loaded symbols for /lib64/libnss_files-2.12.so
>>>>>>                 <http://libnss_files-2.12.so>
>>>>>>                 Reading symbols from
>>>>>>                 /lib64/libnss_dns-2.12.so...Reading symbols from
>>>>>>                 /usr/lib/debug/lib64/libnss_dns-2.12.so.debug...done.
>>>>>>                 done.
>>>>>>                 Loaded symbols for /lib64/libnss_dns-2.12.so
>>>>>>                 <http://libnss_dns-2.12.so>
>>>>>>                 Core was generated by
>>>>>>                 `/usr/local/kamailio_proxy/sbin/kamailio -f
>>>>>>                 /usr/local/kamailio_proxy/etc/kamail'.
>>>>>>                 Program terminated with signal 11, Segmentation
>>>>>>                 fault.
>>>>>>                 #0  __pthread_mutex_destroy
>>>>>>                 (mutex=0x7fd32cce7be8) at pthread_mutex_destroy.c:28
>>>>>>                 28 if ((mutex->__data.__kind &
>>>>>>                 PTHREAD_MUTEX_ROBUST_NORMAL_NP) == 0
>>>>>>                 Missing separate debuginfos, use:
>>>>>>                 debuginfo-install
>>>>>>                 cyrus-sasl-lib-2.1.23-15.el6_6.2.x86_64
>>>>>>                 cyrus-sasl-plain-2.1.23-15.el6_6.2.x86_64
>>>>>>                 db4-4.7.25-19.el6_6.x86_64
>>>>>>                 keyutils-libs-1.4-5.el6.x86_64
>>>>>>                 krb5-libs-1.10.3-42.el6.x86_64
>>>>>>                 libcom_err-1.41.12-22.el6.x86_64
>>>>>>                 libcurl-7.19.7-46.el6.x86_64
>>>>>>                 libidn-1.18-2.el6.x86_64
>>>>>>                 libselinux-2.0.94-5.8.el6.x86_64
>>>>>>                 libssh2-1.4.2-1.el6_6.1.x86_64
>>>>>>                 libunistring-0.9.3-5.el6.x86_64
>>>>>>                 libxml2-2.7.6-20.el6.x86_64
>>>>>>                 mysql-libs-5.1.73-5.el6_6.x86_64
>>>>>>                 nspr-4.10.8-1.el6_6.x86_64
>>>>>>                 nss-3.18.0-5.3.el6_6.x86_64
>>>>>>                 nss-util-3.18.0-1.el6_6.x86_64
>>>>>>                 openldap-2.4.40-5.el6.x86_64
>>>>>>                 openssl-1.0.1e-42.el6.x86_64 zlib-1.2.3-29.el6.x86_64
>>>>>>                 (gdb) bt
>>>>>>                 #0  __pthread_mutex_destroy
>>>>>>                 (mutex=0x7fd32cce7be8) at pthread_mutex_destroy.c:28
>>>>>>                 #1  0x00007fd32c86f9e6 in
>>>>>>                 _mongoc_ssl_thread_cleanup () at
>>>>>>                 src/mongoc/mongoc-ssl.c:555
>>>>>>                 #2  _mongoc_ssl_cleanup () at
>>>>>>                 src/mongoc/mongoc-ssl.c:106
>>>>>>                 #3  0x00007fd32c8606a9 in _mongoc_do_cleanup ()
>>>>>>                 at src/mongoc/mongoc-init.c:127
>>>>>>                 #4  0x000000344480cdb3 in pthread_once () at
>>>>>>                 ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_once.S:104
>>>>>>                 #5  0x00007fd32c8523df in __do_global_dtors_aux
>>>>>>                 () from /usr/lib64/libmongoc-1.0.so.0.0.0
>>>>>>                 #6  0x0000000000000000 in ?? ()
>>>>>>                 (gdb) 
>>>>>>
>>>>>>
>>>>>>
>>>>>>                 mongo-c-driver version 1.1.10
>>>>>>
>>>>>>                 version: kamailio 4.3.2 (x86_64/linux) b5980b-dirty
>>>>>>                 flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP,
>>>>>>                 TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE,
>>>>>>                 USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP,
>>>>>>                 PKG_MALLOC, F_MALLOC, DBG_F_MALLOC, USE_FUTEX,
>>>>>>                 FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE,
>>>>>>                 USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST,
>>>>>>                 HAVE_RESOLV_RES
>>>>>>                 ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE
>>>>>>                 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024,
>>>>>>                 BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
>>>>>>                 poll method support: poll, epoll_lt, epoll_et,
>>>>>>                 sigio_rt, select.
>>>>>>                 id: b5980b -dirty
>>>>>>                 compiled on 13:35:24 Sep 28 2015 with gcc 4.
>>>>>>
>>>>>>
>>>>>>                 _______________________________________________
>>>>>>                 SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list
>>>>>>                 sr-users at lists.sip-router.org
>>>>>>                 <mailto:sr-users at lists.sip-router.org>
>>>>>>                 http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>>>>
>>>>>                 -- 
>>>>>                 Daniel-Constantin Mierla
>>>>>                 http://twitter.com/#!/miconda
>>>>>                 <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda
>>>>>                 Book: SIP Routing With Kamailio - http://www.asipto.com
>>>>>                 Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat
>>>>>
>>>>>
>>>>>                 _______________________________________________
>>>>>                 SIP Express Router (SER) and Kamailio (OpenSER) -
>>>>>                 sr-users mailing list
>>>>>                 sr-users at lists.sip-router.org
>>>>>                 <mailto:sr-users at lists.sip-router.org>
>>>>>                 http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
>>>>>
>>>>>
>>>>
>>>>             -- 
>>>>             Daniel-Constantin Mierla
>>>>             http://twitter.com/#!/miconda
>>>>             <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda
>>>>             Book: SIP Routing With Kamailio - http://www.asipto.com
>>>>             Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat
>>>>
>>>>
>>>
>>>         -- 
>>>         Daniel-Constantin Mierla
>>>         http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda
>>>         Book: SIP Routing With Kamailio - http://www.asipto.com
>>>         Kamailio Advanced Training, Sep 28-30, 2015, in Berlin - http://asipto.com/u/kat
>>>
>>>
>>>     <kamailio-local.cfg>
>>>     <kamailio.cfg>
>
>     -- 
>     Daniel-Constantin Mierla
>     http://twitter.com/#!/miconda <http://twitter.com/#%21/miconda> - http://www.linkedin.com/in/miconda
>     Book: SIP Routing With Kamailio - http://www.asipto.com
>
>

-- 
Daniel-Constantin Mierla
http://twitter.com/#!/miconda - http://www.linkedin.com/in/miconda
Book: SIP Routing With Kamailio - http://www.asipto.com
Kamailio Advanced Training, Nov 30-Dec 2, Berlin - http://asipto.com/kat

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sip-router.org/pipermail/sr-users/attachments/20151103/adccf8a4/attachment.html>


More information about the sr-users mailing list