We are using SER with the PostgreSQL module, so that SER and our Asterisk PSTN gateway and voice mail system can use the same back-end database for user records. (Anyone who's interested in how we did this, let me know -- it's still a little shaky, but it pretty much works.)
Recently we have encountered a problem: it seems that if any database error occurs, SER hangs or crashes. We have so far been able to mostly work around these conditions by being Really Careful with the database, but it is a little unsettling.
The database schema we're working from is practically identical to the MySQL schema that ships with SER. Notably for this problem the lengths of all CHARACTER VARYING fields are the same. The User-Agent field is CHARACTER VARYING(50), for instance.
So ... today boss got a copy of Xten's "eyeBeam" software, set it up to register to our SER ... and SER promptly locked up. I looked in the PostgreSQL database log and found this:
Feb 10 13:39:32 mabell postgres[13058]: [267] ERROR: value too long for type character varying(50)
Whoops. SER tried to insert eyeBeam's User-Agent data, PostgreSQL returned a data integrity error, and SER went bye-bye.
So I did a little ALTER TABLE and adjusted the field to length 128 ... then put this in ser.cfg:
# Disallow long user-agent if (search("^User-Agent: .{100,}") ) { log(1, "LOG: User agent too long\n"); sl_send_reply("479", "User agent too long!"); break; };
But this is obviously a kludge ... and doesn't fix the problem of one of the *other* fields being over-long.
Anyone have any thoughts on this? Robustness patches for the PostgreSQL module or usrloc? Better ser.cfg workarounds to protect the world from malformed or over-long SIP header fields? Should I just change all the database fields to TEXT instead of CHARACTER VARYING(whatever) ?
Did ser crash or locked up ? Usrloc has been designed to overcome database issues, thus if a database operation fails, usrloc should continue using internal memory cache.
If it crashed, could you, please, provide me with more information ? Especially SER coredump along with compiled sources would be handy.
Please contact me by private email if you need more information on getting debugging data from SER.
Jan.
On 10-02 14:47, Karl A. Krueger wrote:
We are using SER with the PostgreSQL module, so that SER and our Asterisk PSTN gateway and voice mail system can use the same back-end database for user records. (Anyone who's interested in how we did this, let me know -- it's still a little shaky, but it pretty much works.)
Recently we have encountered a problem: it seems that if any database error occurs, SER hangs or crashes. We have so far been able to mostly work around these conditions by being Really Careful with the database, but it is a little unsettling.
The database schema we're working from is practically identical to the MySQL schema that ships with SER. Notably for this problem the lengths of all CHARACTER VARYING fields are the same. The User-Agent field is CHARACTER VARYING(50), for instance.
So ... today boss got a copy of Xten's "eyeBeam" software, set it up to register to our SER ... and SER promptly locked up. I looked in the PostgreSQL database log and found this:
Feb 10 13:39:32 mabell postgres[13058]: [267] ERROR: value too long for type character varying(50)
Whoops. SER tried to insert eyeBeam's User-Agent data, PostgreSQL returned a data integrity error, and SER went bye-bye.
So I did a little ALTER TABLE and adjusted the field to length 128 ... then put this in ser.cfg:
# Disallow long user-agent if (search("^User-Agent: .{100,}") ) { log(1, "LOG: User agent too long\n"); sl_send_reply("479", "User agent too long!"); break; };But this is obviously a kludge ... and doesn't fix the problem of one of the *other* fields being over-long.
Anyone have any thoughts on this? Robustness patches for the PostgreSQL module or usrloc? Better ser.cfg workarounds to protect the world from malformed or over-long SIP header fields? Should I just change all the database fields to TEXT instead of CHARACTER VARYING(whatever) ?
-- Karl A. Krueger kkrueger@whoi.edu Network Security -- Linux/Unix Systems Support -- Etc. Woods Hole Oceanographic Institution
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
On Fri, Feb 11, 2005 at 10:26:39PM +0100, Jan Janak wrote:
Did ser crash or locked up ? Usrloc has been designed to overcome database issues, thus if a database operation fails, usrloc should continue using internal memory cache.
When this happens, it appears to me that all the ser child processes die, but one locked-up one remains. This one has to be killed with "kill -9" before ser can be restarted.
If it crashed, could you, please, provide me with more information ? Especially SER coredump along with compiled sources would be handy.
I do not have the core, but here's the log entries from the database error forward. ser was running at debug level 4.
Feb 10 13:39:32 mabell /usr/sbin/ser[13053]: PG[360] submit_query query 'insert into location (user_id,contact,expires,q,callid,cseq,replicate,flags,state,user_ agent ) values ('agaylord','sip:agaylord@128.128.64.20:8456','2005-02-10 14:39:2 3 -0500',0.00 ,'196b2f63db33fc68',2,0,0,0,'eyeBeam release 9100a stamp 1646 0 (sn:37067be159c1a125f54c)')', result 'ERROR: value too long for type characte r varying(50) ' Feb 10 13:39:32 mabell /usr/sbin/ser[13057]: DBG: tcp_main_loop: dead child 5 (s hutting down?) Feb 10 13:39:32 mabell /usr/sbin/ser[13035]: child process 13053 exited by a sig nal 11 Feb 10 13:39:32 mabell /usr/sbin/ser[13035]: core was not generated Feb 10 13:39:32 mabell /usr/sbin/ser[13035]: INFO: terminating due to SIGCHLD Feb 10 13:39:32 mabell /usr/sbin/ser[13039]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13038]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13037]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13044]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13054]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13055]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13056]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13057]: INFO: signal 15 received Feb 10 13:39:32 mabell /usr/sbin/ser[13035]: PG[272] db_close entry Feb 10 13:39:32 mabell /usr/sbin/ser[13035]: PG[272] db_close entry
After this point ser is entirely unresponsive.
I am running version 0.8.14 with no source modifications other than to install the PostgreSQL module.
On 11-02 16:29, Karl A. Krueger wrote:
On Fri, Feb 11, 2005 at 10:26:39PM +0100, Jan Janak wrote:
Did ser crash or locked up ? Usrloc has been designed to overcome database issues, thus if a database operation fails, usrloc should continue using internal memory cache.
When this happens, it appears to me that all the ser child processes die, but one locked-up one remains. This one has to be killed with "kill -9" before ser can be restarted.
That means SER crashed. Could you enable core generation using ulimit -c unlimited, it would write the core even if there is one hanging process.
Then, please, use gdb to display the backtrace and send it to me.
Jan.
Hello Mr. Krueger.
Karl A. Krueger schrieb:
We are using SER with the PostgreSQL module, so that SER and our Asterisk PSTN gateway and voice mail system can use the same back-end database for user records. (Anyone who's interested in how we did this, let me know -- it's still a little shaky, but it pretty much works.)
I'm very interested in how you did this. ;-)
Can you give me some hints, how you did this?
[...]
Feb 10 13:39:32 mabell postgres[13058]: [267] ERROR: value too long for type character varying(50)
Whoops. SER tried to insert eyeBeam's User-Agent data, PostgreSQL returned a data integrity error, and SER went bye-bye.
Is this Problem still existing in SER 0.9.3?
Regards Bastian
-
Bastian Schern -
Jan Janak -
Karl A. Krueger