Hi Bogdan, the problem is that in the logs i have a huge quantity of things and i searched with grep for:

ping child nathelper

But i don't know exactly what should be written in the log about this kind of problem. On the other side i'm not a coder so i never used gdb, if you can give me some hint i will be very happy to do it.

thanks, Bye, Marcello

Bogdan-Andrei Iancu wrote:

Hi Marcello,

there is no limit....so...the logs were clean?....if happens again, try to connect with gdb to the timer process to see what it is doing....

regards, bogdan

Marcello Lupo wrote:

...

Hi to all, i have openser 1.2.1 active from some months.. Suddenly on yesterday the natping stop to work. I have 140 contacts in user location almost all of them natted. There is some limit to user in location table at which the ping stops or a particular server load? There is a way to have something in the logs about it?? I didn't found anything in the logs.

This is my config:

modparam("nathelper", "natping_interval", 30) modparam("nathelper", "ping_nated_only", 1)

With a restart of openser all come back to work but i'm afraid if it will happen again?

Hi Bogdan, you were right. I searched for memory and found a lot of:

/data/openser/sbin/openser[2547]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2547]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2541]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2547]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory

Now i enabled again the memlog and will wait the the problem happen again to let you have a memory dump. In the meanwhile if you have some other ideas on how to solve it it is welcome. Thank you, Bye, Marcello

Bogdan-Andrei Iancu wrote:

Hi Marcello,

the problem may be related to private memory size - the communication between the usrloc and nathelper modules is done via private memory buffer.....maybe there was no more mem available (for whatever reason)...look for memory errors from the usrloc module.

Regards, Bogdan

Hi Bogdan, i left the values as default. I read that section you suggested to me already last time that i got memory leak, if you remember. I didn't found, probably i didn't searched well, specification on how to dimension my private memory based on usrloc entry to do natping. If you have something to point me at... Thank you in the meanwhile. Bye, Marcello

Bogdan-Andrei Iancu wrote:

Hi Marcello,

see the memory troubleshooting section on wiki to try to get an idea why the private memory is not enough. Is is a memory shortage (how much mem do you configured)? or is it a mem leak?

Regards, Bogdan

Marcello Lupo wrote:

...

Hi Bogdan, you were right. I searched for memory and found a lot of:

/data/openser/sbin/openser[2547]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2547]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2541]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2547]: ERROR: build_req_buf_from_sip_req: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory /data/openser/sbin/openser[2563]: ERROR:nathelper:nh_timer: out of memory

Now i enabled again the memlog and will wait the the problem happen again to let you have a memory dump. In the meanwhile if you have some other ideas on how to solve it it is welcome. Thank you, Bye, Marcello

Hi Bogdan,

this are my modules loaded:

loadmodule "/data/openser/lib/openser/modules/mi_fifo.so" loadmodule "/data/openser/lib/openser/modules/mysql.so" loadmodule "/data/openser/lib/openser/modules/sl.so" loadmodule "/data/openser/lib/openser/modules/tm.so" loadmodule "/data/openser/lib/openser/modules/rr.so" loadmodule "/data/openser/lib/openser/modules/maxfwd.so" loadmodule "/data/openser/lib/openser/modules/usrloc.so" loadmodule "/data/openser/lib/openser/modules/registrar.so" loadmodule "/data/openser/lib/openser/modules/auth.so" loadmodule "/data/openser/lib/openser/modules/auth_db.so" loadmodule "/data/openser/lib/openser/modules/uri.so" loadmodule "/data/openser/lib/openser/modules/uri_db.so" loadmodule "/data/openser/lib/openser/modules/alias_db.so" loadmodule "/data/openser/lib/openser/modules/nathelper.so" loadmodule "/data/openser/lib/openser/modules/textops.so" loadmodule "/data/openser/lib/openser/modules/avpops.so" loadmodule "/data/openser/lib/openser/modules/domain.so" loadmodule "/data/openser/lib/openser/modules/permissions.so" loadmodule "/data/openser/lib/openser/modules/xlog.so" loadmodule "/data/openser/lib/openser/modules/uac.so" loadmodule "/data/openser/lib/openser/modules/acc.so" loadmodule "/data/openser/lib/openser/modules/exec.so"

Hi attach to you a link where to get the timer process memory log, it was running from 2 hours when i activated the memory log on the system.

http://82.193.22.211/log_timer.txt

If you can check it for possible memory leak because i'm not able to do it.

Thank you again, Bye, Marcello

Bogdan-Andrei Iancu wrote:

Well....there is no such documentation for how to dimension private memory due nat ping....normally the default memory should be quite enough for couple of Ks of pings.....but probably it depends of what other modules/services you are running (so what other private mem consumers you have)....

use the memory debugger to see if there is a mem leak in the private mem of the timer process....if you need help with this, let me know.

Regards, Bogdan

Marcello Lupo wrote:

...

Hi Bogdan, i left the values as default. I read that section you suggested to me already last time that i got memory leak, if you remember. I didn't found, probably i didn't searched well, specification on how to dimension my private memory based on usrloc entry to do natping. If you have something to point me at... Thank you in the meanwhile. Bye, Marcello

Hi Marcello,

I haven;t looked on the yesterday log, but this from today doesn't look from the timer process ...please confirm this..

Anyhow, the dump does not show any mem leak. Just probably a large script and a lot of avps/var used :)..

Please check if it is really the timer process (use "openserctl ps" to check it ). Also wait for the error from the timer process (for pkg mem).

regards, bogdan

Marcello Lupo wrote:

Hi Bogdan, i don't know if you got the time to analyze the log i sent you yesterday but today i have another log of memory from one process in out of memory that is not the timer process.

It is at:

http://82.193.22.211/log_memory_leak_20070904.txt

It is in out of pkg memory. Can you check it can be memory leak or not?? Anyway i increaased the pkg memory to 4 Mb and recompiled this morning. Thank You, Bye, Marcello

Bogdan-Andrei Iancu wrote:

...

Well....there is no such documentation for how to dimension private memory due nat ping....normally the default memory should be quite enough for couple of Ks of pings.....but probably it depends of what other modules/services you are running (so what other private mem consumers you have)....

use the memory debugger to see if there is a mem leak in the private mem of the timer process....if you need help with this, let me know.

Regards, Bogdan

Marcello Lupo wrote:

...

Hi Bogdan, i left the values as default. I read that section you suggested to me already last time that i got memory leak, if you remember. I didn't found, probably i didn't searched well, specification on how to dimension my private memory based on usrloc entry to do natping. If you have something to point me at... Thank you in the meanwhile. Bye, Marcello

Bogdan-Andrei Iancu wrote:

...

Hi Marcello,

see the memory troubleshooting section on wiki to try to get an idea why the private memory is not enough. Is is a memory shortage (how much mem do you configured)? or is it a mem leak?

Regards, Bogdan

Hi Marcello,

I just looked over the first trace, but I cannot tell if it is the timer process or not (there is only the mem dump). But definitely there is no memory leak.

As it looks you have a quite large and complex script (based on the memory used by it), I suggest to increase the private memory size.

regards, Bogdan

Marcello Lupo wrote:

Hi Bogdan,

yes i confirm to you that it is NOT timer process, it is a process went in out of memory this morning.

And what about the log of timer process of yesterday??

Thank you Bogdan, Bye, Marcello

Bogdan-Andrei Iancu wrote:

...

Hi Marcello,

I haven;t looked on the yesterday log, but this from today doesn't look from the timer process ...please confirm this..

Anyhow, the dump does not show any mem leak. Just probably a large script and a lot of avps/var used :)..

Please check if it is really the timer process (use "openserctl ps" to check it ). Also wait for the error from the timer process (for pkg mem).

regards, bogdan

Marcello Lupo wrote:

...

Hi Bogdan, i don't know if you got the time to analyze the log i sent you yesterday but today i have another log of memory from one process in out of memory that is not the timer process.

It is at:

http://82.193.22.211/log_memory_leak_20070904.txt

It is in out of pkg memory. Can you check it can be memory leak or not?? Anyway i increaased the pkg memory to 4 Mb and recompiled this morning. Thank You, Bye, Marcello

Hi Marcello,

I would say that the 4M of private mem per process should me more than enough. Let me know if the problem is solved.

Regards, Bogdan

Marcello Lupo wrote:

Hi Bogdan, yes , the first log was timer process and i confirm you that i have a quite large and complex script. More over i had never had problems with the same logic of the script on Openser 1.0.0 . Now i converted it to the logic of 1.2.1 and this server have almost all his clients behind nat with natping enabled and rtpproxy that the old 1.0.0 had not. So i think it can be a memory usage problem. Now i increased the memory to 4*1024*1024 . Do you think it is enough? Thank you anyway for all the support. Bye, Marcello

Bogdan-Andrei Iancu wrote:

...

Hi Marcello,

I just looked over the first trace, but I cannot tell if it is the timer process or not (there is only the mem dump). But definitely there is no memory leak.

As it looks you have a quite large and complex script (based on the memory used by it), I suggest to increase the private memory size.

regards, Bogdan

Marcello Lupo wrote:

...

Hi Bogdan,

yes i confirm to you that it is NOT timer process, it is a process went in out of memory this morning.

And what about the log of timer process of yesterday??

Thank you Bogdan, Bye, Marcello

Bogdan-Andrei Iancu wrote:

...

Hi Marcello,

I haven;t looked on the yesterday log, but this from today doesn't look from the timer process ...please confirm this..

Anyhow, the dump does not show any mem leak. Just probably a large script and a lot of avps/var used :)..

Please check if it is really the timer process (use "openserctl ps" to check it ). Also wait for the error from the timer process (for pkg mem).

regards, bogdan

Marcello Lupo wrote:

...

Hi Bogdan, i don't know if you got the time to analyze the log i sent you yesterday but today i have another log of memory from one process in out of memory that is not the timer process.

It is at:

http://82.193.22.211/log_memory_leak_20070904.txt

It is in out of pkg memory. Can you check it can be memory leak or not?? Anyway i increaased the pkg memory to 4 Mb and recompiled this morning. Thank You, Bye, Marcello