Hey,
On 13.05.2011 11:11, Timo Reimann wrote:
On 12.05.2011 15:55, Anton Roman wrote:
my answer is inline:
2011/5/12 Timo Reimann <timo.reimann@1und1.de mailto:timo.reimann@1und1.de> As to the reason of the segfault, the dialog structure or hash table may already be gone when unref_dlg() is called. Can you go to stack #0 and tell us what the value of each of the following data structures is (use "p <data structure> in gdb):
*dlg d_table d_table->entriesHere you have:
(gdb) p *dlg $1 = {ref = 793790803, next = 0xa0d4b4f20303032, prev = 0x504953203a616956, h_id = 808333871, h_entry = 1346655535, state = 775174432, lifetime = 841888562, start_ts = 892219952, dflags = 808794678, sflags = 1648046134, toroute = 1668178290, toroute_name = { s = 0x62344768397a3d68 <Address 0x62344768397a3d68 out of bounds>, len = 946221643}, from_rr_nb = 1886534457, tl = { next = 0x72460a0d30363035, prev = 0x6f6e4122203a6d6f, timeout = 1869445486}, callid = { s = 0x6f6e613a7069733c <Address 0x6f6e613a7069733c out of bounds>, len = 1869445486}, from_uri = { s = 0x3230322e33322e34 <Address 0x3230322e33322e34 out of bounds>, len = 1043739950}, to_uri = {
[...]
As I suspected, your dialog seems outdated already: The reference count is 793790803, and the Call-ID is supposed to have a rough 2 billions characters. That's what I call unique. :)
I could ask you for more details on the dump but it'd probably be easiest if I could take a direct (gdb-)look at it. Would you mind sending it to me in private (i.e., no CC to the mailing list) to the address I am writing from?
I (and Marius -- credits!) digged through your coredump and found a few curiosities. Before I bug you with the details, let me just say this: There might be something wrong the dialog reference counter that determines when a dialog is a to be removed from the hash table. In fact, your call stack indicates that an unreference operation was attempted on a hash table which looks empty:
(gdb) frame 0 #0 unref_dlg (dlg=0x7f08a9f67da8, cnt=1) at dlg_hash.c:598 598 dlg_lock( d_table, d_entry); (gdb) p *d_table->entries $53 = {first = 0x0, last = 0x0, next_id = 1124074261, lock_idx = 0}
Looking through the mailing-list archive, I noticed you brought attention to another reference counter-related bug which Daniel provided a fix for with commit 2c28a251a. Since you reported that no more issues appeared with that fixed version, I just backported the patch into 3.1. However, I can see from your core dump that you are not using a Kamailio version that includes the fix.
Before we continue with any bug hunting, could you try a version of Kamailio that comes with Daniel's "safer unref of terminated dialogs" patch? This can be master branch copy or a recent copy of the 3.1 git branch. I'd suggest the latter so we can ensure that no bleeding-edge features added to the dialog module distort our analysis.
Thanks and
Cheers,
--Timo