On 4/22/13 5:30 PM, Juha Heinanen wrote:
Andreas Granig writes:
Let me know if this is stupid and/or a complete overkill, but what about introducing some kind of dummy mode, where you'd pipe a message into kamailio via stdin, and get the resulting message out on stdout (e.g. in ngrep style with ip information as first line, plus the content following), plus a dump of internals (e.g. vars, avps) on stderr as they're being assigned, and once the message is processed, kamailio would just shut down again?
in addition to the message, you would need to be able to tell which ip addr it is coming from and which ip/port in kamailio it is going to. also, on the output side, kamailio would need to tell which proto/ip/port it would use to send the message out.
Details about going out can be printed on onsend_route. In this route block, one can execute drop and nothing is sent to the wire. It can be a config started with a special define specified with -A parameter.
Enabling debugger module with cfg trace should give valuable information about what has been executed from config.
From what Andreas suggesting, printing the value of variables as they are assigned is missing, probably can be added by hooking in the interpreter when doing the assignment operation.
Grouping all above under some global/command line parameter can be useful to make it easier to do a dry run.
Cheers, Daniel
22 apr 2013 kl. 19:34 skrev Daniel-Constantin Mierla miconda@gmail.com:
On 4/22/13 5:30 PM, Juha Heinanen wrote:
Andreas Granig writes:
Let me know if this is stupid and/or a complete overkill, but what about introducing some kind of dummy mode, where you'd pipe a message into kamailio via stdin, and get the resulting message out on stdout (e.g. in ngrep style with ip information as first line, plus the content following), plus a dump of internals (e.g. vars, avps) on stderr as they're being assigned, and once the message is processed, kamailio would just shut down again?
in addition to the message, you would need to be able to tell which ip addr it is coming from and which ip/port in kamailio it is going to. also, on the output side, kamailio would need to tell which proto/ip/port it would use to send the message out.
Details about going out can be printed on onsend_route. In this route block, one can execute drop and nothing is sent to the wire. It can be a config started with a special define specified with -A parameter.
Enabling debugger module with cfg trace should give valuable information about what has been executed from config.
From what Andreas suggesting, printing the value of variables as they are assigned is missing, probably can be added by hooking in the interpreter when doing the assignment operation.
Grouping all above under some global/command line parameter can be useful to make it easier to do a dry run.
We could steal an idea from ASterisk, where we have a directory that asterisk monitors for "call files".
We could have a module that monitors a directory for SIP messages and injects them into the routing script. The file could have a few lines of metadata, like faking transport and sender's IP and port, an empty line and then a full SIP message.
/O
Hi,
On 04/22/2013 07:34 PM, Daniel-Constantin Mierla wrote:
Details about going out can be printed on onsend_route. In this route block, one can execute drop and nothing is sent to the wire. It can be a config started with a special define specified with -A parameter.
Enabling debugger module with cfg trace should give valuable information about what has been executed from config.
From what Andreas suggesting, printing the value of variables as they are assigned is missing, probably can be added by hooking in the interpreter when doing the assignment operation.
Grouping all above under some global/command line parameter can be useful to make it easier to do a dry run.
Alright, thanks, so there seem to be some possibilities to do what I want. I'm going to check the possible options (debugger, onsend-route with drop, passing defines on startup to control the behaviour without changing the config between real operations and dry runs) and will provide some findings and proof of concepts, before digging into developing missing parts.
Andreas
2013/4/22 Daniel-Constantin Mierla miconda@gmail.com:
From what Andreas suggesting, printing the value of variables as they are assigned is missing, probably can be added by hooking in the interpreter when doing the assignment operation.
Can you point me to the right direction here? What are the files involved?
On 5/23/13 10:47 AM, Victor Seva wrote:
2013/4/22 Daniel-Constantin Mierla miconda@gmail.com:
From what Andreas suggesting, printing the value of variables as they are assigned is missing, probably can be added by hooking in the interpreter when doing the assignment operation.
Can you point me to the right direction here? What are the files involved?
Look for ASSIGN_T in core, one place to go to and check is lval_assign(...) function, not sure there are other in core.
Then some vars are set by module functions, might be hard to spot as there are cases when avps/xavps are added directly in the list or some cases when pv api is used.
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
Look for ASSIGN_T in core, one place to go to and check is lval_assign(...) function, not sure there are other in core.
Ok. First attempt to get this done.
I've created a new core cfg parameter "log_assign_actions" to activate/deactivate de log. No problem there.
I've created to new functions on lvalue.c in order to log the assign action on lval_assign()
questions: - How can I get the name of the pvar if I only have pv_spec_t struct? - Why is always called log_assign_action_pvar() with this kamailio.cfg example?
xdbg("test assign_action: $$var(temp) = $$fu;\n"); $var(temp) = $fu; xdbg("test assign_action: $$var(temp) = $$null;\n"); $var(temp) = $null; xdbg("test assign_action: $$var(temp) = 2;\n"); $var(temp) = 2; xdbg("test assign_action: $$avp(s:temp_avp) = $$si;\n"); $avp(s:temp_avp) = $si; xdbg("test assign_action: $$avp(s:temp_avp) = "hi";\n"); $avp(s:temp_avp) = "hi"; xdbg("test assign_action: $$avp(s:temp_avp) = 3;\n"); $avp(s:temp_avp) = 3;
4(26395) DEBUG: <script>: test assign_action: $var(temp) = $fu; 4(26395) DEBUG: <core> [parser/parse_addr_spec.c:885]: parse_addr_spec(): end of header reached, state=10 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 4 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): sip:janakj@dhcp246.fokus.gmd.de 4(26395) DEBUG: <script>: test assign_action: $var(temp) = $null; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 28 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 0 4(26395) DEBUG: <script>: test assign_action: $var(temp) = 2; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 28 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 2 4(26395) DEBUG: <script>: test assign_action: $avp(s:temp_avp) = $si; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 4 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 127.0.0.1 4(26395) DEBUG: <script>: test assign_action: $avp(s:temp_avp) = "hi"; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 4 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): hi 4(26395) DEBUG: <script>: test assign_action: $avp(s:temp_avp) = 3; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 28 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 3
Thanks in advance, Victor
First a general recommendation: try to avoid patching core that much for non-common use cases.
The pv spec does not store the pv name in most of the cases, storing it there will increase size of structure and it is not needed in typical use cases.
At this moment, I think a good solution could be: - most of the code in debugger module - in core only a hook for a callback function for assignment operations that will print the desired info - update the interpreter to use pv cache instead of own spec per pv (I can do it, being in my list and hopefully is no big change) - if this feature is enabled in debugger module, it will create a hash table where to index pv set functions by pointer and point to the cache item - this index has to be created in child_init with rank PROC_INIT to be sure all pvs were resolved to cache item - not being a large set of writable variables, but also a debug case, looking up the pointer in hash table should be fast enough
Where to look: - adding a callback in the core - see how the callback to get the per module debug level was added - hash table - a hash table was created for per module debug level as well in debugger module - pvapi.c show the structure of pv cache items
Cheers, Daniel
On 5/23/13 5:29 PM, Victor Seva wrote:
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
Look for ASSIGN_T in core, one place to go to and check is lval_assign(...) function, not sure there are other in core.
Ok. First attempt to get this done.
I've created a new core cfg parameter "log_assign_actions" to activate/deactivate de log. No problem there.
I've created to new functions on lvalue.c in order to log the assign action on lval_assign()
questions:
How can I get the name of the pvar if I only have pv_spec_t struct?
Why is always called log_assign_action_pvar() with this kamailio.cfg example?
xdbg("test assign_action: $$var(temp) = $$fu;\n"); $var(temp) = $fu; xdbg("test assign_action: $$var(temp) = $$null;\n"); $var(temp) = $null; xdbg("test assign_action: $$var(temp) = 2;\n"); $var(temp) = 2; xdbg("test assign_action: $$avp(s:temp_avp) = $$si;\n"); $avp(s:temp_avp) = $si; xdbg("test assign_action: $$avp(s:temp_avp) = \"hi\";\n"); $avp(s:temp_avp) = "hi"; xdbg("test assign_action: $$avp(s:temp_avp) = 3;\n"); $avp(s:temp_avp) = 3;
4(26395) DEBUG: <script>: test assign_action: $var(temp) = $fu; 4(26395) DEBUG: <core> [parser/parse_addr_spec.c:885]:
parse_addr_spec(): end of header reached, state=10 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 4 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): sip:janakj@dhcp246.fokus.gmd.de 4(26395) DEBUG: <script>: test assign_action: $var(temp) = $null; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 28 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 0 4(26395) DEBUG: <script>: test assign_action: $var(temp) = 2; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 28 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 2 4(26395) DEBUG: <script>: test assign_action: $avp(s:temp_avp) = $si; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 4 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 127.0.0.1 4(26395) DEBUG: <script>: test assign_action: $avp(s:temp_avp) = "hi"; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 4 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): hi 4(26395) DEBUG: <script>: test assign_action: $avp(s:temp_avp) = 3; 4(26395) DEBUG: <core> [lvalue.c:397]: log_assign_action_pvar(): value.flags: 28 4(26395) DEBUG: <core> [lvalue.c:401]: log_assign_action_pvar(): $var(unknown): 3
Thanks in advance, Victor
SIP Express Router (SER) and Kamailio (OpenSER) - sr-users mailing list sr-users@lists.sip-router.org http://lists.sip-router.org/cgi-bin/mailman/listinfo/sr-users
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
First a general recommendation: try to avoid patching core that much for non-common use cases.
Sure. I was just trying to see how this could be done. I don't want to mess with the core. :-)
[snip]
At this moment, I think a good solution could be:
[snip]
- update the interpreter to use pv cache instead of own spec per pv (I can
do it, being in my list and hopefully is no big change)
I will try to do it myself just to get familliar with this area. Let's see how it goes.
- if this feature is enabled in debugger module, it will create a hash table
where to index pv set functions by pointer and point to the cache item
- this index has to be created in child_init with rank PROC_INIT to be sure
all pvs were resolved to cache item
Sorry. But I don't get this. Can you, please, explain me the rationale behind this hash table?.
Thank you for your help!
Cheers, Victor
On 5/23/13 10:45 PM, Victor Seva wrote:
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
First a general recommendation: try to avoid patching core that much for non-common use cases.
Sure. I was just trying to see how this could be done. I don't want to mess with the core. :-)
[snip]
At this moment, I think a good solution could be:
[snip]
- update the interpreter to use pv cache instead of own spec per pv (I can
do it, being in my list and hopefully is no big change)
I will try to do it myself just to get familliar with this area. Let's see how it goes.
- if this feature is enabled in debugger module, it will create a hash table
where to index pv set functions by pointer and point to the cache item
- this index has to be created in child_init with rank PROC_INIT to be sure
all pvs were resolved to cache item
Sorry. But I don't get this. Can you, please, explain me the rationale behind this hash table?.
First about the pv_cache - it is a hash table with specs of pvs used in config file. Not all pvs are using the cache yet, but should be migrate to it. The nebefit is that if a same pv is used more than one, only one spec is stored in cache. For example, if there will be no pv cache used, if you have 10 of $ru, used memory is 20*sizeof(pv_spec). If pv cache will be used everywhere, the will be one pv_spec and 10 pointers to it. Because the size of pvspec is much larger than pointer size, it obviously results in lower memory usage. More that that, if one needs to parse a pv name at runtime, will not allocate memory each time, but only first time when added to cache, avoiding memory leak because some pv_spec have dynamic structure and no free function.
The pv_cache contains items of:
{ pvname pvspec ... }
A lookup by pvname in pv_cache returns the address of pvspec in the above structure.
If the config interpreter uses pv_cache, you get access to pointer of pvspec field. You can use structure offset to get the start of the item and from there pvname, but because cache is not used name, what you get from config interpreter can be a standalone pointer and will result in segmentation fault.
You can lookup the pointer in the pv_cache, by iterating through all slots until a match. Could be good/fast enough for debugging. I was thinking of using a hash table to index needed pv_cache items.
A virtual example of config:
$x1 = $x2 + $x3 + $x4; xlog("$x5 $x6 $x7"); $x8 = $x1 + $x9; $x1 = $x10 + $x11 + $x4;
The pv cache will have 11 items, but you need to know only the name of two pvs: $x1 and $x2, because the others are not assigned. To speed up, you look first for the pointer to $x1 on debugger hash table, it is not found, iterate in pv cache, find it and store the pointer in debugger hash table so next time you need it, you find it quickly.
Cheers, Daniel
On 5/24/13 12:00 AM, Daniel-Constantin Mierla wrote:
On 5/23/13 10:45 PM, Victor Seva wrote:
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
First a general recommendation: try to avoid patching core that much for non-common use cases.
Sure. I was just trying to see how this could be done. I don't want to mess with the core. :-)
[snip]
At this moment, I think a good solution could be:
[snip]
- update the interpreter to use pv cache instead of own spec per pv
(I can do it, being in my list and hopefully is no big change)
I will try to do it myself just to get familliar with this area. Let's see how it goes.
- if this feature is enabled in debugger module, it will create a
hash table where to index pv set functions by pointer and point to the cache item
- this index has to be created in child_init with rank PROC_INIT to
be sure all pvs were resolved to cache item
Sorry. But I don't get this. Can you, please, explain me the rationale behind this hash table?.
First about the pv_cache - it is a hash table with specs of pvs used in config file. Not all pvs are using the cache yet, but should be migrate to it. The nebefit is that if a same pv is used more than one, only one spec is stored in cache. For example, if there will be no pv cache used, if you have 10 of $ru, used memory is 20*sizeof(pv_spec). If pv cache will be used everywhere, the will be one pv_spec and 10 pointers to it. Because the size of pvspec is much larger than pointer size, it obviously results in lower memory usage. More that that, if one needs to parse a pv name at runtime, will not allocate memory each time, but only first time when added to cache, avoiding memory leak because some pv_spec have dynamic structure and no free function.
The pv_cache contains items of:
{ pvname pvspec ... }
A lookup by pvname in pv_cache returns the address of pvspec in the above structure.
If the config interpreter uses pv_cache, you get access to pointer of pvspec field. You can use structure offset to get the start of the item and from there pvname,
but because cache is not used name,
wanted to say: "but because the cache is not used everywhere" -- also some modules use core actions from inside the code, therefore you search for pointer in cache, you don't find, then print $unknown...
what you get from config interpreter can be a standalone pointer and will result in segmentation fault.
You can lookup the pointer in the pv_cache, by iterating through all slots until a match. Could be good/fast enough for debugging. I was thinking of using a hash table to index needed pv_cache items.
A virtual example of config:
$x1 = $x2 + $x3 + $x4; xlog("$x5 $x6 $x7"); $x8 = $x1 + $x9; $x1 = $x10 + $x11 + $x4;
The pv cache will have 11 items, but you need to know only the name of two pvs: $x1 and $x2, because the others are not assigned. To speed up, you look first for the pointer to $x1 on debugger hash table, it is not found, iterate in pv cache, find it and store the pointer in debugger hash table so next time you need it, you find it quickly.
Cheers, Daniel
2013/5/23 Victor Seva linuxmaniac@torreviejawireless.org:
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
- update the interpreter to use pv cache instead of own spec per pv (I can
do it, being in my list and hopefully is no big change)
I will try to do it myself just to get familliar with this area. Let's see how it goes.
You mean modifiing cfg.y on:
pvar: PVAR { pv_spec=pkg_malloc(sizeof(*pv_spec)); if (!pv_spec) { yyerror("Not enough memory"); YYABORT; } memset(pv_spec, 0, sizeof(*pv_spec)); s_tmp.s=$1; s_tmp.len=strlen($1); if (pv_parse_spec(&s_tmp, pv_spec)==0){ yyerror("unknown script pseudo variable %s", $1 ); pkg_free(pv_spec); pv_spec=0; YYABORT; } $$=pv_spec; } ;
avp_pvar: AVP_OR_PVAR { lval_tmp=pkg_malloc(sizeof(*lval_tmp)); if (!lval_tmp) { yyerror("Not enough memory"); YYABORT; } memset(lval_tmp, 0, sizeof(*lval_tmp)); s_tmp.s=$1; s_tmp.len=strlen(s_tmp.s); if (pv_parse_spec2(&s_tmp, &lval_tmp->lv.pvs, 1)==0){ /* not a pvar, try avps */ /* lval_tmp might be partially filled by the failed pv_parse_spec2() (especially if the avp name is the same as a pv class) => clean it again */ memset(lval_tmp, 0, sizeof(*lval_tmp)); lval_tmp->lv.avps.type|= AVP_NAME_STR; lval_tmp->lv.avps.name.s.s = s_tmp.s+1; lval_tmp->lv.avps.name.s.len = s_tmp.len-1; lval_tmp->type=LV_AVP; }else{ lval_tmp->type=LV_PVAR; } $$ = lval_tmp; DBG("parsed ambigous avp/pvar "%.*s" to %d\n", s_tmp.len, s_tmp.s, lval_tmp->type); } ;
Not malloc pv_spec and instead use pv_cache_get()?
Cheers, Victor
Hello,
right direction, but maybe the change is not required everywhere or maybe inside some functions, you have to analyze where is pvar and where is ser-style avp.
Cheers, Daniel
On 5/24/13 9:22 AM, Victor Seva wrote:
2013/5/23 Victor Seva linuxmaniac@torreviejawireless.org:
2013/5/23 Daniel-Constantin Mierla miconda@gmail.com:
- update the interpreter to use pv cache instead of own spec per pv (I can
do it, being in my list and hopefully is no big change)
I will try to do it myself just to get familliar with this area. Let's see how it goes.
You mean modifiing cfg.y on:
pvar: PVAR { pv_spec=pkg_malloc(sizeof(*pv_spec)); if (!pv_spec) { yyerror("Not enough memory"); YYABORT; } memset(pv_spec, 0, sizeof(*pv_spec)); s_tmp.s=$1; s_tmp.len=strlen($1); if (pv_parse_spec(&s_tmp, pv_spec)==0){ yyerror("unknown script pseudo variable %s", $1 ); pkg_free(pv_spec); pv_spec=0; YYABORT; } $$=pv_spec; } ;
avp_pvar: AVP_OR_PVAR { lval_tmp=pkg_malloc(sizeof(*lval_tmp)); if (!lval_tmp) { yyerror("Not enough memory"); YYABORT; } memset(lval_tmp, 0, sizeof(*lval_tmp)); s_tmp.s=$1; s_tmp.len=strlen(s_tmp.s); if (pv_parse_spec2(&s_tmp, &lval_tmp->lv.pvs, 1)==0){ /* not a pvar, try avps */ /* lval_tmp might be partially filled by the failed pv_parse_spec2() (especially if the avp name is the same as a pv class) => clean it again */ memset(lval_tmp, 0, sizeof(*lval_tmp)); lval_tmp->lv.avps.type|= AVP_NAME_STR; lval_tmp->lv.avps.name.s.s = s_tmp.s+1; lval_tmp->lv.avps.name.s.len = s_tmp.len-1; lval_tmp->type=LV_AVP; }else{ lval_tmp->type=LV_PVAR; } $$ = lval_tmp; DBG("parsed ambigous avp/pvar "%.*s" to %d\n", s_tmp.len, s_tmp.s, lval_tmp->type); } ;
Not malloc pv_spec and instead use pv_cache_get()?
Cheers, Victor