Kex module updated to also give some details about memory used by each module. Changed f_*, q_* and tlsf_* memory stubs to also include the module name. Added 2 new memory API functions(for each of the memory algos) which will iterate throughout the memory fragments and agregate the memory size used by each function in order to print it. Tested with MEMMNG=0,1,2 and MEMDBG=0,1. Use "kamcmd mem.stat mod_name pkg/shm/all" to test Updated doku. You can view, comment on, or merge this pull request online at:
https://github.com/kamailio/kamailio/pull/225
-- Commit Summary --
* kex: per module memory debugging
-- File Changes --
M events.c (25) M events.h (4) M mem/f_malloc.c (135) M mem/f_malloc.h (47) M mem/mem.h (32) M mem/q_malloc.c (113) M mem/q_malloc.h (40) M mem/shm_mem.c (4) M mem/shm_mem.h (32) M mem/src_loc.h (2) M mem/tlsf.c (105) M mem/tlsf.h (32) M modules/ctl/binrpc_run.c (12) M modules/kex/doc/kex_admin.xml (76) M modules/kex/kex_mod.c (7) A modules/kex/mod_stats.c (288) A modules/kex/mod_stats.h (37)
-- Patch Links --
https://github.com/kamailio/kamailio/pull/225.patch https://github.com/kamailio/kamailio/pull/225.diff
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225
One approach to this feature would have been to create some callbacks to update the memory stats, called by malloc/realloc/free functions. This approach would have been questionable because processing has to be done each time somememory functions were called. The current approach is to export some memory api functions (something similar to q_sums) which will itereate through the memory fragments and agregate their size. The kamcmd callback will call those functions and print the agregated results.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-116713520
The new feature is under the DBG_MOD_STATS define (currently enabled in the f_,q_ and tlsf headers). While implementing this I've noticed that the structures I have to use were similar to those used when the MEMDBG=1. So I decided to "merge" some parts of the code(stubs and some structure params) with the code under the DBG_QM_MALLOC, while still maintaining log differences (this feature does not log anything).
Please tell me what do you think about it.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-116713708
Updated pull request. Created 2 new functions (similar to qm_sums): - qm_mod_get_stats() fills a list of _struct mem_counter_ with details - qm_mod_free_stats() frees the above allocated list
I have a question related to compiling using different memory allocation algorithms (as seen here http://www.kamailio.org/wiki/tutorials/troubleshooting/memory): - MEMMNG=1 MEMDBG=0/1 make cfg - will use the F_* allocation algorithm - MEMMNG=2 MEMDBG=0/1 make cfg - will use the F_* allocation algorithm
How can I compile using Q_* and TLSF_* algorithms in order to make tests for them?
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153000095
I was just looking at the old patch to see what can be done with it, thanks for the update. I will merge it.
MEMMNG is deprecated now, all memory managers (f_malloc, q_malloc and tlsf_malloc) are compiled always -- use command line parameter -x to select the one you want (see kamailio -h).
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153001507
Closed #225.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#event-451936205
I merged the patch, but I split it per component so there are many commits pushed to the master branch - some remarks:
* the patch to ctl module was changed, as it was planned to increase default values for those buffers (note that you can also change them via modparams of ctl module) * I noted that you changed how core is referenced in mem/src_loc.h from "<core>" to "core" -- I assumed that was to make it coherent with how module names are printed * I plan to add a mod param for kex to enable/disable the new feature, just in case someone needs to tune a bit more performances -- it will be enabled by default
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153011270
Thank you for helping to organize this better: - nice, that skipped me - you are right, I changed the name from "<core>" to "core" or else someone would have to issue the command writing "<core>" - an enable/disable modparam would be nice but I'm not sure if the code as it is right now affects performance in any way i.e. **only** when the "kamcmd mod.stats ..." is issued, when debugging is enabled, some processing actually happens; supposing the performance tests are done when debugging is disabled, this should not interfere in any way; please correct me if I'm wrong.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153031394
I thought that the new events are going to be executed on operations, but tracking the code now I see that mod_update_pkg_mod_stats_list() just returns 0; same for shm.
So, what is the purpose for SREV_MODULE_PKG_STATS and SREV_MODULE_SHM_STATS, they don't seem to be used. If not and no plat to use them in the near future, then they should be removed.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153033185
I put them just because I thought they might be used sometimes by someone for some purpose. They are not called right now, and no plan to used them from our side. I'll remove them.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153036511
Then it is better to be removed.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153040804
Removed them in commit 53cd7df83c4065fd7a34cd3ce15b6e215ce39278. Updated http://www.kamailio.org/wiki/tutorials/troubleshooting/memory to include mod.stats command and the "kamailio -x" info.
--- Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/225#issuecomment-153054929
-
Daniel-Constantin Mierla -
Stefan Mititelu