I am aware of, and have used sucessfully, the %s substitution option in redis_cmd, however when trying to do 4 values I'm getting a parse error about too many arguments. A review of the module's source appears to indicate that there is a hard limit of 3 substitution values when using this method.
Is there a suitable workaround anyone has for this to enable insertion of values with spaces in them that won't get interpreted by Kamailio itself? Escaping the value, even by using $_s to eval a dynamic string, ends up with Kamailio itself apparently trying to parse the escape and breaking things further.
Thanks! Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
A common approach to avoid delimiter issues in general is to store base64-encoded strings in Redis rather than the original strings. If whatever is reading from Redis can un-encode them, that pretty well solves the problem.
On Sep 15, 2022, at 12:09 PM, Brooks Bridges bbridges@call48.com wrote:
I am aware of, and have used sucessfully, the %s substitution option in redis_cmd, however when trying to do 4 values I'm getting a parse error about too many arguments. A review of the module's source appears to indicate that there is a hard limit of 3 substitution values when using this method.
Is there a suitable workaround anyone has for this to enable insertion of values with spaces in them that won't get interpreted by Kamailio itself? Escaping the value, even by using $_s to eval a dynamic string, ends up with Kamailio itself apparently trying to parse the escape and breaking things further.
Thanks! Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Unfortunately due to the volume of records, I don't think that's going to be really feasible as I'm dumping these out directly to a csv to be archived and imported into a database using pg_bulkload, and I don't think the additional overhead of having to process every record (of which there will be at least 1M per minute, likely more in the future) is feasible without impacting the performance of the system.
-----Original Message----- From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Alex Balashov Sent: Thursday, September 15, 2022 09:11 To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] ndb_redis value substitution or escaping spaces
A common approach to avoid delimiter issues in general is to store base64-encoded strings in Redis rather than the original strings. If whatever is reading from Redis can un-encode them, that pretty well solves the problem.
On Sep 15, 2022, at 12:09 PM, Brooks Bridges bbridges@call48.com wrote:
I am aware of, and have used sucessfully, the %s substitution option in redis_cmd, however when trying to do 4 values I'm getting a parse error about too many arguments. A review of the module's source appears to indicate that there is a hard limit of 3 substitution values when using this method.
Is there a suitable workaround anyone has for this to enable insertion of values with spaces in them that won't get interpreted by Kamailio itself? Escaping the value, even by using $_s to eval a dynamic string, ends up with Kamailio itself apparently trying to parse the escape and breaking things further.
Thanks! Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
__________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
base64 is a pretty trivial algorithm. You’d be surprised.
On Sep 15, 2022, at 12:37 PM, Brooks Bridges bbridges@call48.com wrote:
Unfortunately due to the volume of records, I don't think that's going to be really feasible as I'm dumping these out directly to a csv to be archived and imported into a database using pg_bulkload, and I don't think the additional overhead of having to process every record (of which there will be at least 1M per minute, likely more in the future) is feasible without impacting the performance of the system.
-----Original Message----- From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Alex Balashov Sent: Thursday, September 15, 2022 09:11 To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] ndb_redis value substitution or escaping spaces
A common approach to avoid delimiter issues in general is to store base64-encoded strings in Redis rather than the original strings. If whatever is reading from Redis can un-encode them, that pretty well solves the problem.
On Sep 15, 2022, at 12:09 PM, Brooks Bridges bbridges@call48.com wrote:
I am aware of, and have used sucessfully, the %s substitution option in redis_cmd, however when trying to do 4 values I'm getting a parse error about too many arguments. A review of the module's source appears to indicate that there is a hard limit of 3 substitution values when using this method.
Is there a suitable workaround anyone has for this to enable insertion of values with spaces in them that won't get interpreted by Kamailio itself? Escaping the value, even by using $_s to eval a dynamic string, ends up with Kamailio itself apparently trying to parse the escape and breaking things further.
Thanks! Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
I'm more concerned about the time and load involved in having to loop through a massive csv and run that against specific columns of each line. There may be an option to avoid having to do that though, I'll experiment with it a bit. Thanks for the suggestion!
-----Original Message----- From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Alex Balashov Sent: Thursday, September 15, 2022 09:39 To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] ndb_redis value substitution or escaping spaces
base64 is a pretty trivial algorithm. You’d be surprised.
On Sep 15, 2022, at 12:37 PM, Brooks Bridges bbridges@call48.com wrote:
Unfortunately due to the volume of records, I don't think that's going to be really feasible as I'm dumping these out directly to a csv to be archived and imported into a database using pg_bulkload, and I don't think the additional overhead of having to process every record (of which there will be at least 1M per minute, likely more in the future) is feasible without impacting the performance of the system.
-----Original Message----- From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Alex Balashov Sent: Thursday, September 15, 2022 09:11 To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] ndb_redis value substitution or escaping spaces
A common approach to avoid delimiter issues in general is to store base64-encoded strings in Redis rather than the original strings. If whatever is reading from Redis can un-encode them, that pretty well solves the problem.
On Sep 15, 2022, at 12:09 PM, Brooks Bridges bbridges@call48.com wrote:
I am aware of, and have used sucessfully, the %s substitution option in redis_cmd, however when trying to do 4 values I'm getting a parse error about too many arguments. A review of the module's source appears to indicate that there is a hard limit of 3 substitution values when using this method.
Is there a suitable workaround anyone has for this to enable insertion of values with spaces in them that won't get interpreted by Kamailio itself? Escaping the value, even by using $_s to eval a dynamic string, ends up with Kamailio itself apparently trying to parse the escape and breaking things further.
Thanks! Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
__________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions * sr-users@lists.kamailio.org Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe: * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
I see what you mean: if postprocessing the CSV is required, that sounds like a pain. Ideally there’s a way to unencode the column values as they’re streamed into pg_bulkupload somehow, even if it requires an additional programmatic shim.
On Sep 15, 2022, at 12:47 PM, Brooks Bridges bbridges@call48.com wrote:
I'm more concerned about the time and load involved in having to loop through a massive csv and run that against specific columns of each line. There may be an option to avoid having to do that though, I'll experiment with it a bit. Thanks for the suggestion!
-----Original Message----- From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Alex Balashov Sent: Thursday, September 15, 2022 09:39 To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] ndb_redis value substitution or escaping spaces
base64 is a pretty trivial algorithm. You’d be surprised.
On Sep 15, 2022, at 12:37 PM, Brooks Bridges bbridges@call48.com wrote:
Unfortunately due to the volume of records, I don't think that's going to be really feasible as I'm dumping these out directly to a csv to be archived and imported into a database using pg_bulkload, and I don't think the additional overhead of having to process every record (of which there will be at least 1M per minute, likely more in the future) is feasible without impacting the performance of the system.
-----Original Message----- From: sr-users sr-users-bounces@lists.kamailio.org On Behalf Of Alex Balashov Sent: Thursday, September 15, 2022 09:11 To: Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org Subject: Re: [SR-Users] ndb_redis value substitution or escaping spaces
A common approach to avoid delimiter issues in general is to store base64-encoded strings in Redis rather than the original strings. If whatever is reading from Redis can un-encode them, that pretty well solves the problem.
On Sep 15, 2022, at 12:09 PM, Brooks Bridges bbridges@call48.com wrote:
I am aware of, and have used sucessfully, the %s substitution option in redis_cmd, however when trying to do 4 values I'm getting a parse error about too many arguments. A review of the module's source appears to indicate that there is a hard limit of 3 substitution values when using this method.
Is there a suitable workaround anyone has for this to enable insertion of values with spaces in them that won't get interpreted by Kamailio itself? Escaping the value, even by using $_s to eval a dynamic string, ends up with Kamailio itself apparently trying to parse the escape and breaking things further.
Thanks! Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system.
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-- Alex Balashov | Principal | Evariste Systems LLC
Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/
Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
Confidentiality Notice: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named on the e-mail. If the reader of this e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system. __________________________________________________________ Kamailio - Users Mailing List - Non Commercial Discussions
- sr-users@lists.kamailio.org
Important: keep the mailing list in the recipients, do not reply only to the sender! Edit mailing list options or unsubscribe:
-
Alex Balashov -
Brooks Bridges