[SR-Users] Understanding regex format for subst_hf

Tue Oct 5 19:06:09 CEST 2021

Hi Daniel,

Thank you for clarifying how subst_hf works!

Regarding the size, it does in fact go over the 1024 limit, in this setup
we are using firebase tokens which are 1162 characters long, and what I'm
trying to do is to prefix the value we receive with some more characters,
for a total of 1178 characters.

So something along:

subst_hf("X-myheader", "/^(.*)/myprefix \1/", "a")

Would it be possible to expose that define for REPLACE_BUFFER_SIZE as a
modparam to be configurable?

Thanks for looking into this!

Best,
Joel.

On Tue, Oct 5, 2021 at 4:17 AM Daniel-Constantin Mierla <miconda at gmail.com>
wrote:

> Regarding:
>
> """
> Oct  4 10:44:44 kamailio-n1 kamailio[20324]: ERROR: <core>
> [core/re.c:363]: replace_build(): Buffer too small
> """
>
> That seems to be related to the define of REPLACE_BUFFER_SIZE, which is
> 1024. Changing it requires to recompile kamailio. Is the value to be set
> bigger than 1024?
>
> Cheers,
> Daniel
> On 05.10.21 10:05, Daniel-Constantin Mierla wrote:
>
> Hello,
>
> the regular expressions in textops have to be POSIX format (extended,
> iirc), the remark for perl/sed-like was referring to the format being
> '/match/replace/flags' that is common for Perl scripting and sed parameters.
>
> Practically textops uses the regex functions from libc, does not depend on
> libpcre that offers PCRE format for expressions.
>
> The regex supports matching with PCRE and dialplan offers a way to do
> substitutions by matching with PCRE library.
>
> I updated the readme of textops to reflect somehow better the above.
>
> Cheers,
> Daniel
> On 04.10.21 19:50, Joel Serrano wrote:
>
> So now when combined with subst_hf to actually edit the header value, I'm
> getting this error:
>
> Oct  4 10:44:44 kamailio-n1 kamailio[20324]: ERROR: <core>
> [core/re.c:363]: replace_build(): Buffer too small
> Oct  4 10:44:44 kamailio-n1 kamailio[20324]: ERROR: textops
> [textops.c:3110]: subst_hf_f(): textops subst_run failed
>
> Can anyone remind me the name of the parameter I have to adjust to
> increase the buffer size?
>
> Thanks,
> Joel.
>
> On Mon, Oct 4, 2021 at 10:46 AM Joel Serrano <joel at textplus.com> wrote:
>
>> Update:
>>
>> I managed to achieve it using the "regex" module with "pcre_match()",
>> that will take negative lookahead nicely.
>>
>>
>>
>> On Mon, Oct 4, 2021 at 10:18 AM Joel Serrano <joel at textplus.com> wrote:
>>
>>> Hey guys,
>>>
>>> I'm trying to do a replacement regex using subst_hf but I can't seem to
>>> get the format right and the docs are confusing.
>>>
>>> *subst_hf: *docs say it uses perl-like regex (as I understand it, that
>>> would mean PCRE). Docs also say in the 'subexp' param that it uses the same
>>> format as *subst.*
>>>
>>> *subst: *at the beginning it says it uses perl or sed format, but in
>>> the '/re/repl/flags' param it says that sed format.
>>>
>>>
>>> https://www.kamailio.org/docs/modules/devel/modules/textops.html#textops.f.subst_hf
>>>
>>> 4.20.  subst_hf(hf, subexp, flags)
>>>
>>> Perl-like substitutions in the body of a header field.
>>>
>>> Meaning of the parameters is as follows:
>>>
>>>    -
>>>
>>>    *hf* - header field name.
>>>    -
>>>
>>>    *subexp* - substitution expression in the same format as of the
>>>    'subst' function parameter.
>>>    -
>>>
>>>    *flags* - control flags - it has to be one of: a - all headers
>>>    matching the name; f - only first header matching the name; l - only the
>>>    last header matching the name.
>>>
>>>
>>> 4.16.  subst('/re/repl/flags')
>>>
>>> Replaces re with repl (sed or perl like).
>>>
>>> Meaning of the parameters is as follows:
>>>
>>>    -
>>>
>>>    *'/re/repl/flags'* - sed like regular expression. flags can be a
>>>    combination of i (case insensitive), g (global) or s (match newline don't
>>>    treat it as end of line).
>>>
>>>    're' - is regular expression
>>>
>>>    'repl' - is replacement string - may contain pseudo-variables
>>>
>>>    'flags' - substitution flags (i - ignore case, g - global)
>>>
>>>
>>>
>>>
>>>
>>> I'm trying to use this:
>>>
>>> subst_hf("X-myheader", "/^(?!string1 |example2-|text3 )(.*)/myprefix
>>> \1/", "a")
>>>
>>> In an online tester using PCRE, the format is correct, but Kamailio
>>> doesn't like it:
>>>
>>> Oct  4 09:29:16 cops-n1 cops[15957]: ERROR: <core> [core/re.c:285]:
>>> subst_parser(): bad regular expression ^(?!string1 |example2-|text3 )(.*)
>>> in /^(?!string1 |example2-|text3 )(.*)/myprefix \1/
>>>
>>>
>>> If I remove the negative lookahead chars (?!) then kamailio doesn't
>>> complain but the regex doesn't do what I want.
>>>
>>> Does this mean that Kamailio won't support negative lookahead for subst
>>> regex replacements?
>>>
>>> What I'm trying to achieve is something along:
>>>
>>> "If the value of the header Myheader doesn't start with ABC, DEF or HIJ,
>>> then prefix it with XYZ"
>>>
>>> Any suggestions/ideas on how to achieve this?
>>>
>>>
>>> Thanks,
>>> Joel.
>>>
>>>
>>>
>>>
> __________________________________________________________
> Kamailio - Users Mailing List - Non Commercial Discussions
>   * sr-users at lists.kamailio.org
> Important: keep the mailing list in the recipients, do not reply only to the sender!
> Edit mailing list options or unsubscribe:
>   * https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
> --
> Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda
> Kamailio Advanced Training - Online
> Nov 08-11, 2021 (Europe Timezone) - Nov 22-25, 2021 (America Timezone)
>   * https://www.asipto.com/sw/kamailio-advanced-training-online/
>
> --
> Daniel-Constantin Mierla -- www.asipto.comwww.twitter.com/miconda -- www.linkedin.com/in/miconda
> Kamailio Advanced Training - Online
> Nov 08-11, 2021 (Europe Timezone) - Nov 22-25, 2021 (America Timezone)
>   * https://www.asipto.com/sw/kamailio-advanced-training-online/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.kamailio.org/pipermail/sr-users/attachments/20211005/2c68595d/attachment.htm>