[sr-dev] importing kamailio tools

Jan Janak jan at iptel.org
Fri May 29 11:18:45 CEST 2009


Henning,

On 27-05 15:04, Henning Westerholt wrote:
>    On Mittwoch, 27. Mai 2009, Jan Janak wrote:
>    > > kamailio tools (kamctl and kamdbctl) have not been imported yet. The
>    > > usual place for them was 'scripts' folder but now there are
>    overlapping
>    > > names, related to the folders holding sql scripts.
>    > >
>    > > Shall we create a subfolder in scripts directory, e.g.,
>    scripts/kamctl?
> 
>    Hi Jan,
> 
>    > What about importing them into the tools directly?
> 
>    do you mean the tools directory? Hm, not sure what then the purpose of the
>    scripts directory would be, perhaps this could be then removed?

Currently the script directory contains all the database related script, and
some stuff that is not being used much (or at all), i.e. filter_log.sh,
serstats, kam_to_sr.sh.

I think we can use the scripts directory to store supporting scripts that
people typically run once or occasionally.

The tools directory could than be used for all other admin related tools, such
as sercmd, gen_ha1, kamdbctl.

We can also get rid of the utils directory.

>    > > The xml definition of db tables were in db directory, which now is
>    > > library srdb1(same for ser, are now in srdb2). Do we let them there or
>    > > is another more suitable place for them as they are not dependent of
>    DB
>    > > api but of what DB structure to be used (some modules using db were
>    > > moved in common folder already)?
>    >
>    > I would suggest that we keep them there for now. The scripts that are
>    used
>    > to generate db schema need, in my opinion, an overhaul, so we might just
>    do
>    > it in one go.
> 
>    I refactored the k XSL scripts (which were imported from SER) in the last
>    year somewhat and also improved/ added support for DBs!=mysql. They do
>    IMHO a good job now to generate the needed database scripts from XML
>    sources, so i don't see a big need to rework them. What i removed was the
>    possibility to also create insert statements, i think, not sure if this is
>    needed from SER.

I think that xsltproc (or XSLT) is the wrong tool for the job. If you take a
look at the conversion scripts, they use lots of variables, conditional
statements, recursions, and functions specified in the XPath specification. In
other words, we use xlst as a real programming language, but unlike many other
languages available today, it has horrible syntax. Heck, even Perl scripts are
more readable than XSLT.

XSLT is probably OK if you want to generate something that looks like XML out
of a XML document, but if you try to build an output with a less regular
syntax, like the SQL scripts or dbtext scripts we generate, you are stuck with
all those extension functions from XPath, here are prime examples of its
cumbersome syntax:

  <xsl:value-of select="normalize-space($select/name[@db=$db])"/>
  <xsl:value-of select="translate(normalize-space($select/type[@db=$db]),
       'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz')"/>

The first statement is an equivalent of the function trim which removes
leading and trailing whitespace and the second statement converts a string to
lower case.

Another major problem is that the scripts are not portable. They work with
xsltproc but would not work with other xsl processors due to subtle
differences in implementation. I couldn't even generate a consistent output
across different versions of xsltproc.

If I had to write such scripts again, I would have chosen another language,
probably Python or Perl. They are both wide-spread, they have decent XML
parsers and less terryfing syntax (well, in case of perl this is questionable
:-). 

With python or perl we could do much more than just generate plain-text
output, we could create the tables directly in the database, setup usernames
and passwords, handle backups and upgrades conveniently in one tool, and so
on.

While writing those scripts in XSLT was a good experience, I think we are
using the language outside its domain. But at least we learned it, which is
good if one needs to modify the docbook xslt scripts, for example.

   Jan.



More information about the sr-dev mailing list