[Serusers] Carrier-grade framework for SER

[Andreas Granig]

I've changed to the serdev list, I think it's ontopic there.

Greger V. Teigre wrote:
> DNS SRV.  I think some kind of reference design using Linux HA or 
> virtual server would be useful for the community. I have seen several 
> people asking questions, but I haven't seen documentation of a 
> successful implementation. The reference design must of course handle 
> NAT traversal and the possibility of RTP proxying.

Ack. I heard about people testing
http://www.linuxvirtualserver.org/software/ipvs.html with SER, but the
system design and the results are, well, closed.

> yes, I see you here define a more general network wrapper to the FIFO.  
> Why not extend the scope right now to support any FIFO implemented 
> command, also future?

Of course this should be possible.

>    If FIFO writing is not possible, I believe an alert should be made 
> and the request left in a transaction queue.  There is probably a reason 
> for the FIFO error and also writing directly to the database will mess 
> up the synch between DB and server.

I meant that when you definitely know that the SER is currently down you
could write directly to the database. On the other hand writing to the
DB would mean that you've to know the usrloc backend, so I agree that
this is not a good idea.

>    Bringing up new clients will of course need some special thoughts in 
> terms of their status and at what point in replication they should 
> enter. Probably each transaction should use a unique transaction id...

Adding new SERs to a node isn't the common case I think. So I'd keep it
rather simple in the beginning and hardcode the peers in a file or
something like this. New contacts are automatically pushed to new nodes, 
and the overall location table has to be pulled by the new node.
Just my opinion.

> Why do you think SOAP should be dropped?
>    If this is to be a general module for both provisioning and 
> replication across servers, we must handle many different usage 
> scenarios, ex. provisioning over the Internet, and we should use a 
> framework where we can get as much as possible for "free", ex. SSL/TLS, 
> authentication etc.
>    In my experience, SOAP can be both simple and fast and there are far 
> more auto-generation tools/frameworks etc for SOAP.  A simple RPC-type 
> SOAP message is not really far from XML-RPC, basically an envelope 
> extra.  The ability to auto-generate a client based on a WSDL file is 
> pretty nice, as well.

I didn't perform a benchmark, just asked the mailserver guys here who
use SOAP for provisioning, and they say it's ok for a few requests per
second, but it will not perform for, say, 100 requests per second.

> I have here contributed to increasing the scope... But I think the 
> correct approach would be to make something simple in the beginning, 
> maybe only the provisioning and the queue and a text-based config file 
> for clients to replicate to.  We could then add the ser module for 
> forwarding usrloc to the local client.

My current status is:

I've written a SER module that hooks into the usrloc module by using the 
  existing callback mechanism. On each change of a contact (insert, 
update, delete, expire) the module connects via UNIX socket to a 
external client and serializes the contact to the client via XML:

<?xml version="1.0" standalone="yes"?>
<contact>
<reptype>update</reptype>
<domain>bG9jYXRpb24=</domain>
<aor>MDE4OTA5MDAy</aor>
<c>c2lwOjAxODkwOTAwMkA4My42NS40MC44Nzo1MDYwO3VzZXI9cGhvbmU7dHJhbnNwb3J0PXVkcA==</c>
<expires>1106312247</expires>
<q>-1</q>
<callid>MTkzNDY0MjcxN0A4My42NS40MC44Nw==</callid>
<cseq>15</cseq>
<replicate>0</replicate>
<state>1</state>
<flags>0</flags>
<user_agent>Q2lzY28gQVRBIDE4NiAgdjMuMi4wIGF0YXNpcCAoMDQxMTExQSk=</user_agent>
</contact>

Note that strings except reptype are base64 encoded.
The structure of the message will maybe change alot when we've defined 
the details (which protocol, which messages etc), the current module 
version is just a proof of concept, how contacts can be passed out of 
the SER. So please continue contributing ideas ;o)

Btw, for serializing the ucontact_t structure to XML I use the structs 
mechanism of libpdel (http://www.dellroad.org/pdel/index).

The external client (most of it has still to be implemented) has a list 
of peers where this message has to be send to. If sending fails, it will 
be put in a queue, and a timer triggers a retransmission.
Clients also listen for such replication messages from other peers and 
write them to the SER fifo (or maybe directly via the module, don't know 
yet ;o)

The external client can also be reused for provisioning, because you 
just have to build the XML string in your web application and send it 
via UNIX socket to the client like the SER module does.

Btw, as you can see: My time scope is ASAP ;o)

Andy