New subject: qpid-proton and rabbitmq (and Kamailio kazoo module)

3 Apr 2017

On 30/03/17 13:25, Gordon Sim wrote:
...
  On 30/03/17 09:52, Daniel Pocock wrote:
  Sending to RabbitMQ with qpid-proton python
 -------------------------------------------

 I started with the simple_send.py[3] example and trimmed it down to send
 a single string and then stop:

     def on_sendable(self, event):
         if self.msg_body is not None:
             print("on_sendable !")
             print ("sending : %s" % (self.msg_body,))
             msg = Message(body=self.msg_body)
             event.sender.send(msg)
             self.msg_body = None

 I found that the connection to RabbitMQ would be dropped and an error
 would appear in the RabbitMQ log.  I tried changing the body type to a
 dict containing a string, e.g. {'my_body':'foo'} and then it would
 successfully pass the message to RabbitMQ but then the receiver would
 complain that it was a map and not a string.

 I found that the problem could be fixed by adding "inferred=True" to the
 Message constructor:

             msg = Message(body=self.msg_body, inferred=True)  

 The python proton library will encode a binary message body as a Data
 section if inferred is not set to true (otherwise as a binary value in a
 Value section). RabbitMQ does not support Data sections yet I believe.

Is there a bug or feature request for Data sections in RabbitMQ?  I
couldn't find their bug tracker.  I've opened a bug report[1] against
the Debian package.

...
  As Robbie mentioned, in python 2.x string literals are
not always
 explicitly utf8 and are then treated as binary. You can either do u'my
 utf8' or import unicode_literals from __future__. If you do that then
 you should get a Value section regardless of the inferred property.

In my case the strings are coming from the command line, so now I change
the string to unicode like this:

   _s = unicode(s, "utf-8")

and then:

a) I can send without inferred=True and the broker accepts the message

b) in the C++ receiver, the message is now received as a string and I
can use:

    _json = proton::get<std::string>(m.body());

Could this be made more obvious in the examples perhaps, e.g.
helloworld.py could change from:

    event.sender.send(Message(body="Hello World!"))

to

    event.sender.send(Message(body=unicode("Hello World!", "utf-8")))

I realize that is a bit verbose and redundant when you already have
unicode_literals, but for people who are cutting and pasting from the
examples they will get up and running more quickly.

...
   Below is a
complete example of the error from the RabbitMQ log when
 inferred=True is not present, the body in this case was the string "foo"

 Receiving from RabbitMQ with qpid-proton C++
 --------------------------------------------

 I looked at the receive example simple_recv.cpp[4] and other examples
 and noticed code like this being used to access the message body as
 std::string:

 proton::get<std::string>(m.body())

 However, that didn't work for me, it threw a conversion_error exception
 "unexpected type, want: string got: binary" complaining that the body
 was binary.  I had to use something like this to convert my message body
 to a std::string:

    proton::binary __b = proton::get<proton::binary>(m.body());
    std::string _s = (std::string)__b;

 If I send a string from Python, should it appear as binary in the
 receiver?  
 As above it sounds like the message *is* actually being sent as a binary.

  What is the suggested way to write a receiver
that can handle
 any arbitrary message that started as a string and may arrive in some
 other format, especially if the message broker or client is changed at
 some arbitrary time in the future?

 I also observed similar issues receiving messages that had been sent
 into RabbitMQ by Kamailio's kazoo module[5], it is linked with the
 librabbitmq client library.  
 I'm assuming that is sending over AMQP 0-10. If so, how that is
 converted into AMQP 1.0 is dependent on the RabbitMQ implementation. It
 may well be always sent as a binary value. However hopefully it would
 have a content-type to indicate how to interpret the data (though
 structly speaking the AMQP 1.0 spec disallows that).

The kazoo module source code is on Github[2], I had a quick look at it
and it isn't immediately obvious to me which AMQP version it uses or
which data type it is using for the message body.  Is anybody more
familiar with AMQP C client programming able to spot any keywords in
that code that suggest what it is doing?  I've added the kazoo module
developers on CC in case they can comment on this, it would be useful to
answer this in the module's README file.

Regards,

Daniel

1. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859399
2. https://github.com/sipwise/kamailio/tree/master/modules/kazoo

Re: [sr-dev] qpid-proton and rabbitmq (and Kamailio kazoo module)