Just to update on this discussion, if someone is going to read it in the future...
Apparently the support for projection was added in the mongo-c-driver library v1.5.0.
With older versions it proved that using projection to specify the list of returned columns triggered errors. The master branch has a fix for working with older versions than 1.5.0, it will be backported.
Cheers, Daniel
On 01.03.18 15:21, Stephane wrote:
Thanks very much Daniel ! This is working now, with both mongo server versions. Kind regards, Stephane
On 1 March 2018 at 14:31, Daniel-Constantin Mierla <miconda@gmail.com mailto:miconda@gmail.com> wrote:
Hello, indeed, it was a similar case in another function. Can you fetch again master and try? Cheers, Daniel On 01.03.18 13:32, Stephane wrote:Hello Daniel, Maybe a similar issue somewhere else in the code? 0(5462) ERROR: db_mongodb [mongodb_dbase.c:557]: db_mongodb_convert_bson(): field [projection] not found in result iterator 0(5462) ERROR: db_mongodb [mongodb_dbase.c:708]: db_mongodb_convert_result(): failed to convert bson at pos 0 0(5462) ERROR: db_mongodb [mongodb_dbase.c:776]: db_mongodb_store_result(): failed to set the rows in result 0(5462) ERROR: db_mongodb [mongodb_dbase.c:965]: db_mongodb_query(): failed to store result 0(5462) ERROR: db_mongodb [mongodb_dbase.c:974]: db_mongodb_query(): failed to do the query Kind regards, Stephane On 1 March 2018 at 12:58, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: Hello, thanks for testing -- the new error was on matching returned columns, as now they are stored in the projection field document. I just pushed a new patch, can you try again with latest master? Cheers, Daniel On 01.03.18 11:14, Stephane wrote:Hello Daniel, The format of the mongo request is better I think. But there is the following new error (with both mongo server versions) 0(30473) ERROR: db_mongodb [mongodb_dbase.c:386]: db_mongodb_get_columns(): field [projection] not found in result iterator 0(30473) ERROR: db_mongodb [mongodb_dbase.c:752]: db_mongodb_store_result(): failed to set the columns 0(30473) ERROR: db_mongodb [mongodb_dbase.c:945]: db_mongodb_query(): failed to store result 0(30473) ERROR: db_mongodb [mongodb_dbase.c:954]: db_mongodb_query(): failed to do the query The pcap traces are in attachment. Thanks ! Stéphane On 1 March 2018 at 08:34, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: Hello, can you try with latest master branch or apply the patch from the next commit to your branch? * https://github.com/kamailio/kamailio/commit/d50498cf483cd22ac4e62af5aaa86fc27654f640 <https://github.com/kamailio/kamailio/commit/d50498cf483cd22ac4e62af5aaa86fc27654f640> If possible, test it with both mongo server versions and if works fine with both, then I am going to backport. Cheers, Daniel On 28.02.18 18:49, Stephane wrote:Hello Daniel, I tried with mongoc 1.8.0 (the trace I have sent) and also mongoc 1.9.2. The result is the same : a "failed to parse" with as indication "unrecognized field 'table_version'" when using mongo server version 3.6.2. Both mongoc versions are running fine with a mongo server version 3.0.15. Kind regards, Stephane On 28 February 2018 at 15:25, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: Hello, I see in the pcap that the returned error contains: Failed to parse: { find: "version", filter: { table_name: "location" }, table_version: 1, $readPreference: { mode: "secondaryPreferred" }, $db: "kamailio" }. Unrecognized field 'table_version'. What is the version of mongoc library? I tried to quick search on google and found a similar report : - https://github.com/mongodb/mongo-php-driver/issues/516 <https://github.com/mongodb/mongo-php-driver/issues/516> On the other hand, it works with same lib but for different versions of mongo server ... so might be something different. Cheers, Daniel On 27.02.18 11:26, Stephane wrote:Hello Daniel, You will find two pcap files in attachment. I am using the same mongoc library version in both cases. Let me know if you need more info. Kind regards, Stephane On 27 February 2018 at 09:01, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: Hello, can you send here the requests sent by kamailio for the two mongodb versions? To compare and see where it can be fixed. Also, were you using the same mongoc library version in both cases? Cheers, Daniel On 26.02.18 12:57, Stephane wrote:Hello, I made some additional tests. The problem is only present when kamailio is facing mongo release 3.2 (or higher). With mongo version 3.0 (or lower), the request sent by kamailio is different: it contains $query and $table_version elements and mongo accepts it. Kind regards, Stephane On 23 February 2018 at 10:48, Stephane <mandalavoip@gmail.com <mailto:mandalavoip@gmail.com>> wrote: Hello Daniel, I took a pcap trace. And I see the following in the answer sent by mongo Failed to parse: { find: "version", filter: { table_name: "location" }, table_version: 1, $db: "kamailio", $readPreference: { mode: "primaryPreferred" }, $clusterTime: { clusterTime: Timestamp(1519379980, 1), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } } }. Unrecognized field 'table_version'.codecodeNameFailedToParseoperationTimeZ$clusterTimeXclusterTimeZsignature3hashkeyId Kind regards, Stephane On 23 February 2018 at 09:39, Daniel-Constantin Mierla <miconda@gmail.com <mailto:miconda@gmail.com>> wrote: Hello, I don't know if you can see the result sent back by mongodb in its logs, it doesn't show it there. Maybe you can sniff the traffic on the network and see if mongodb sends the proper response back. Kamailio logs like nothing was retrieved. Cheers, Daniel On 22.02.18 18:02, Stephane Focant wrote:> Hello, > I have an issue when I try to use > kamailio 5.1 with usrloc and > db_mongdb modules. > When I try to start kamailio, I get > the following error: > > 0(22216) DEBUG: <core> [db.c:314]: > db_do_init2(): connection > 0x7f735b5c4100 not found in pool > 0(22216) DEBUG: db_mongodb > [mongodb_connection.c:55]: > db_mongodb_new_connection(): > connection open to: > mongodb://10.0.12.149 > http://10.0.12.149,10.0.12.150,10.0.12.151/kamailio?replicaSet=sre_location&slaveOk=true&readPreference=primaryPreferred > http://10.0.12.151/kamailio?replicaSet=sre_location&slaveOk=true&readPreference=primaryPreferred > 0(22216) DEBUG: db_mongodb > [mongodb_dbase.c:864]: > db_mongodb_query(): query to > collection [version] > 0(22216) DEBUG: db_mongodb > [mongodb_dbase.c:898]: > db_mongodb_query(): query filter: { > "table_name" : "location" } > 0(22216) DEBUG: db_mongodb > [mongodb_dbase.c:917]: > db_mongodb_query(): columns filter: > { "table_version" : 1 } > 0(22216) DEBUG: <core> > [db_res.c:120]: db_new_result(): > allocate 56 bytes for result set at > 0x7f735b5c4450 > 0(22216) DEBUG: db_mongodb > [mongodb_dbase.c:746]: > db_mongodb_store_result(): no result > from mongodb > 0(22216) DEBUG: <core> [db.c:404]: > db_table_version(): no row for table > location found > 0(22216) ERROR: <core> [db.c:450]: > db_check_table_version(): invalid > version 0 for table location found, > expected 9 (check table structure > and table "version") > 0(22216) ERROR: usrloc > [dlist.c:659]: register_udomain(): > error during table version check. > 0(22216) DEBUG: <core> > [db_pool.c:100]: pool_remove(): > removing connection from the pool > 0(22216) ERROR: registrar > [registrar.c:651]: domain_fixup(): > failed to register domain > 0(22216) ERROR: <core> > [core/route.c:1154]: fix_actions(): > fixing failed (code=-1) at > cfg:/usr/local/kamailio-5.1/etc/kamailio/kamailio.cfg:660 > 0(22216) ERROR: <core> > [core/rvalue.c:3860]: > fix_rval_expr(): failure in cfg at > line: 660 col: 22 > 0(22216) ERROR: <core> > [core/rvalue.c:3860]: > fix_rval_expr(): failure in cfg at > line: 660 col: 22 > 0(22216) ERROR: <core> > [core/route.c:1154]: fix_actions(): > fixing failed (code=-1) at > cfg:/usr/local/kamailio-5.1/etc/kamailio/kamailio.cfg:663 > ERROR: error -1 while trying to fix > configuration > 0(22216) DEBUG: tm [t_funcs.c:85]: > tm_shutdown(): start > 0(22216) DEBUG: tm [t_funcs.c:88]: > tm_shutdown(): emptying hash table > 0(22216) DEBUG: tm [t_funcs.c:90]: > tm_shutdown(): removing semaphores > 0(22216) DEBUG: tm [t_funcs.c:92]: > tm_shutdown(): destroying tmcb lists > 0(22216) DEBUG: tm [t_funcs.c:95]: > tm_shutdown(): done > 0(22216) INFO: <core> > [core/sctp_core.c:53]: > sctp_core_destroy(): SCTP API not > initialized > 0(22216) DEBUG: <core> > [core/mem/shm.c:261]: > shm_destroy_manager(): destroying > memory manager: q_malloc > 0(22216) DEBUG: <core> > [core/mem/q_malloc.c:1165]: > qm_shm_lock_destroy(): destroying > the shared memory lock > 0(22216) DEBUG: <core> > [core/mem/pkg.c:91]: > pkg_destroy_manager(): destroying > memory manager: q_malloc > > > In mongoDB, I have the collection > "version" in the db kamailio, and > the entries needed for usrloc > (location and location_attrs): > sre_location:PRIMARY> use kamailio > switched to db kamailio > sre_location:PRIMARY> > db.getCollection("version").find() > { "_id" : > ObjectId("5a8ec43c601b262bef82a595"), > "table_name" : "location", > "table_version" : 9 } > { "_id" : > ObjectId("5a8edb72a3ec40427577b8d0"), > "table_name" : "location_attrs", > "table_version" : 1 } > > > And logs in mongo show the following > info when kamailio starts: > > 2018-02-22T17:13:02.604+0100 I > NETWORK [conn90] received client > metadata from 10.0.12.180:56320 > http://10.0.12.180:56320 conn: { > driver: { name: "mongoc", version: > "1.9.2" }, os: { type: "Linux", > name: "CentOS", version: "6.9 > (Final)", architecture: "x86_64" }, > platform: "cfg=0x1563000 > posix=200809 CC=GCC 4.4.7 20120313 > (Red Hat 4.4.7-18) CFLAGS="" > LDFLAGS=""" } > 2018-02-22T17:13:02.604+0100 I > COMMAND [conn90] command admin.$cmd > command: isMaster { isMaster: 1, > client: { driver: { name: "mongoc", > version: "1.9.2" }, os: { type: > "Linux", name: "CentOS", version: > "6.9 (Final)", architecture: > "x86_64" }, platform: "cfg=0x1563000 > posix=200809 CC=GCC 4.4.7 20120313 > (Red Hat 4.4.7-18) CFLAGS="" > LDFLAGS=""" }, compression: [], > $readPreference: { mode: > "secondaryPreferred" }, $db: "admin" > } numYields:0 reslen:716 locks:{} > protocol:op_query 0ms > 2018-02-22T17:13:02.604+0100 I > COMMAND [conn90] command > kamailio.$cmd command: find { find: > "version", filter: { table_name: > "location" }, table_version: 1, $db: > "kamailio", $readPreference: { mode: > "primaryPreferred" }, $clusterTime: > { clusterTime: Timestamp(1519315979, > 1), signature: { hash: BinData(0, > 0000000000000000000000000000000000000000), > keyId: 0 } } } numYields:0 > reslen:531 locks:{} protocol:op_msg 0ms > 2018-02-22T17:13:02.605+0100 I > NETWORK [conn90] end connection > 10.0.12.180:56320 > http://10.0.12.180:56320 (9 > connections now open) > > I probably miss something: it is not > clear to me why kamailio is not able > to retrieve the value of > table_version for the table_name > location. Any idea would be very > welcome ! > > I use mongo-c-driver 1.9.2 and > mongodb 3.6 > > Thank you. > Stephane > > > _______________________________________________ > Kamailio (SER) - Users Mailing List > sr-users@lists.kamailio.org > mailto:sr-users@lists.kamailio.org > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com>-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com>-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com>-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com>-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com>-- Daniel-Constantin Mierla www.twitter.com/miconda <http://www.twitter.com/miconda> -- www.linkedin.com/in/miconda <http://www.linkedin.com/in/miconda> Kamailio Advanced Training - March 5-7, 2018, Berlin - www.asipto.com <http://www.asipto.com> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com <http://www.kamailioworld.com>