Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
We wrote a little perl script just to do that. Takes all calls from ACC table and outputs them to CDRS table. Its simple and it works. Its free so use it whatever way you want.
Andres.
Frank Fischer wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
That perl script gives me calls with negative duration :), is there any way to differenciate different calls from the acc table? I noticed that the sip_callid is NOT unique to 1 call, a concat(from_tag, to_tag) isnt't unique for 1 call either, this realy makes normalising them a lot harder.
Kind regards,
Erik
Andres wrote:
We wrote a little perl script just to do that. Takes all calls from ACC table and outputs them to CDRS table. Its simple and it works. Its free so use it whatever way you want.
Andres.
Frank Fischer wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
#!/usr/bin/perl
#Script used to generate SER Billing Records, it will look for the first INVITE Message and match it to a BYE Message #that has the same CALLID #Input is taken from the ACC table #Output is witten to MySQL 'cdrs' table and file '$file_res' #MySQL Perl Module Available at: http://search.cpan.org/author/OYAMA/Net-MySQL-0.08/MySQL.pm
#You can create the cdrs table by putting the following in a file and then feeding it to mysql like this: #mysql -p test < "filename" #-----------cut here-------------------- #CREATE TABLE cdrs ( #timestamp TIMESTAMP, #start_date VARCHAR(18), #end_date VARCHAR(18), #caller VARCHAR(50), #destination VARCHAR(50), #callid VARCHAR(50), #duration INT #); #--------------cut here------------------ #Written by: Ricardo Villa (ricvil @ telesip.net) use Net::MySQL; use Time::Local;
my $mysql = Net::MySQL->new( hostname => 'db.xxxx.net', # Default use UNIX socket database => 'ser', user => 'xxxxxx', password => 'xxxxxx' );
my $mysql2 = Net::MySQL->new( hostname => 'db.xxxx.net', # Default use UNIX socket database => 'xxxxxxx', user => 'xxxxxxx', password => 'xxxxxxx' );
$mysql->query(q{SELECT * FROM acc}); #Get Records from Accounting Table my $record_set = $mysql->create_record_iterator; while (my $record = $record_set->each) { $sip_method = $record->[3]; $username = $record->[9]; $to_uri = $record->[7]; $sip_method = $record->[3]; $sip_callid = $record->[8]; $enddate = $record->[13]; $timestamp = $record->[14]; if ($sip_method eq "BYE") {&bye_match();}; };
&cleanup_acc();
$mysql->close; $mysql2->close; exit; ###---------------------------------------------------------------------------------------------------- ### Subroutines sub bye_match { $bye_query = "SELECT * FROM acc where sip_callid = '$sip_callid';";
$mysql->query( $bye_query );
my $record_set2 = $mysql->create_record_iterator; while (my $record2 = $record_set2->each) { $sip_method2 = $record2->[3]; $username2 = $record2->[9]; $to_uri2 = $record2->[7]; $sip_method2 = $record2->[3]; $sip_callid2 = $record2->[8]; $startdate = $record2->[13]; $timestamp2 = $record2->[14]; if (($sip_callid eq $sip_callid2) & ($sip_method2 eq "INVITE")) { &input_processing();
#Convert MySQL Timestamp to Variables $startstamp_year = substr($timestamp2,0,4); $startstamp_month_o = substr($timestamp2,4,2); $startstamp_month = substr($timestamp2,4,2)-1; $startstamp_day = substr($timestamp2,6,2); $startstamp_hour = substr($timestamp2,8,2); $startstamp_minute = substr($timestamp2,10,2); $startstamp_second = substr($timestamp2,12,2); $endstamp_year = substr($timestamp,0,4); $endstamp_month_o = substr($timestamp,4,2); $endstamp_month = substr($timestamp,4,2)-1; $endstamp_day = substr($timestamp,6,2); $endstamp_hour = substr($timestamp,8,2); $endstamp_minute = substr($timestamp,10,2); $endstamp_second = substr($timestamp,12,2); #End Conversion #Convert start and end times to seconds since 1970 $startstamp = timelocal($startstamp_second,$startstamp_minute,$startstamp_hour,$startstamp_day,$startstamp_month,$startstamp_year); $startdate2 = "$startstamp_year-$startstamp_month_o-$startstamp_day $startstamp_hour:$startstamp_minute:$startstamp_second"; $endstamp = timelocal($endstamp_second,$endstamp_minute,$endstamp_hour,$endstamp_day,$endstamp_month,$endstamp_year); $enddate2 = "$endstamp_year-$endstamp_month_o-$endstamp_day $endstamp_hour:$endstamp_minute:$endstamp_second"; $call_duration = $endstamp-$startstamp; #print "Start = $startstamp_year $startstamp_month $startstamp_day $startstamp_hour $startstamp_minute $startstamp_second\n"; &update_table; }; }; };
sub input_processing { ($a_dest, $b_dest) = split(/sip:/,$to_uri2); ($a_dest) = split(/@/,$b_dest); #Isolate the Destination of the Call $to_uri2=$a_dest; }
sub update_table { #Now Insert the Call Detail record into the Database $database_insert = "INSERT INTO cdrs (start_date, end_date, caller, destination, callid, duration) VALUES ('$startdate2', '$enddate2', '$username2', '$to_uri2', '$sip_callid2', $call_duration);"; $mysql2->query( $database_insert ); #and delete from ACC Table $database_delete = "DELETE FROM acc where sip_callid = '$sip_callid2';"; $mysql->query( $database_delete ); }
sub cleanup_acc { #Delete all other garbage from ACC Table $database_delete = "DELETE FROM acc "; $mysql->query( $database_delete ); }
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Erik wrote:
That perl script gives me calls with negative duration :), is there any way to differenciate different calls from the acc table? I noticed that the sip_callid is NOT unique to 1 call, a concat(from_tag, to_tag) isnt't unique for 1 call either, this realy makes normalising them a lot harder.
Kind regards,
Erik
I thing I remember somebody complaining about Grandstream phones repeating the sip_callid. Is that your case? In our case we only use Cisco/Linksys stuff and they never appear to repeat the sip_callid.
Just one quick note on the script. That one is to be run once a day when there is no traffic since the last thing it does is a cleanup of the acc table. You can remove that cleanup line and just run the script every minute to reduce the chance of duplicate sip_callid. Unless of course you have simultaneus calls with same sip_callid which is probably due to a bug in the UA. Maybe the random string it is choosing is not so random after all.
Andres.
Andres wrote:
Erik wrote:
That perl script gives me calls with negative duration :), is there any way to differenciate different calls from the acc table? I noticed that the sip_callid is NOT unique to 1 call, a concat(from_tag, to_tag) isnt't unique for 1 call either, this realy makes normalising them a lot harder.
Kind regards,
Erik
I thing I remember somebody complaining about Grandstream phones repeating the sip_callid. Is that your case? In our case we only use Cisco/Linksys stuff and they never appear to repeat the sip_callid. Just one quick note on the script. That one is to be run once a day when there is no traffic since the last thing it does is a cleanup of the acc table.
I just found out the hard way :) However, we do traffic 24x7
You can remove that cleanup line and just run the script
every minute to reduce the chance of duplicate sip_callid. Unless of course you have simultaneus calls with same sip_callid which is probably due to a bug in the UA. Maybe the random string it is choosing is not so random after all.
It seems to be random, however it looks like it's starting with the same seed if it's restarted :)
Andres.
You can do it "almost" automatically using RADIUS.
Install FreeRadius and have SER write records START and STOP records to RADIUS (corresponding to INVITE and BYE) .
What happens is that when a BYE is received, the RADIUS looks for record corresponding to call-id (and a couple of other checks) and it then UPDATES the corresponding record with the Call-Termination-Time field. So YOU GET A CDR but you do no have a SESSION time.
So what you need to do is that you edit the freeradius "sql.conf" file and modify the "account_stop_query" variable to calculate the SESSION time and update the AcctSessionTime column.
This will allow you to have the CDR with AcctStartTime, AcctStopTime and AcctSessionTime fields in addition to the CalledStationId and CallingStationId (calling phone#) fields and you have a CDR.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
__________________________________ Yahoo! Music Unlimited Access over 1 million songs. Try it free. http://music.yahoo.com/unlimited/
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
__________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Regards Frank
-----Original Message----- From: Dave [mailto:ddx66@yahoo.com] Sent: Wednesday, November 30, 2005 10:57 PM To: Frank Fischer; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
__________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Check your configuration, you should ALWAYS have a BYE or CANCEL, and a CDR should be generated for either one. Both the BYE and CANCEL will generate a CDR with its corresponding call-id.
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@lists.iptel.org] On Behalf Of Frank Fischer Sent: Wednesday, November 30, 2005 5:15 PM To: 'Dave' Cc: serusers@lists.iptel.org Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Regards Frank
-----Original Message----- From: Dave [mailto:ddx66@yahoo.com] Sent: Wednesday, November 30, 2005 10:57 PM To: Frank Fischer; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
__________________________________ Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
_______________________________________________ Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
If I am correct I believe CDRTool uses mediaproxy to prevent the phantom BYE record. Adrian can correct me if I read wrong. Invites without BYE's are common No one answered the phone. We just Log and drop records that dont have matching INVITE,ACK, and BYE. The reason as I have seen for the missing BYE (discussed alot on here) is the UA or the Gateway dont send a BYE for whatever reason. We havent seen that many , infact not one yet knock on wood. We thought about just estimating and calling them one minute calls.
The problem we are having is we route some wholesale traffic thru us via trusted IP and account for it (Easy Part) but we also get DID's from that Company so all traffic comes from same IP. DID's work fine but it broke our parser. LOL was charging them for our incoming calls OOPS!!! I guess for now I am going to move the wholesale Trusted IP stuff to its own box.
Eric eric@rackspeed.net
----- Original Message ----- From: "Frank Fischer" frank.fischer@digitalnomads.ch To: "'Dave'" ddx66@yahoo.com Cc: serusers@lists.iptel.org Sent: Wednesday, November 30, 2005 5:14 PM Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Regards Frank
-----Original Message----- From: Dave [mailto:ddx66@yahoo.com] Sent: Wednesday, November 30, 2005 10:57 PM To: Frank Fischer; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
It is usually enough to match INVITE with --first-- of recorded BYEs with the same CallID. It is fairly accurate. From my experience I would say every month 1 or 2 calls have missing BYEs. You give them "1 minute" duration or just forget ;-)
BTW this is just a top of an iceberg somewhere in front of us. And it does not only concern billing. It concerns management, provisioning, terminal capabilities, charging, multimedia and so on. We surf towards IMS-like architectures, wherever we want or not.
Look at today - we create our own billing systems where every one is different, proprietary and acting on different requirements. IMHO we should rather work together, wherever you call it OpenSource or not.
Let's start with billing. Let's define common requirements, like:
1) tarrification policy (peak/offpeak,initial,post-call charges and so on) 2) way to manage endprices,margins,currencies 3) different database versions to let everybody choose his backend: sql or directory database 4) common data interchange format (I propose IPDR, VoIP profile) 5) common communication protocol (I propose SOAP)
We should not work on payment management, as BSS systems should do it for us, provided that nice data is exported from SER. But if someone wants - OK, payment management may be done as well.
Some may say that SER is not a place for SOAP, IPDR and related stuff. But let's be honest - world is moving into similar direction and technologies such as Parlay X Web Services, JSLEE or IMS are just few examples. There is also ongoing effort to integrate network services around directory databases, wherever it is JNDI (in JSLEE) or LDAP (in OpenSource projects).
I was working for most of this stuff, with success, but a little outside of SER. SER was always somewhere there, but with MySQL stripped almost completely (in fact only 'location' table is used as a kind of HLR). And I see from private emails or contributions that many (all?) of us have done the same. People are using LDAP backend with RADIUS (worked great for us) or without RADIUS (directly to LDAP, worked great too). I have got reports of people having customer base of 5 milion and looking for 10k transactions/second with LDAP backend.
In fact this is OK - you just choose those parts of OpenSource project that fit your needs and integrate. Thats what OpenSource is for. Tens of great people, like SER core developers are working on great software that SER is now, they work for free and make this software better.
But so many users are going towards the 'next big thing'. At the same time tens of other people are working on proprietary platforms for billing, LCR, Web Services or even IMS in their companies. It is OK, but why not provide them a good '--next killer platform--' and let them concentrate on creating better --services --?
Consider this quotation from SLEE primer available at opencloud.com:
"We must, as an industry, make it abundantly clear to all commentators that we are in business to create the most potent and fundamental part of the social fabric of the 21st century. (...) Operators must recognise (...) not just a single service, like video or gaming. By implication, therefore, (...) operators should be creating a killer environment."
Copyright Nicolas Foggin, Director of Strategy & Futurology, Orange SA Copyright Alcatel Telecommunications Review, 2002
You see? Provide killer environment. Let's start with billing.
-- Arek Bekiersz
Eric Haskins wrote:
If I am correct I believe CDRTool uses mediaproxy to prevent the phantom BYE record. Adrian can correct me if I read wrong. Invites without BYE's are common No one answered the phone. We just Log and drop records that dont have matching INVITE,ACK, and BYE. The reason as I have seen for the missing BYE (discussed alot on here) is the UA or the Gateway dont send a BYE for whatever reason. We havent seen that many , infact not one yet knock on wood. We thought about just estimating and calling them one minute calls.
The problem we are having is we route some wholesale traffic thru us via trusted IP and account for it (Easy Part) but we also get DID's from that Company so all traffic comes from same IP. DID's work fine but it broke our parser. LOL was charging them for our incoming calls OOPS!!! I guess for now I am going to move the wholesale Trusted IP stuff to its own box.
Eric eric@rackspeed.net
----- Original Message ----- From: "Frank Fischer" frank.fischer@digitalnomads.ch To: "'Dave'" ddx66@yahoo.com Cc: serusers@lists.iptel.org Sent: Wednesday, November 30, 2005 5:14 PM Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Arek Bekiersz wrote:
It is usually enough to match INVITE with --first-- of recorded BYEs with the same CallID. It is fairly accurate.
Not if there are 2 calls at the same time using the same call_id
From my experience I would say every month 1 or 2 calls have missing BYEs. You give them "1 minute" duration or just forget ;-)
BTW this is just a top of an iceberg somewhere in front of us. And it does not only concern billing.
Just correct CDR's will do for now ;)
You see? Provide killer environment. Let's start with billing.
-- Arek Bekiersz
Eric Haskins wrote:
If I am correct I believe CDRTool uses mediaproxy to prevent the phantom BYE record. Adrian can correct me if I read wrong. Invites without BYE's are common No one answered the phone. We just Log and drop records that dont have matching INVITE,ACK, and BYE. The reason as I have seen for the missing BYE (discussed alot on here) is the UA or the Gateway dont send a BYE for whatever reason. We havent seen that many , infact not one yet knock on wood. We thought about just estimating and calling them one minute calls.
The problem we are having is we route some wholesale traffic thru us via trusted IP and account for it (Easy Part) but we also get DID's from that Company so all traffic comes from same IP. DID's work fine but it broke our parser. LOL was charging them for our incoming calls OOPS!!! I guess for now I am going to move the wholesale Trusted IP stuff to its own box.
Eric eric@rackspeed.net
----- Original Message ----- From: "Frank Fischer" frank.fischer@digitalnomads.ch To: "'Dave'" ddx66@yahoo.com Cc: serusers@lists.iptel.org Sent: Wednesday, November 30, 2005 5:14 PM Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
When do you have 2 calls at the same time using the same Call-ID?
-- Arek Bekiersz
Erik wrote:
Arek Bekiersz wrote:
It is usually enough to match INVITE with --first-- of recorded BYEs with the same CallID. It is fairly accurate.
Not if there are 2 calls at the same time using the same call_id
From my experience I would say every month 1 or 2 calls have missing BYEs. You give them "1 minute" duration or just forget ;-)
On Thu, 01 Dec 2005 11:34:51 +0100, Erik wrote
Arek Bekiersz wrote:
It is usually enough to match INVITE with --first-- of recorded BYEs with the same CallID. It is fairly accurate.
Not if there are 2 calls at the same time using the same call_id
While this is technically against the RFC, there was some discussion that, on occasion, a UA would reuse a callID... but there should be NO situation in which 2 UAs are simultaneously using the same CallID unless you wrote the UA yourself to specifically do this. Do you have information about which UAs were doing this? I'd like to know which UAs to bar from usage on my network. :)
In theory, it SHOULD be enough to match Call ID, but matching CallID, from, and to together should take care of 100% of the known records. If it doesn't something has gone terribly wrong somewhere.
N.
On Thu, 01 Dec 2005 11:34:51 +0100, Erik wrote
Arek Bekiersz wrote:
It is usually enough to match INVITE with --first-- of
recorded BYEs
with the same CallID. It is fairly accurate.
Not if there are 2 calls at the same time using the same call_id
While this is technically against the RFC, there was some discussion that, on occasion, a UA would reuse a callID... but there should be NO situation in which 2 UAs are simultaneously using the same CallID unless you wrote the UA yourself to specifically do this. Do you have information about which UAs were doing this? I'd like to know which UAs to bar from usage on my network. :)
If i understand this right, the UAs are choosing the callId? If so, how could an UA know if the callId it has choosen, isn't already used by another UA in another call? Based on what are callIds created by the UA? Is there some part in the callId which is unique for a certain UA (in fact i see that the ip address of the client is a part of the callId, but this would be not unique enough i guess). Or does SER check for same CallIDs on different simultaneouse calls to make sure that their is no such case? This would mean SER would have to reject calls with same callIds?
- Frank
According to SIP RFC:
" Call-ID contains a globally unique identifier for this call, generated by the combination of a random string and the softphone's host name or IP address. The combination of the To tag, From tag, and Call-ID completely defines a peer-to-peer SIP relationship between Alice and Bob and is referred to as a dialog. "
I would not bother about Call-ID repeating, in normal use it is rather impossible.
Thus if someone has the same Call-ID for more than one call, he has really serious broken UAs or phones. Or I miss smthing?
-- Arek Bekiersz
Frank Fischer wrote:
On Thu, 01 Dec 2005 11:34:51 +0100, Erik wrote
Arek Bekiersz wrote:
It is usually enough to match INVITE with --first-- of
recorded BYEs
with the same CallID. It is fairly accurate.
Not if there are 2 calls at the same time using the same call_id
While this is technically against the RFC, there was some discussion that, on occasion, a UA would reuse a callID... but there should be NO situation in which 2 UAs are simultaneously using the same CallID unless you wrote the UA yourself to specifically do this. Do you have information about which UAs were doing this? I'd like to know which UAs to bar from usage on my network. :)
If i understand this right, the UAs are choosing the callId? If so, how could an UA know if the callId it has choosen, isn't already used by another UA in another call? Based on what are callIds created by the UA? Is there some part in the callId which is unique for a certain UA (in fact i see that the ip address of the client is a part of the callId, but this would be not unique enough i guess). Or does SER check for same CallIDs on different simultaneouse calls to make sure that their is no such case? This would mean SER would have to reject calls with same callIds?
- Frank
If there's not a BYE message (called party doesn't answer or timeout) then the calling party WILL send a CANCEL. The CANCEL can be used to generate the STOP record
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@lists.iptel.org] On Behalf Of Eric Haskins Sent: Thursday, December 01, 2005 3:26 AM To: Frank Fischer; 'Dave' Cc: serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
If I am correct I believe CDRTool uses mediaproxy to prevent the phantom BYE
record. Adrian can correct me if I read wrong. Invites without BYE's are common No one answered the phone. We just Log and drop records that dont have matching INVITE,ACK, and BYE. The reason as I have seen for the missing BYE (discussed alot on here) is the UA or the Gateway dont send a BYE for whatever reason. We havent seen that many , infact not one yet knock on wood. We thought about just estimating and calling them one minute calls.
The problem we are having is we route some wholesale traffic thru us via trusted IP and account for it (Easy Part) but we also get DID's from that Company so all traffic comes from same IP. DID's work fine but it broke our parser. LOL was charging them for our incoming calls OOPS!!! I guess for now I am going to move the wholesale Trusted IP stuff to its own box.
Eric eric@rackspeed.net
----- Original Message ----- From: "Frank Fischer" frank.fischer@digitalnomads.ch To: "'Dave'" ddx66@yahoo.com Cc: serusers@lists.iptel.org Sent: Wednesday, November 30, 2005 5:14 PM Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Regards Frank
-----Original Message----- From: Dave [mailto:ddx66@yahoo.com] Sent: Wednesday, November 30, 2005 10:57 PM To: Frank Fischer; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
_______________________________________________ Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
What about accounting failed transactions? Then you will see INVITEs with sip_status=200 for successful calls and sip_status=408 or sip_status=487 (cancel) for non sucessful calls.
klaus
Lenir wrote:
If there's not a BYE message (called party doesn't answer or timeout) then the calling party WILL send a CANCEL. The CANCEL can be used to generate the STOP record
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@lists.iptel.org] On Behalf Of Eric Haskins Sent: Thursday, December 01, 2005 3:26 AM To: Frank Fischer; 'Dave' Cc: serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
If I am correct I believe CDRTool uses mediaproxy to prevent the phantom BYE
record. Adrian can correct me if I read wrong. Invites without BYE's are common No one answered the phone. We just Log and drop records that dont have matching INVITE,ACK, and BYE. The reason as I have seen for the missing BYE (discussed alot on here) is the UA or the Gateway dont send a BYE for whatever reason. We havent seen that many , infact not one yet knock on wood. We thought about just estimating and calling them one minute calls.
The problem we are having is we route some wholesale traffic thru us via trusted IP and account for it (Easy Part) but we also get DID's from that Company so all traffic comes from same IP. DID's work fine but it broke our parser. LOL was charging them for our incoming calls OOPS!!! I guess for now I am going to move the wholesale Trusted IP stuff to its own box.
Eric eric@rackspeed.net
----- Original Message ----- From: "Frank Fischer" frank.fischer@digitalnomads.ch To: "'Dave'" ddx66@yahoo.com Cc: serusers@lists.iptel.org Sent: Wednesday, November 30, 2005 5:14 PM Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Regards Frank
-----Original Message----- From: Dave [mailto:ddx66@yahoo.com] Sent: Wednesday, November 30, 2005 10:57 PM To: Frank Fischer; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Exactly, for those unsuccessful calls, the CANCEL will generate the stop record, just like it would with a BYE. the sip_status is irrelevant, except for informational purposes. You should account for any STOP record regadless of the termination cause of the call.
-----Original Message----- From: Klaus Darilion [mailto:klaus.mailinglists@pernau.at] Sent: Thursday, December 01, 2005 11:16 AM To: Lenir Cc: 'Eric Haskins'; 'Frank Fischer'; 'Dave'; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
What about accounting failed transactions? Then you will see INVITEs with sip_status=200 for successful calls and sip_status=408 or sip_status=487 (cancel) for non sucessful calls.
klaus
Lenir wrote:
If there's not a BYE message (called party doesn't answer or timeout) then the calling party WILL send a CANCEL. The CANCEL can be used to generate
the
STOP record
-----Original Message----- From: serusers-bounces@iptel.org [mailto:serusers-bounces@lists.iptel.org] On Behalf Of Eric Haskins Sent: Thursday, December 01, 2005 3:26 AM To: Frank Fischer; 'Dave' Cc: serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
If I am correct I believe CDRTool uses mediaproxy to prevent the phantom
BYE
record. Adrian can correct me if I read wrong. Invites without BYE's are common
No
one answered the phone. We just Log and drop records that dont have matching
INVITE,ACK,
and BYE. The reason as I have seen for the missing BYE (discussed alot on here) is the UA or the Gateway dont send a BYE for whatever reason. We havent seen that many , infact
not
one yet knock on wood. We thought about just estimating and calling them one minute calls.
The problem we are having is we route some wholesale traffic thru us via trusted IP and account for it (Easy Part) but we also get DID's from that Company so all traffic
comes
from same IP. DID's work fine but it broke our parser. LOL was charging them for our incoming
calls
OOPS!!! I guess for now I am going to move the wholesale Trusted IP stuff to its own box.
Eric eric@rackspeed.net
----- Original Message ----- From: "Frank Fischer" frank.fischer@digitalnomads.ch To: "'Dave'" ddx66@yahoo.com Cc: serusers@lists.iptel.org Sent: Wednesday, November 30, 2005 5:14 PM Subject: RE: [Serusers] Creating CDRs from SERs accouning records
Do i get this right - the matching is just done by searching a matching INVITE for each BYE where matching means that the callid must be the same? Anf of course by calculating the callduration, normalize certain fields and "grap" phonenumbers out of the sip uri? Is it really that "simple"? Since, on one hand side, i read about cases where there is no BYE at the end of a call (i think it had to do with mediaproxy), on the other hand side, when i look at "my" acc table, i can find some records, where there are multiple BYEs with the same callid or INVITEs without a BYEs.
I understand, that missing BYEs primarly are a problem of the accounting, not of the normalization, but, how ever, i'm quite sure that i won't be possible to have a 100% accurate accounting so i thought that there might be the need/possibility to do some "fixing" during normalization. I.e. i think i understood that the commercial version of CDRTool does so for missing BYEs?
Am i completely wrong?
Regards Frank
-----Original Message----- From: Dave [mailto:ddx66@yahoo.com] Sent: Wednesday, November 30, 2005 10:57 PM To: Frank Fischer; serusers@lists.iptel.org Subject: Re: [Serusers] Creating CDRs from SERs accouning records
The downloadabe CDRTool gives you nothing practical. You can not do anything useful with it. So I would not waste my time on that.
--- Frank Fischer frank.fischer@digitalnomads.ch wrote:
Hi all
i'm looking for a free/opensource application that generates CDRs (not XDRs, must not be rated) from the accounting data/events generated by SER (i think this is usually called normalization). I think this would have to be an application that is able to match INVITEs and BYEs and form CDRs with call start timestamp, call duration and so on.
I had a look at the list of accounting software on the iptel.org website, but most of there applications are highly integrated accounting/billing solutions, far more than what i need (and also most of them seem to be commercial products). In the mailinglist archive i only found some references to CDRTool which seems also to be commercial. Obv. there must be a free version of CDRTool but neither i'm sure whether it implements exactly what i need so i could "grab it" for my own purposes nor could i find a place to download the free version).
Does anyone know if there is such a kind of application available? Or any other idea how to deal with my requirements?
Thanks a lot for your help Frank
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Start your day with Yahoo! - Make it your home page! http://www.yahoo.com/r/hs
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers
Serusers mailing list serusers@lists.iptel.org http://lists.iptel.org/mailman/listinfo/serusers