AITIA International, Inc.	SIP CDR Assembler module	SIPAssembler

SIP CDR Assembler module
`("SIPAssembler")`

Features

SIPAssembler is a CDR (Call Data Record) generator module, based on SIP (Session Initiation Protocol) messages.
The software supports the following RFCs:
The software is able to receive SIP messages from multiple input connections, in parallel. The following input formats are allowed:
- File (sga format)
- TCP stream (sga format)
- SgaMUFI2 storage format
The output of the software is a CDR data file. Text-based output is also available for debugging.

User interface

The main menu

Menu	Menuitem	Meaning
[File]
	Load state	Load internal states from a previously saved state file.
	Save state	Save internal states to file.
	Log counters	Write counters to log view, into a new line.
	Save counters	Save counters to file.
	Exit	Exit program
Tools
	Clear state	Reset internal states to initial value.
	Flush state	Send messages to output channels from internal buffers.
	Clear counters	Reset counters to 0.
	Settings	Set ini file settings from menu.
	Dump active calls	Write active MSISDNs into a text file.
Help
	About	About the Software

Multiple views

SIPAssembler has 3 views:

Log
Assembler
Connections

"Log" view

If we start the software, the Log view is the default window.
The log level can be set through a drop-down list, the default value is defined in the ini file. If the ini file does not exist (first-start without ini file), the default value is "Normal".
To search in the log entries, there is a text-field next to the drop-down list. The next button applies the text search and shows the next match.

Each entry has a category, which is represented by the following icons:

Information about the operation
Warning: probably unplanned event occurred
Error: unplanned error during operation

"Assembler" view

The "Assembler" window shows counters about the call data record generation. There are four sections:

General
Call statistics
Message statistics
Transaction statistics

Counters in "General" section:

Current message time: time stamp of the last processed message.
Incoming messages: Number of processed SIP messages.
Incoming non SIP messages: Number of processed non-SIP messages.
SIP messages written to CDRs: Number of SIP messages, which are collected to CDR data files.
SIP records written to CDRs: Number of written records.
Active message count: Number of SIP messages in internal buffers.
Active transaction count: Number of active records in internal buffers.
Active call count: Number of call-summaries, which IDs are currently active.
Active user id count: Number of different From-MSISDNs.
Transaction UID success ratio: Rate of records, which contains or not contains MSISDN or IMPI value.
Call UID success ratio: Rate of call-summaries, which contains or not contains MSISDN or IMPI value.
Processing error count: Number of SIP messages, which result internal processing error.
Invalid SIP messages: Number of SIP messages without Method, From tag and To tag parameter.

Counters in "Call statistics" section:

Normal open: Number of opened call-summaries, in which the INVITE dialogs contains normal opening handshake.
Incomplete open: Number of opened call-summaries, in which the opening messages are missing.
Normal close: Number of closed call-summaries, in which the INVITE dialogs contains normal closing handshake.
Error close: Number of closed call-summaries, which contains failure response message code.
Limited close: Number of closed call-summaries by the software, because of memory or message count limitation.
Timeout: Number of closed call-summaries by the software, because of expired time interval.

Counters in "Message statistics" section:

This section counts the SIP Request and Response message types, in the following format: Each row means a Request message type, and each column means a Response class to the Request (except REQ column, which shows the Request count).
The supported SIP Request message types are: INVITE, ACK, OPTIONS, BYE, CANCEL, REGISTER, PRACK, SUBSCRIBE, NOTIFY, MESSAGE, INFO and UPTDATE.

Counters in "Transaction statistics" section:

This section counts the call data records, based on its type.
Each row means a CDR type, and each column means a CDR opening/closing reason.
CDR opening/closing reasons:
- Normal open: Number of records, in which the first message of the CDR is a normal SIP transaction/dialog opening message.
- Normal open(call): Number of INVITE type records, in which the first message is not initial-INVITE, but based on other messages, it is a normal operation.
- Incomplete: Number of records, in which the first message(s) of the SIP transaction/dialog is missing.
- Active: Number of active records, which are not closed yet.
- Normal close: Number of closed records, which contains normal SIP transaction/dialog closing procedure.
- Normal close(call): Number of closed records, in which the normal transaction/dialog closing messages are missing, but based on other messages, it is a normal operation.
- Error close: Number of records, in which the transaction/dialog closing message has failure code.
- Timed out close: Number of records, which are closed after an expired time interval.
- Limited close: Number of closed records by the software, because of memory or message count limitation.
- Overlapped close: Number of closed records by the software, because of reappeared SIP parameters.

"Connections" view

The "Connections" window shows the current input and output connections. Connections can be configured in the ini file, or in the "Connections" windows. After closing the software, the opened connections are saved to the ini file.

To open an input connection, type one from Connection's URI scheme in the text field, and left click to "Open" button:
To stop the incomming or outgoing traffic on a connection, click on the "Suspend" button.
To restart/enable the incomming or outgoing traffic on a connection, click on the "Resume" button.
To close the incomming or outgoing connection, click on the "Close" button.

Each connection has a statistic counter set.

Counters in an incomming connection: (outgoing connection counters are same)

Channel name: name of the opened connection.
Channel state: state of the connection (Opening, Opened, Suspended).
Channel resource: path to the output module, or the incoming message path.
Incoming bytes: summarized bytes, which are received/sent.
Incoming messages: summarized number of received sga items.
Lost bytes: lost bytes during reception.
Lost messages: lost messages during reception or processing.
Buffer usage: buffer usage ratio.
File position: index position during MUFI file processing.
Mufi file name: name of the Mufi file, in a case of MUFI file processing.

SIP Call-Summary

A SIP Call-Summary is a set of INVITE dialogs, which are grouped by the ICID (IMS Charging ID) and by the Call-ID parameters.
Each Call-Summary belongs to one user (call) activity.
Each Call-Summary has a Call-Summary-ID, which is searchable in the CDRViewer software.
The SIPAssembler generates statistics for the Call-Summaries, in the Call statistics section of the Assembler view.

Configuration (`SIPAssembler.ini`) file settings

Section	Entry	Example	Meaning
[CDR]
	CompressionLevel	6	Level of the output data file compression. 0: no compression, 9: maximum compression.
	DataFilePeriodMin	5	Time interval of the output data file, in minutes.
	Encrypt	1	Output file encryption. 0: do no encrypt data, 1: encrypt data
	PathPrimary	d:\SIP_CDRs	Data path of the generated CDR files.
	PathAlternate	d:\SIP_CDRs_alternate	Alternative path for the generated CDR files, if the PathPrimary is unreachable.
	PreallocationSize	0
[Input]
	StoragePointerFilePath	d:\DataFiles-7700\Sga-7700	File path of the SGA-7700 files.
	InputFromStorage	d:\DataFiles	File path of the input sga files.
[Log]			Logging related parameters
	LogDetailLevel	1	Stored level for Log message filter. {0-Off,1-Normal,2-Detailed,3-Debug}
	LogFilesPath	d:\LogFiles	Directory used for storing log files.
[Output]			Other output definitions. Please visit Connection's URI scheme.
[RemoteUI]
	TCP_host	127.0.0.1	IP Address for command interface.
	TCP_port	16001	Port for command interface.
[SIPAssembler]
	ActiveCallDumpFile	d:\LogFiles\calldump.txt	Path for the list of active MSISDNs.
	CallDelayCloseTime	180	Call-Summary closing delay (in seconds) after a closing event.
	CallRollingTime	300	The size of the buffer per INVITE type records, in seconds. If the time interval expires, and the buffer contains messages, the software writes it to the output.
	CallTimeout	3600	Closing time interval (in seconds) for INVITE type records. If the time interval expired and the closing message do not arrived, the Call-Summary and the INVITE records are closed by the software.
	CounterHistoryFile	d:\StatFiles	Path for the counter statistics file (.csv format).
	MaxCallCount	500000	Number of maximum active Call-Summary IDs. 0 - no maximum value
	MaxDatagramCount	200000	Number of maximum SIP messages, stored in the internal buffers.
	MaxTransactionCount	200000	Number of maximum active record IDs.
	StateFile	d:/StatFiles	Path for the state file, which represents the last saved internal states.
	TransactionDelayCloseTime	5	A time interval (in seconds) for non-INVITE records, which represent a delayed close after a closing event.
	TransactionRollingTime	10	The size of the buffer per non-INVITE records, in seconds. If the time interval expires, and the buffer contains messages, the software writes it to the output.
	TransactionTimeout	20	A time interval (in seconds) for an opened non-INVITE record. If the time interval expired and the closing message do not arrived, the record is closed by the software.
[TextWriter]
	Append	false	The output will be appended to the end of the existing file if the value is true.
	OutputFile	d:\TimeoutCDRs.txt	The full path and the name of the output file.
	OutputFilter	false	Keeps the record on the output if the string exists in the text.
	OutputOptions	Off	Off/All/Normal/TimedOut/Overlapped

Connection's URI scheme

SIPAssembler module is able to operate with the following URI-like format:

file-pcap://path
file-sga://path
socket-sga://IPAddress:TCPPort
socket-sga://IPAddress:TCPPort?listen=1
mufi-sga://path\__DATE__.SgaMon\SgaMon__TIME__.sga?mufi=path\Sga-7700&fileheader=false

If the file name contains space, file-sga://'path' format must be used.

Version history

v1.52

(n) Dynamic rolling if the datagram count limit has been reached
(n) Channel config saving. The input/output channels are saved into the ini file.
(c) MUFI config is changed, the URI must contain the "index" parameter to declare the mufi files location
(n) Unprocessed message saving into the state file
(n) Invite transactions with 'Proxy auth req (407)' is closed and a new transaction is opened in the same call.
(n) MRFC initiated transaction handling
(f) State loading is enabled only if the processing is inactive

v1.51

(c) The "Stop processing request by operator" and "Start processing (automatic)" lines are added to the log.
(n) Connection opening/closing functionality is logged
(n) State clearing is completed (fixed), the whole state is cleared
(n) Counters logging feature is removed
(n) ISUP ended call summary closing based on the 'to' user identity
(n) The 403 'Forbidden' response closes the INVITE and the MESSAGE transactions
(n) The row and column indexes replaced with the table row and column names on the counters CSV output

v1.50

(c) More memory usage allowed by switching on large address space handling

v1.49

First release

Known bugs

ICID collision handling

SIP CDR Assembler module("SIPAssembler")