TSP Lab	SgaS1APAssembler - S1AP CDR assembler module	SgaS1APAssembler

The S1AP CDR assembler module
`("SgaS1APAssembler")`

Basic Operation
User interface
The network view
The miscellaneous view
Configuration
Version history

Basic Operation

Accepts connection from S1APFilter module
Receives and processes S1AP traffic on the S1-MME interface
Selects the S1AP messages to process
Assembles the connections based on the mme-ue-s1ap-id and the enb-ue-s1ap-id
Assigns S1AP/Pagings to connections based on the M-TMSI
Prints out the assembled connections into records
Connects to the KeyServer4G and asks it for ciphering keys and IMSIs
Updates KeyServer4G's memory with collected information
Assigns IMSIs to connections acquired from the Key Servers
Deciphers protected NAS contents
Deciphered messages are forwarded to a MessageStore
Logs the counters quarter-hourly and print out in .csv format

User Interface

The main menu

Menu	Menuitem	Shortcut(s)	Meaning
[Actions]
	Exit		Exit
	Load state		Loads a previously saved memory dump
	Save state		Saves the content of the memory into a dump file
	Clear counters		Resets Misc view counters
	Log counters		Saves counters to log file
	Save counters		Saves counters to .csv file
	Refresh text writer config		Reloads configuration of text writing feature from the [TextWriter] section of the INI
	Reload thread number config		Reloads thread numbers from INI/[Decipher]/Threads entry
[Connections]
	Stop/start data processing		Sets data processing inactive/active
	Close incoming channels		Shuts down incoming channels
	Reload incoming channels config		Reload and activates incoming channels config from INI
[Log]
	Find line containing...	Ctrl+F	Find expression in logfile
	Find next line	F3	Find next occurrence of the expression above
	Log level		Four options available, each one increases information amount written to log: 0: Off 1: Normal 2: Detailed 3: Debug
[View]
	Detailed network status		Not used.
	Show log	Ctrl+L	Show log window
	Show network	Ctrl+N	Show network window
	Show miscellaneous	Ctrl+M	Show misc window

The network view

This view shows the status of communications channels.

Incoming channels

Name: Name of the channel and the current state:
- Opened: Remote side connected
- Opening Input TCP port is opened and application is listening on it
Resource: Local IP address and port of SgaS1APAssembler module
Incoming bytes: Number of received bytes
Incoming messages: Number of received messages

Outgoing channel

Name: Name of the channel and the current state
- Opening Trying to establish the TCP connection
Resource: Remote IP address and port of Message Store module
Outgoing bytes: Number of sent bytes
Outgoing messages: Number of sent messages

The miscellaneous view

Counters in the miscellaneous view

General

Processing state: State of processing (Active/Inactive)
Key server connection state: Key server socket connection state
Program start time (GMT): Indicates the date and time when the program started
Current message time (GMT): Indicates the date and time of the currently processed incoming message
Processing speed: The fraction of the message time and the 'real' time. The values under 1 indicate that the assembler is unable to process the incoming data flow. Normally this counter oscillates near by 1.
Program path: The path of the program executable file
Log file path: The log file path
Counter history path: The path of the counter history file
CDR primary path: The primary path of CDR files
CDR alternate path: The alternate path of CDR files
Timeout queue length: Number of connections waiting for the expiration of the time out
Allocated memory units count: The count of the allocated memory blocks
Allocated memory size: The size of the allocated memory
IMSI ratio in CDRs (all): Ratio of CDRs containing any IMSI in all CDRs
IMSI ratio in CDRs (certain): Ratio of CDRs with certain IMSI (seen in message or validated by decipher) in all CDRs
S1AP messages under processing: The difference of the incoming and the outgoing messages. The value depends on the incoming traffic speed, but it must be always limited.
S1AP messages written to CDRs: Number of messages written into CDRs

S1AP Messages

Processed S1AP messages: Number of processed S1AP messages / message rate
Dropped S1AP messages: Number of dropped S1AP messages (which do not contain user related information) / message rate
S1AP messages without known MMEID: Number of S1AP message with non-defined MME (The IP address of the MME is not listed in INI/[Assembly]/MMEParams entry) / message rate
Disordered messages: The count of those messages which arrived with earlier time stamp then the lastly processed message. It may indicate monitoring problems.

S1AP Paging Groups

S1AP paging groups: Number of incoming paging groups / paging group rate
S1AP paging groups attached: Number of paging messages that are attached to a connection / paging group rate
S1AP paging groups timed out: Number of paging groups that can not be attached to any connection within the INI/[Assembly]PagingTimeoutSecvalue / paging group rate

S1AP Connections

Opening reasons
- Normal: Connections with a regular opening message, i.e. Paging or InitialUEMessage
- Incomplete: Connections not containing any opening message (missing opening message due to some message loss, e.g. incomplete monitoring)
- No paging: Connections supposed to be opened with paging (InitialUEMessage/RRC-Establishment-Cause=2 mt-Access) but no related paging has been seen
- Handover: Connections opened with HandoverRequest message
Connections opened: Number of connections have been opened since the module was started
Connections active: Number of open connections in memory

Closing reasons
- Release: Number of connections closed with UEContextRelease procedure
- Reset: Number of connections closed with Reset procedure
- Error: Number of connections closed with ErrorIndication procedure
- Overlapped: Number of connections that are closed due to another connections have been opened with the same connection identifiers
- Timed out: Number of connections for that there have not seen any closing message within INI/[Assembly]/ConnectionTimeoutMin
- Memory limit: Number of connections that are closed due to exceeding the memory limit (INI/[Assembly]/MemoryLimitMB)
Connections closed: Number of closed connections

Decipher

Connections under deciphering: Number of active connections known by decipher
Deciphering success ratio: Ratio of deciphered messages in all ciphered messages
Messages under deciphering: All ciphered or non-ciphered messages waiting or under processing by decipher
Keys cached by MTMSI: Number of key vectors currently cached in decipher
Non-protected messages: Non-protected messages processed
Protected messages: Proteced (non-ciphered) messages processed
Deciphered messages: Successfully deciphered message
Ciphered messages use unknown key: Ciphered messages which no actual or any key found for
Cip. msgs. with insufficient key: Ciphered messages with non-4G actual key
Corrupt ciphered messages: Ciphered messages with failed attempt of deciphering due to wrong key or NAS counter
Messages deciphering failed on: Ciphered messages with failed attempt of deciphering due to unknown message direction, insufficient memory or internal error.

Key server update

Update GUTI: Number of GUTI updates sent for key server
Update NAS up count: Number of NAS up counter updates sent for key server
Update NAS down count: Number of NAS down counter updates sent for key server
Update algorithm: Number of algorithm updates sent for key server
Update XRES: Number of XRES updates sent for key server
Delete IMSI: Number of IMSI deletes sent for key server
Recalc KASME: Number of KASME recalculations sent for key server

Key server query

Active requests: Number of responses of sent requests currently the decipher waiting for
Sent request IMSI: Number of IMSI queries sent
Sent request info with GUTI: Number of key by GUTI queries sent
Sent request info with IMSI: Number of key by IMSI queries sent

Request type
- Found: Number of requests when the KeyServer found the requested information
- Not Found: Number of requests when the KeyServer did not find the requested information
- Timeout: Number of requests when the KeyServer did not answer within timeout interval ([Decipher]/KeyServerTimeoutMSec)
Request IMSI: Number of requests for IMSI by M-TMSI
Request info with GUTI: Number of requests for cipher info with GUTI
Request info with IMSI: Number of requests for cipher info with IMSI

Short cuts (only for debug purpose)

Alt-D: Dump whole application states into a text file in JSon format.
Alt-K: Load keys for decipher from JSon format key file.
Ctrl-Alt-K: Dump keys from decipher key cache into a JSon format key file.
Alt-B: Switch on deciphering debug info in message output.
Ctrl-Alt-B: Switch off deciphering debug info in message output.
Ctrl-A: Write only ciphered messages into message output.
Ctrl-Alt-A: Write all messages into message output.

Configuration (SgaS1APAssembler.ini) file settings

Section	Entry	Example	Meaning
[Window]
	MainLeft	0	Distance from the left edge of the screen at startup
	MainTop	173	Distance from the top of the screen at startup
	MainWidth	1280	Width of the window
	MainHeight	358	Height of the window
	MainZoomed	0	Startup screen size: 0: Normal 1: Fullscreen
	SilentClosing	0	If 1 the "Are you sure?" window does not appear at exit. If 0 the "Are you sure?" window appears at exit.
[Log]
	Directory	d:\LogFiles	Path of the logfiles
	Level	0	Log detail level at startup: 0: Off 1: Normal 2: Detailed 3: Debug
[Input]
	SocketInput	socket-null://127.0.0.1:42001?listen=1	Name and definition of input connection
[Assembly]
	ProcessingState	0	Initial state of data processing
	PagingTimeoutSec	5	Time out for attaching paging groups to an open connection
	ConnectionTimeoutMin	120	Time out for inactive connections. The closing reason value in the CDRs is "time out".
	ConnectionDelayedCloseIntervalSec	6	Defines how long the module should wait for late coming messages for a connection marked as closed
	ConnectionRollingTimeSec	300	Messages of a connection are flushed into a CDR when this time elapses after the last processed message. The connection remains opened in the memory with its parameters, only the messages are written into a record and deleted from the memory.
	ConnectionPSHoldTimeMSec	100	Hold time for connection ids' after successful path switch
	MemoryLimitMB	1024	When the size of allocated memory exceeds this value then the oldest connections are being closed. The closing reason in the CDRs indicates this case.
	CounterHistory	d:\	Path for CSV file containing counters
	CounterReset	0	If 1 then Misc view counters are reset in every 15 minutes after flushing them into the log and the S1APFilter_counters.csv. If 0 then the counters are not reset.
	StatePath	d:\	Path for memory dump
	MMEParams	ip:10.242.136.248	Defines MME parameters (listed in one row as a single entry)
[Text Writer]			Entries in this section can be reloaded by executing [Action]/Refresh text writer config menu item on the GUI
	WriteMessages	false	Enables/disables messages in TextWriter output
	WriteConnections	false	Enables/disables connections in TextWriter output
	OutputCondition	false	If a string is defined here then the output contains only those lines which contain the string. Empty entry means all lines are written.
	OutputFileName	e:\Sga-S1AP-CDR\CDR\assembler.out	Output file for TextWriter. If this entry is empty or commented with a ';' then TextWriter feature of disabled
[CDR]
	PathPrimary	e:\Sga-S1AP-CDR\CDR\	Primary path for CDR files
	PathAlternate	c:\SgaS1APAssembler\CDR-Sec\	Alternative path for CDR files for the case of inaccessibility of the primary path
	DataFilePeriodMin	15	Period of closing database files
	PreallocationSize	10000	Preallocated size of CDR files, in bytes. PreallocationSize is read on-the-fly
	CompressionLevel	3	Compression level for CDRs. 0 means compression is switched off.
	Encrypt	1	Enables encryption of CDR files; 0 means off.
[Decipher]
	MCC	216	Used to build GUTI for key server query, when only PTMSI known from S1AP message
	MNC	30	Used to build GUTI for key server query, when only PTMSI known from S1AP message
	MMEGroupId	32768	Used to build GUTI for key server query, when only PTMSI known from S1AP message
	OwnMMECode	180	MME Code value of the current MME
	OwnNRIValue	22	NRI value of SGSN associated with the current MME
	OwnNRIValueBitWidth	5	Bit width of representation of OwnNRIValue
	KeyTimeoutSec	3600	Time interval while keys stored in a local key cache after received from key server. Note: keys won't be queried from key server while they stored locally.
	Threads	4	Number of parallel threads decipher will be working on
	LinkIdOverride	QDC	New link ID for outgoing messages
	KeyServerIP	127.0.0.1	IP address of the key server
	KeyServerPort	8011	Port of the key server
	KeyServerTimeoutMSec	100	Time interval which the decipher allowed to wait for response at any key server query Note: waiting will block the specific working thread
[MessageWriter]
	Output	socket-sga://127.0.0.1:1001	Message output. Format: < path > \| < ip:port > \| < port > \| < uri > Currently ip:port is used for passing messages to Message Store
	OnlyCiphered	1	Writes only ciphered messages into message output
	DebugInfo	1	Writes deciphering debug info into message output
[Debug]
	LogNasCounterUpdate	1	Will cause logging of NAS counter tracking into SGA file debug info
	AutoFlush	1	Flush messages from decipher after input connection stopped costs run-time, use this only for testing purpose

Version history

v0.64

(c) Sending Delete update inhibited for specific TAU reject causes.

v0.63

(c) DeriveRoundTripKASME sent only if Additional GUTI MME Code match with value of new INI parameter: OwnMMECode besides NRI value.
(c) GMT System time added to debug info lines of update requests.

v0.62

(c) Always queries IMSI before sending recalc*KASME to be able to remove cached key.

v0.61

(c) 'AddGUTI' in DeriveRoundTripKASME debug log line changed to 'MTMSI'
(c) INI parameters 'OwnMMECode*' renamed to 'OwnNRIValue*'
(f) 'Messages under processing' calculation corrected, now shows all messages currently stored in memory.

v0.60

(c) DeriveRoundTripKASME parameter: GUTI3G removed.
(c) RecalcKASME renamed to deriveKASME, handover/idle parameter added.
(n) Assembler desides which KASME derivation should be used based on configured MME code.
(n) IMSI queried with additional GUTI after deriveRoundTripKASME sent.
(n) New INI parameters introduced: OwnMMECode (def=22), OwnMMECodeBitWidth (def=5)
(c) OwnMMECode*ForKeyFile parameters removed.
(c) KASME derive type decision removed from KeyServer emulator.

v0.59

(n) New update request 'derivateRoundTripKASME' introduced, sent on seq.mode command if Additional GUTI known.
(c) Key cache won't recalculate KASME any more.
(n) Key file processing use new INI parameters: 'OwnMMECodeForKeyFile', 'OwnMMECodeMaskForKeyFile' to decide which KASME calculation should be used on 'derivateRoundTripKASME' request.
(f) Mufi won't try to remove an invalid data file and log warnings about it on startup.

v0.58

(c) Logic error log messages extended with message time stamp.

v0.57

(n) GUTI/IMSI processed from non connection-starter non periodic AUR.
(f) NASNonDeliveryIndication always contains downlink NAS message.
(c) Float counters always use fixed format.
(f) Deciphering ratio shows zero when no message processed yet.
(f) Crash when TextWriter can not open specified file fixed.
(n) TextWriter indicates direction of the message along with its linkID.
(c) Processing state indicator moved to Misc view in Filter.

v0.56

(n) Asks if state loading needed at startup if state file exists.
(c) Connection open/close are logged.
(c) Channel closing in S1APFilter is logged.
(c) Socket debug logs contain socket address and port from now.
(f) Tapper input format fixed.

v0.55

(f) Filter does not disconnect on Assembler processing stop.
(c) Debug log into Sga file on NAS counter jump (change > 5).
(c) Key server query/update debug log removed, debug info in SGA file can be used further on.
(c) Partial NAS counter tracking handles disordered messages.

v0.54

(n) Message writer filter can be changed during runtime with Ctrl-a, Ctrl-Alt-a shortcuts.
(n) Menu for setting thread number from INI.
(c) Certain IMSI ratio written to log and CSV.
(f) Processing state change menu corrected at startup if app. started with no processing.

v0.53

(c) System time logged into debug info on Keyserver query and response.
(f) 5 bit nas counter re-guessing before deciphering fixed.
(c) CDRWriter: IMSI is always taken from the connection object.
(n) IMSI is certain flag stored in connection, set if IMSI seen in first TARQ, !.icAUR, IdRsp msg or ciphering, protected checking was success
(n) Connection closing logged.
(c) UpdateFC on connection close sent when last message processing. For sake of timeing (and debug logging)
(c) Socket connection logs socket address and port on fail along with the error message.
(n) Processing state logged.
(n) Counter for certain IMSI in CDRs
(n) Connection logging in S1APFilter application switched on.
(n) CDR path indicated on assembler GUI.
(c) Full XRES and 6 byte of AUTN written to debug-info on UpdateXRES.
(c) Actual time written to counter CSV in local time.
(c) First & non ATRQ & !.icAUR & integrity protected message checked
(c) Revert due to runtime problem: integrity protected messages checked by decipher

v0.52

(c) Integrity protected messages checked by decipher
(f) State saving handles connections existing only in decipher
(c) Alt-d state dump includes assembler logic states also, format changed to json

v0.51

(n) [Debug]/LogNasCounterUpdate INI parameter will cause logging of NAS counter tracking into SGA file debug info.
(f) NAS counter tracked after NAS message processed to prevent counter update before query counter from previous connection.
(c) Prevent message order disruption by decipher worker feeder when 1 thread used.

v0.50

(n) Whole decipher state can be dumped to yaml format for testing purpose (Alt-d).
(f) Various fixes in state loading/saving.
(f) Counter time period loading form ini is fixed
(f) Rolling count handling is fixed
(f) Fixed under deciphering counters after state loading
(c) CDRWriter: the context IMSI is stored only if there is no IMSI in the messages.

v0.31

(n) Disordered messages are not processed
(n) Counters for disordered messages is added
(n) Path switch failure counter is removed
(n) Rolling opening reason is applied for non initial connections open
(f) Connection closing reason is fixed if the PDU type indicates failure
(n) Disordered messages is written in text output
(f) Occasiaonal text writer runtime config update crash is fixed

v0.29

(c) CDRWriter: Single paging groups now also have connection IDs.
(n) Counter reset is optional
(n) Reset init message is attached to all related connection
(n) Improved incomplete path switch handling, fixed message lost
(c) PathSwitchFailure opening reason has been removed
(f) Fixed alternate connection handling (the connections remained in the container because of wrong hash code)
(n) Counter for under processing message count
(f) Fixed message lost
(f) State saving fixes
(f) CDRWriter: Messages are now stored in time order.
(n) CDR counter, IMSI ratio counter
(n) Hold time for connection ids after succeesfull path switch
(n) New INI entry: [Assembly]/ConnectionPSHoldTimeMSec //Kapott txt-ből kimaradt

v0.27

(n) Counter for incoming and outgoing message count difference
(n) Counter for CDR records with IMSI
(f) State saving is fixed and checked
(n) Handling alternative connection ids after path switch
(n) Time order message storing
(n) Multiple reset message handling
(n) Periodical counter reset
(n) New INI entry: [Assembly]/CounterReset //Kapott txt-ből kimaradt
(n) State saving dialog before exit
(n) Connection integrity checking and logging

v0.26

(n) DTAPEMM ESM message type and cause in CDR
(n) S1AP cause in CDR

v0.25

(f) Fixed RAI handling in case of 4G key server query

v0.24

(n) Memory limitation capability
(n) New INI entry: [Assembly]/MemoryLimitMB //Kapott txt-ből kimaradt
(n) Connection timeout unit has been changed to minute

v0.23

First official release

The S1AP CDR assembler module("SgaS1APAssembler")

Table of Contents