AITIA International, Inc. INet Reassembler module INet Reassembler

Internet and Transport Layer Reassembler module
("INetreassembler")

Table of Contents

Features

User interface

The main menu

Menu Menuitem Shortcut(s) Meaning
[File]      
  Load state Load internal states from a previously saved state file.
  Save state Save internal states to file.
  Log counters Write counters to log view, into a new line.
  Save counters Save counters to file.
  Exit Exit program.
Tools      
  Start processing Start processing given input(s).
  Stop processing Stops processing of the input(s).
  Clear state Reset internal states to initial value.
  Flush state Send messages to output channels from internal buffers.
  Clear counters Reset counters to 0.
  Settings Set INI file settings from menu.
  Save Settings Save settings to INI file.
Help      
  Help Brings up help screen.
  About About the software.

Multiple views

 The INet Reassembler has 9 views:

If we start the software, the Log view is the default window.
The log level can be set through a drop-down list, the default value is defined in the INI file. If no INI file exists, the default value is "Normal".
To search in the log entries, there is a text-field next to the drop-down list. The next button applies the text search and shows the next match.

Each entry has a category, which is represented by the following icons:


The "INet view" window shows counters for the overall operations. There are four sections: Counter in the "General" section Counters in the "Result details" section Counters in the "Runtime" section Counters in the "Buffer" section

The "IPv4 view" window shows counters about IPv4 message reassembly. There are three sections: Counters in "General" section: Counters in "Traffic details" section: Counters in "Result details" section:

The "IPv6 view" window shows counters about IPv6 message reassembly. There are three sections: Counters in "General" section: Counters in "Traffic details" section: Counters in "Result details" section:

The "UDP view" window shows counters about UDP message reassembly. There are three sections: Counters in "General" section: Counters in "Traffic details" section: Counters in "Result details" section:

The "TCP view" window shows counters about TCP stream reassembly. There are three sections: Counters in "General" section: Counters in "Traffic details" section: Counters in "Result details" section:

The "SCTP view" window shows counters about SCTP message reassembly. There are three sections: Counters in "General" section: Counters in "Traffic details" section: Counters in "Result details" section:

This view shows the currently used inputs and outputs. Connections can be configured in the INI file, or in the "Connections" windows. After closing the software, the opened connections are saved into the INI file.

Details of connections can be collapsed and expanded using the "expand" triangle button at the right up corner of the connections.

Parts of "Connections" view

Connection management

To open an input connection, type one from possible connection's URI schemes in the text field, and left click to "Open" button.

Each connection has a statistic counter set.

Counters in an incoming connection: (outgoing connection counters are same)

The "Terminal" provides direct access to the core logic using commands. Available commands are listed on the right. Typically used for debug purposes.

Supported protocols

Ethernet

The Ethernet protocol assembler can process input Ethernet frames. Frames with VLAN 802.1Q headers are supported. It does not collect any detailed statistics about the link layer.

The assembler will leave the 4 bytes CRC sequence at the end of the frame if present because it can be missing in some input and there is no way to recognize if it is present or not at this point of the processing.

IPv4/IPv6

The IP protocol assemblers have multiple functions: defragmentation, checksum validation, extension header processing, and payload extraction.

Duplicated IP fragments are filtered out during processing.

Supported extension headers: hop-by-hop, routing, fragment, AH, ESP, Destination OP.

UDP

The UDP protocol assembler simply extracts the upper-layer payload and validates the checksum if present and validation is configured.

TCP

The TCP protocol assembler follows the TCP connections and reassembles the pair of streams associated with the connections. It stores a message queue by streams and reorders the TCP segments by their sequence numbers. Fragment duplication, overlappings of sequence numbers and payloads are handled, every byte of the stream will be present in the output only once.

The assembler follows TCP ACKs and indicates if the reassembled segment was acknowledged.

TCP checksums can be validated if the function is configured.

SCTP

The SCTP protocol assembler follows the SCTP associations, defragments and reorders the embedded application-layer payloads.

The assembler follows SCTP ACKs and indicates if the reassembled segment was acknowledged.

SCTP checksums can be validated if the function is configured.

The logic ignores the posibility of multihoming and presumes that all SCTP association is established between two end point defined with their singular IP address and port.

Input and output

Input

The application expects captured Ethernet frames or IPv4/IPv6 packets on its input. Besides the normal input, transport layer packets with existing IP meta information (attached pseudo-header) can also be processed.

The application can process and assemble unordered input traffic if the time delay between messages are below the configured timeout interval of the affected protocol reassemblers.

Output

The application outputs reassembled packets or stream segments of the application layer. The output order for stream-based protocols based on the stream order instead of captured timestamp, however, the output message timestamp will be rewritten to have a monotone increasing timestamps.

UDP and TCP payloads get unknown payload protocol id, SCTP payloads can get the corresponding application protocol id if the SCTP packet contains the necessary meta-information.

The application appends meta-information about the IP address and port pair to every output message in the form of a pseudo IP header. Other TCP, SCTP transport-related meta info can also be added to the packet if configured.

If configured, the application can bypass and send out every input message without modification on its output and can track the position of the application layer packet in the original input messages. This information can be appended to output messages as additional meta-information if configured. In case of message duplication, the application is able to send duplicate message events on its output indicating which original messages are duplicates. These functions are useful if original messages are needed to be edited later on the processing chain based on the processed application layer protocol, for example, to redact sensitive information from application layer data in the original messages.

The output can be directed to different targets based on the resulted payload protocol using the output protocol filter (see protocol connection URI parameter).

Messages which cannot be processed successfully are getting dropped. These dropped messages can be directed to an output connection for debugging purposes (see dropped connection URI parameter). If configured, dropped packages appended with debug information containing the reason why the processing failed.

Connection's URI scheme

The INet module is able to operate with the following URI-like format: If the file name contains space, file-sga://'path' format must be used.

Performance

 Internet and transport layer protocols are processed in two separate threads. The software provides some non-essential functionality that can be switched off to increase processing speed.

Configurable non-essential functions

Processing of transport layers (UDP, TCP, SCTP) can be switched off. In this case, the application will output the transport layer packets unprocessed. These messages can be processed further by other INet reassembler instance(s) with enabled transport layer processing.

Memory consumption

Messages for IP, TCP, SCTP are stored in the memory during processing. The available storage space can be controlled via config to prevent memory overuse by setting maximum stored message count and timeout.

Statistics calculated by the INet ReAssembler software

 The INet ReAssembler module collects and calculates SCTP statistics when it is enabled by the [SCTPReassembler] / CollectStreamStats INI entry (True|False), and writes them into Sga-7001.NNN files (path definition: INI / [SCTPReassembler] / StatsSuperMonPath entry) for the SuperMon module which forwards them to the Sga-7N Poller. SCTP statistics written out in every 15 min and at exiting - accumulated by linkID.

Statistics:

StatID Description
7000 Number of incoming SCTP messages without IP header
7001 Number of incoming SCTP message bytes without IP header
7002 Number of Data chunks
7003 Number of Init chunks
7004 Number of Init Ack chunks
7005 Number of Sack chunks
7006 Number of Heartbeat chunks
7007 Number of Heartbeat Ack chunks
7008 Number of Abort chunks
7009 Number of Shutdown chunks
7010 Number of Shutdown Ack chunks
7011 Number of Error chunks
7012 Number of Cookie Echo chunks
7013 Number of Cookie Ack chunks
7014 Number of Ecne chunks
7015 Number of Cwr chunks
7016 Number of Shutdown Complete chunks
7017 Number of duplicated messages
7018 Number of out of sequnce messages
7019 Number of missing messages
7020 Number of Gap Ack Blocks
7021 Number of duplicate TSNs the endpoint received

Configuration (INetReassembler.ini) file settings

Section Entry Example Meaning
[Counters]      
  ClearOnWrite False Clear counters after history file written.
  HistoryFile D:\LogFiles\INetReassembler.csv The path of the counter history file.
  HistoryPeriod 15 The counter history file writing period in minutes.
[INetReassembler]      
  BypassInput False Send out every input message when received beside processing it.
  DebugInfo False Write debug info on output for dropped messages.
  SendEventsOnDuplicates False Send out events when message filtered out due to duplication.
  TrackPayload False Write position info of payload in original message for output messages.
[IPReassembler]      
  MaxActivePackets 10000 Maximum number of packets buffered.
  PacketTimeOutMSec 1000 Maximum lifetime of IP packets in millisecond.
  StoreFragmentsTillTimeout False Delay fragments till timeout to detect duplicated closing fragments.
  VerifyChecksums False Calculate and verify checksum of each packets.
[Input]      
  RestoreChannels True Restores the active channels after restart.
  SortingQueueDelay 0 The sorting queue delay in milliseconds.
  SortingQueueLength 0 The maximum size of the sorting queue.
  StoragePointerFilePath D:\DataFiles-7700 Path for optional storage pointer files (Sga-7700.xxx), if MUFI2 input is used
  SynchronizedIO 0 The input and the output channels will be synchronized.
  InputMufi mufi-sga://E:\Storage1\Datafiles\__DATE__.SgaMon\SgaMon___TIME__.sga?fileheader=false&mufi=E:\Storage1\Datafiles-7700\Sga-7700&verbose=true&storepos=true The input channel definition, called "InputMufi"
[Log]      
  LogDetailLevel Normal Stored level for Log message filter.
  LogFilesPath D:\LogFiles Directory used for storing log files.
[Output]      
  RestoreChannels True Restores the active channels after restart.
  SortingQueueDelay 0 The sorting queue delay in milliseconds.
  SortingQueueLength 0 The maximum size of the sorting queue.
  gtpassembler socket-adg://127.0.0.1:4241 Definition of a TCP socket type output connections
  sga file-sga://INet.sga
[RemoteUI]      
  TCPHost 127.0.0.1 IP Address for command interface.
  TCPPort 16000 Port for command interface.
[SCTPReassembler]      
  CollectStreamStats False Collects statistic informations by streams, needed for any statistics output.
  Enabled True Enables SCTP assembly or sends out SCTP packets untouched.
  LookBehindOnStartUp True Delay packets on startup to handle late packets.
  MaxActivePackets 10000 Maximum number of packets buffered.
  PacketTimeOutMSec 2000 Maximum lifetime of SCTP packets in millisecond.
  SctpInfoOnOutput True Add SCTP assembly related info to output messages.
  StatsCsvPath D:\Statitics\INet_sctp_stat.csv Directory used for storing statistics in CSV format.
  StatsSuperMonPath D:\DataFiles-7001 Directory used for write statistics into Sga-7001 files for SuperMon.
  StreamTimeOutSec 60 Maximum lifetime of SCTP stream after last packet sent in sec.
  TrackMissingChunks True Tracks missing chunk for precise duplicated message calculation.
  VerifyChecksums True Calculate and verify checksum of each packets.
  WaitForAck True Delay packet max. till timeout to collect its ACK.
[State]      
  AutoStart True The initial state of the processing.
  StateFile INet.state
The path of the state file.
[TCPReassembler]      
  Enabled True Enables UDP processing or sends out UDP packets untouched.
  LookBehindOnStartUp True Delay packets on startup to handle late packets, if no SYN have been seen.
  MaxActivePackets 10000 Maximum number of packets buffered.
  PacketTimeOutMSec 2000 Maximum lifetime of TCP packets in millisecond.
  SessionTimeOutSec 60
Maximum lifetime of TCP sessions after last packet sent in sec.
  TcpInfoOnOutput True
Add TCP assembly related info to output messages.
  VerifyChecksums True
Calculate and verify checksum of each packets.
[UDPReassembler]      
  Enabled True Enables UDP processing or sends out UDP packets untouched.
  VerifyChecksums True
Calculate and verify checksum of each packets.

Version history

Known bugs