Name
vtapericsson — Entry point for Ericsson 5g cloud monitoring
Synopsis
vtapericsson CONFIGFILE
vtapericsson [OPTIONS]
EXAMPLES
Generating a default config for an instance named foo:
vtapericsson -d >/etc/aitia/vtapericsson.foo.rc
Note: the application won’t start without setting the mandatory parameters in the generated config.
Starting the foo instance:
vtapericsson /etc/aitia/vtapericsson.foo.rc
DESCRIPTION
vtapericsson is a essential component of AITIA’s 5G monitoring.
The application handles the incoming vTap datastream generated
by Ericsson’s virtual network functions and filters and converts
data format to the sga 5G dataformat described in the design
documentation (SGA_5G_Monitorozas_alapjai).
The application has 3 functional layers:
-
Input layer, which handles the incoming connections and the
IPv4/UDP/GRE/PCAPNG/IPv4/TCP transport/encapsulation layers.
No data is captured from this headers, it is only used for
statistic calculations. 5 minute metadata packets are discarded
currently. Unlike other sga products a single GRE input channel
can accept multiple connections
-
Formatter layer, which converts the JSON payload of the datastream
to the unified sga 5G dataformat and generates linkID. Logic of
the process is described in the design documentation
(SGA_5G_Monitorozas_alapjai).
-
Outbound transport layer forwards the converted datastream for
storage or distribution or writes the data to disk. All connection
lib dataformats are supported. The application is stateless every GRE packet handled separately.
The information used for LinkID creation is stored in SGA MySQL
database which pooled periodically by the application for changes,
changes to the database are applied automatically. If the database is not available,
the application wont start if the database becomes unavailable
during operation then the application will used the cached data until
the database becomes available again.Performance considerations,
packet loss under high loads is highly dependent on the OS’s UDP
buffer size. Debian 11 has a very small buffer by default, it is 208KB.
This should be increased to 64MB.
Query UDP buffer size:
sysctl net.core.rmem_max
net.core.rmem_max = 212992
Set UDP buffer size (volatile):
sudo sysctl -w net.core.rmem_max=67108864
net.core.rmem_max = 67108864
Set UDP buffer size (non-volatile):
sudo echo "net.core.rmem_default=67108864" > /etc/sysctl.conf
sudo echo "net.core.rmem_max=67108864" > /etc/sysctl.conf
Example rc file:
# Command channel socket address in 'host:port' format
configs.set app.address 127.0.0.1:16022
# Automatic linkid database update period
configs.set linkid.refresh 1m
# URI for connecting to the SQL DB for linkid lookup. E.g.: 127.0.0.1?dbname=xxx&user=yyy&pass=zzz
configs.set linkid.uri '10.6.163.92:7008?dbname=sga&user=sga&pass=sga'
# The rulegroup of the application instance
configs.set linkid.group vTapEric
io.open in socket-gre://10.6.200.137:4754?type=udp&critical=false&name=c1
#io.open out file-sga://test_full.sga
io.open out socket-adg://127.0.0.1:9998?name=msgwriter_local
io.sorter in 1000 1000000
process.start
The input and output channel have to be configured and the process have to be started.
If multiple input channels are used the input io.sorter have to be configured for 1s sorting,
and every channel have to be configured as non-critical. The JSON value in the resp
parameter have to be put between ''. The application only uses filtering rules defined
for its group and group "All".
The application detaches itself from the terminal on startup.
It can be accessed via TCP command channel using sga-rc(1) application or
TCP socket. The application also responds to OS signals (see later).
OPTIONS
The config file of the application instance is a mandatory parameter.
Recommended directory for Aitia application config files is /etc/aitia,
recommended naming convention for config files is executable.instance.rc,
where executable is the name of the application binary and instance is
a user selected custom name for the application instance.
Note:
Without specifying a configuration file the application will fail on startup.
List of available options:
-
CONFIGFILE
-
path to the config file
-
-h, --help
-
print this help and exit
-
-j, --jsondoc
-
print documentation in json format
-
-v, --version
-
print version number
-
-d, --default
-
print default config file
CONFIGS
The config file contains the same commands that are available from
command interface.
Note: The application will fail on start if the specification of any mandatory parameter is missing.
Note: Time duration type config parameters need the specification of
the time unit. Available time units are: ns, us, ms, s, m, h, d.
Examples: 100ms, 5m.
List of available config parameters:
-
app.address (mandatory)
-
Command channel socket address in host:port format
-
linkid.group
-
Group name for Alias rule lookup
-
linkid.refresh (def: 1m)
-
Automatic linkid database update period
-
linkid.uri
-
URI for connecting to the SQL DB for linkid lookup. E.g.: 127.0.0.1?dbname=xxx&user=yyy&pass=zzz
COMMANDS
The application can receive user commands on the specified TCP/IP interface.
Command interface is using a bash like format. Every command should be in a new separate line.
Commands can have parameters, the command name and parameters are separated with whitespace characters.
If the parameter value contains whitespace, it should be escaped or quoted.
The backslash (\
) character escapes the following character (e.g: state.load /dir\ with\ spaces/statefile
).
The single quote ('
) character can escape multiple characters (e.g: state.load '/dir with spaces/statefile'
).
The hash (#
) symbol can be used to comment out parts of the command. The commented part starts at the comment character
and ends at the end of the line (e.g: state.load /path/statefile #opens the state file
).
io.sorter: io.sorter can be configured to empty sorter buffer if no message received in the last n milliseconds.
After the last incoming message a real-time timeout starts based on the CPU’s timestamp-counter register (rdtsc),
if no message received in the configured timeout then the sorter will be flushed.
Format: io.sorter <delay> <length> <timeout> , if timeout set for 0, then there is no flushing legacy mode
-
Legacy mode is recommended for non-test, multi input softwares, which work from buffered data channels eg.: modules after storage1
-
New mode recommended for non-buffered inputs (vtaps, writers) and very low rate links
-
Can be imprecise on very old hardware: X7-X8. Works well and tested on X9-X10-X11-AMD Epyc gen2-4
List of available commands:
-
app.quit <Force>
Quits the application. Inactive processing and saved state is required before non-forced quit.
-
<Force>
normal quit (by default) or forced.
(available values: force, normal)
-
configs.get <Parameter>
Returns a config parameter.
Note: Glob pattern can be used to specify more parameters.
-
<Parameter>
-
config name or glob pattern
-
configs.set <Parameter> <Value>
Changes a config parameter.
-
<Parameter>
-
config name
-
<Value>
-
New value to be stored
-
info.get
-
Returns general application info in JSon format.
-
io.close <Direction> <Connection>
Close connection(s).
-
<Direction>
direction of the connection
(available values: in, out, both)
-
<Connection>
-
connection id or glob pattern
-
io.doc <Direction> <Transport> <Format>
Transport, format and parameter info of channels in JSon.
-
<Direction>
direction of the connection or glob pattern
(available values: in, out)
-
<Transport>
transport name or glob pattern
(available values: socket, sgacard, mufi, file, std, pcap, dir, pipe, ws, http2, jbin, sgasocket)
-
<Format>
format name or glob pattern
(available values: adg, pcap, sga, hex, pstream, tapper, json, yaml, perldump, raw, gre, avro)
-
io.open <Direction> <Uri>
Opens a new input or output channel.
-
<Direction>
direction of the connection
(available values: in, out)
-
<Uri>
-
'transport-format://location?optional_parameter=value&optional_parameter=value
-
io.sorter <Direction> <Queue delay> <Queue length> <Queue timeout>
Sets message sorter for a connection.
-
<Direction>
direction of the connection
(available values: in, out, both)
-
<Queue delay>
-
sorting queue delay in milliseconds
-
<Queue length>
-
maximum size of the sorting queue
-
<Queue timeout>
-
sorting queue is flushed after inactivity (in clk cycles/1 ms)
-
process.start
-
Starts the processing of the incoming data.
-
process.stop
-
Stops the processing of the incoming data.
-
process.stopsync
-
Stops the processing of the incoming data if there is no incoming traffic.
-
state.clear
-
Clears the state of the application. The temporally stored messages will be dropped.
-
state.load <State file>
Loads application states from file.
-
<State file>
-
state file name
-
state.save <State file>
Stores application states from file.
-
<State file>
-
state file name
-
vars.clear
-
Clears the nonprotected application counters.
-
vars.get <Counter>
Returns one or more counters in json format.
Note: Glob pattern can be used to specify more parameters.
-
<Counter>
-
counter name or glob pattern
COUNTERS
List of counters:
-
io.all.input.bytes (unit: B)
-
Count of incoming bytes
-
io.all.input.msgs (unit: Msg)
-
Count of incoming messages
-
io.all.input.sorter
-
Current state of the incoming message sorter
-
io.all.input.time
-
Time of the last incoming message
-
io.all.output.bytes (unit: B)
-
Count of outgoing bytes
-
io.all.output.msgs (unit: Msg)
-
Count of outgoing messages
-
io.all.output.sorter
-
Current state of the outgoing message sorter
-
io.all.output.time
-
Time of the last outgoing message
-
io.state
-
Current state of processing
STATES
The application can load and save its states on demand (receiving corresponding commands).
There is no automatic state handling.
Note: The state file is a TAR format file pack.
SIGNALS
The application handles the following Signals from the OS:
-
SIGHUP
-
Restarts the TCP command channel.
-
SIGTERM, SIGINT, SIGQUIT
-
The application exits gracefully stopping the processing and closing the connection channels.
-
SIGABRT, SIGFPE, SIGILL, SIGSEGV
-
The application exits ungracefully.
CONNECTIONS
The application can handle multiple input/output channels. The channels
are composed of a transport method, a capture formatter used for packet
encapsulation and optional filters. There are multiple transports and
formats available, which can be used in different variations.
Channels are configured using URI format:
<transport>-<format>://<resource>?<parameters>
Examples:
Input ADG format over listen socket:
socket-adg://127.0.0.1:12345?listen=1&name=MyInput
Output ADG format over socket:
socket-adg://127.0.0.1:12345?name=MyOutput
-
Input channel parameters:
-
name
-
User defined name of the channel, will be generated if not supplied.
-
resetwait
-
Waiting time before reset in msec.
-
wait
-
Waiting time in msec.
-
buffer
-
Nonzero value specifies the buffered messages count.
-
critical
-
If the criticality of a channel is set to false, processing will not stop if the channel stops. (def: true)
-
Output channel parameters:
-
name
-
User defined name of the channel, will be generated if not supplied.
-
resetwait
-
Waiting time before reset in msec.
-
wait
-
Waiting time in msec.
-
buffer
-
Nonzero value specifies the buffered messages count.
-
critical
-
If the criticality of a channel is set to false, processing will not stop if the channel stops. (def: true)
-
Input transport: socket
Reads data from TCP socket.
-
Parameters:
-
listen
-
Creates listen socket if set.
-
type
-
The type of the socket, values: tcp, udp. The default is: tcp
-
bindaddr
-
The local binding IP address of the socket.
-
Input transport: sgacard
-
Reads data from SGA card.
-
Input transport: mufi
Reads multiple data files splitted by date/time using MuFi files as file pointers.
-
Parameters:
-
mufi
-
Mufi file path and name without extention.
-
verbose
-
Enables debug log on the transport.
-
cleanup
-
Deletes input data file after processing.
-
Input transport: file
Reads data from file.
-
Parameters:
-
loop
-
Number of repeated reading of the file.
-
from
-
The start position in the file.
-
Input transport: std
-
Reads data from standard input.
-
Input transport: pcap
-
Reads data from card using libPCap.
-
Input transport: dir
Reads data from multiple files in a directory.
-
Parameters:
-
mask
-
Glob matching pattern for files to use.
-
from
-
Filename of the first file to be used.
-
pos
-
Filename lst used by previous session.
-
loop
-
Number of repeated reading of the directory.
-
Input transport: pipe
Reads data from named pipe.
-
Parameters:
-
listen
-
Creates listen namedpipe if set.
-
Input transport: ws
Reads data from websocket.
-
Parameters:
-
sslcert
-
SSL Certificate File.
-
Input transport: http2
-
Reads data from HTTP2 stream over TCP socket.
-
Input transport: jbin
-
Generate raw binary samples based from JSON description.
-
Input transport: sgasocket
Reads data from TCP socket.
-
Parameters:
-
listen
-
Creates listen socket if set.
-
Output transport: null
-
Discards any data sent out.
-
Output transport: socket
Writes data to TCP socket.
-
Parameters:
-
listen
-
Creates listen socket if set.
-
type
-
The type of the socket, values: tcp, udp. The default is: tcp
-
bindaddr
-
The local binding IP address of the socket.
-
Output transport: file
-
Writes data to file.
-
Output transport: std
-
Writes data to standard output.
-
Output transport: dir
Writes data to the specified directory.
-
Parameters:
-
mask
-
Glob matching pattern for files to use.
-
from
-
Filename of the first file to be used.
-
pos
-
Filename lst used by previous session.
-
file
-
The static part of the output file name.
-
ext
-
Extention of the output files.
-
interval
-
Time interval for file splitting in seconds.
-
Output transport: pipe
Reads data from named pipe.
-
Parameters:
-
listen
-
Creates listen namedpipe if set.
-
Output transport: ws
Writes data to websocket.
-
Parameters:
-
sslcert
-
SSL Certificate File.
-
Output transport: sgasocket
Writes data to TCP socket.
-
Parameters:
-
listen
-
Creates listen socket if set.
-
Format: adg
Native format of the datagram.
-
Parameters:
-
compress
-
Activates the compression of the transmitted data.
-
Format: pcap
Payload and capture info (parts can be represented in) PCap file format.
-
Parameters:
-
fileheader
-
Data stream starts with a file header.
-
Format: sga
Payload and all additional info in SGA file format.
-
Parameters:
-
fileheader
-
Data stream starts with a file header.
-
storepos
-
Insert position of the message from input transport, ignore any position info already included.
-
Format: hex
-
Raw bytes of the datagram in text/hexdump.
-
Format: pstream
Payload and capture info (parts can be represented in) PacketStream/Tapper stream format.
-
Parameters:
-
version
-
Version of packet stream format (autodetected on input).
-
Format: tapper
-
Payload and capture info in Tapper stream format.
-
Format: json
Text serialization of the datagram.
-
Parameters:
-
indent
-
Use indentation if enabled.
-
Format: yaml
Text serialization of the datagram.
-
Parameters:
-
indent
-
Use indentation if enabled.
-
Format: perldump
Text serialization of the datagram.
-
Parameters:
-
indent
-
Use indentation if enabled.
-
Format: raw
Raw format wrapped to/extracted from payload content. Transport should be packet based.
-
Parameters:
-
linkName
-
If set, attaches capture info to input with supplied link name.
-
Format: gre
-
Payload and capture info (parts can be represented in) Dg/GRE stream format.
-
Format: avro
Avro schema based data formatter
-
Parameters:
-
binding
-
Full path of the avro binding descriptor json file
-
schema
-
Full path of the avro schema descriptor json file
COPYING
Copyright © 2020 Aitia International Inc.