Namespace declaration for all of the Myra Canyon helper functions. More...

Namespaces
	L2
	Layer 2 (ethernet).

	L3_ARP
	Layer 3 is ARP.

	L3_IP
	Layer 3 is either IPv4 or IPv6.

	L3_IPv4
	Layer 3 is IPv4.

	L3_IPv6
	Layer 3 is IPv6.

	L4_ICMP
	Definitions and helper functions for when layer 4 is ICMP.

	L4_ICMPv6
	Definitions and helper functions for when layer 4 is ICMPv6.

	L4_TCP
	Definitions and helper functions for when layer 4 is TCP.

	L4_UDP
	Definitions and helper functions for when layer 4 is UDP.

	Session
	Manage session IDs. Used by the Myra Apache module to determine if a user has correctly logged in.

	Traffic
	Namespace to group everything related to network traffic.

Classes
class	Addr6Port
	Similar to AddrPort, but uses IPv6 addresses. More...

class	AddrPort
	This class exists mostly for the ability to easily sort addresses and ports numerically when creating hashes. More...

class	Config
	Class to manage all of configuration read from and written to JSON file. More...

class	DB
	Class DB is used to gain access to the database. More...

class	Flow
	The `Flow` class represents both sides of a network connection. More...

class	FlowExpectation
	The `FlowExpectation` class is used to insert hints that can link future or predicted flows with older existing flows. More...

class	FlowStats
	Statistics used by Myra::Flow. More...

class	FlowTuple
	A 5-tuple used to identify a network flow. More...

class	Pkt
	The Pkt class contains both the raw storage for all the bytes in the packet, as well as the information/analysis on the packet. More...

class	PktInfo
	The PktInfo class is meant to be easier to use than individually calling the L2 and L3 helper functions. More...

class	Server
	Class used to setup the entire Myra Canyon server framework. More...

Typedefs
typedef std::queue< Pkt >	Packet_Queue
	A queue (FIFO) of packets. More...

typedef std::vector< uint8_t >	VBytes
	Vector of bytes. (E.g., to use as a packet buffer.) More...

typedef std::vector< std::string >	VStr
	Vector of strings. More...

typedef std::set< std::string >	SStr
	Set of strings. More...

typedef std::map< std::string, std::string >	MStr
	Map of strings. More...

typedef Json::Value	JSON
	Easy to reference JSON object. More...

typedef uint64_t	FlowHash
	64-bit hash calculated by Myra::hash(). More...

typedef uint64_t	STAT
	Type used for statistics. More...

typedef std::set< FlowHash >	Hashes
	Set of hashes. More...

typedef int	FD
	Native file descriptor. More...

typedef boost::multiprecision::uint128_t	uint128_t
	Unsigned 128-bit integer. More...

typedef boost::multiprecision::int128_t	int128_t
	Signed 128-bit integer. More...

typedef boost::system::error_code	EC
	boost system error code. More...

typedef boost::asio::io_service	ASIO_IO_SERVICE
	boost::asio's link to the operating system's I/O services. More...

typedef boost::asio::posix::stream_descriptor	ASIO_STREAM
	boost::asio's POSIX stream descriptor. More...

typedef boost::asio::signal_set	ASIO_SIGNAL_SET
	boost::asio's signal handling. More...

typedef boost::asio::system_timer	ASIO_TIMER
	boost::asio's timer support. More...

typedef std::chrono::high_resolution_clock::time_point	TP
	High resolution time point (aka "now"). More...

typedef uint8_t	MAC [6]
	MAC addresses. More...

typedef uint32_t	IPv4_ADDR
	32-bit IPv4 addresses. More...

typedef uint128_t	IPv6_ADDR
	128-bit IPv6 addresses. More...

typedef uint16_t	PORT
	16-bit port. More...

typedef PORT	TCP_PORT
	16-bit TCP port. More...

typedef PORT	UDP_PORT
	16-bit UDP port. More...

typedef std::map< IPv4_ADDR, std::string >	HostLookupMap
	Map of address to hostname. More...

typedef std::map < Myra::FlowHash, Myra::Flow >	FlowMap
	A std::map of flows, where the key is the flow hash. More...

typedef bool(*	TrafficExamination )(Flow &flow, const Myra::PktInfo &info)
	A function that examines a packet as part of a flow. More...

typedef std::vector < TrafficExamination >	TrafficExams
	A vector of traffic examination functions. Used in Flow to call all relevant examination functions. More...

Functions
std::string	to_string (const MAC &mac)
	Format a MAC address for pretty-printing. More...

std::string	to_string (const IPv4_ADDR &addr)
	Format a IPv4 address for pretty-printing. More...

std::string	to_string (const IPv6_ADDR &addr)
	Format a IPv6 address for pretty-printing. More...

IPv4_ADDR	ipv4_from_string (const std::string &str)
	Convert a number-and-dots text string to a IPv4 address. More...

IPv6_ADDR	ipv6_from_string (const std::string &str)

Myra::FlowHash	hash (const Myra::PktInfo &info)
	Generate a 64-bit hash value. More...

void	initialize_logging (void)
	Initialize boost::log. More...

void	initialize_file_logging (const std::string &filename)
	Add the specified filename as a log sink. More...

void	reset_logging (void)
	Reset boost::log and disable logging. More...

void	log_backtrace (void)
	Log a line for every function name in the call stack. More...

std::size_t	number_of_processing_units (void)
	Get the number of processing units available. This returns the sum of CPUs, cores, hyperthreads, etc. More...

VStr	run_command_and_get_vector (const std::string &cmd)
	Run a command and get all of the output from STDOUT. More...

std::string	run_command_and_get_string (const std::string &cmd)
	Run a command and get all of the output from STDOUT. More...

std::string	read_text_file (const std::string &filename)
	Read an entire text file. More...

void	write_file (const std::string &filename, const std::string &text)
	Write a text file. This writes the file to a temporary location, then renames it to the requested name. More...

bool	is_root (void)
	Determine if we're running with root-level access. More...

std::string	approximate_time (const time_t tt)
	Build a text string that tells us approximately when a specific event took place. More...

std::string	approximate_time (const TP &when)
	Similar to the other approximate_time() function, but takes a TP reference. More...

std::string	cpp_demangle (const std::string &str)
	Demangle the given C++ name. This is compiler-specific. More...

FD	create_pid_lockfile (const std::string &name)
	Create a PID lock file. This creates and places an exclusive lock on a PID file in /var/run/. More...

void	release_pid_lockfile (FD &pid_fd)
	Release the PID lock file. This unlocks the file in /var/run/ created by create_pid_lockfile(). More...


std::string	hex_out (const std::string &str, const int number_of_rows_to_display=4)
	Produce hex output for the given data. More...

std::string	hex_out (const uint8_t *data, const size_t len, const int number_of_rows_to_display=4)

std::string	format_SI_or_IEC (const long double &bytes, std::string &postfix, const int divider)
	} More...

std::string	format_bytes_SI (const uint64_t &bytes)

std::string	format_bytes_IEC (const uint64_t &bytes)

std::string	format_bps (const uint64_t &bytes, std::size_t seconds)


std::string &	ltrim (std::string &str)
	Trim whitespace from string. More...

std::string &	rtrim (std::string &str)

std::string	ltrim (const std::string &str)

std::string	rtrim (const std::string &str)

std::string	trim (const std::string &str)
	Trim whitespace from string. More...

std::string &	trim (std::string &str)
	Trim whitespace from string. More...

Variables

constexpr std::size_t	KiB = 1024
	kibibyte (10 bits) `1024` More...

constexpr std::size_t	MiB = 1024 * KiB
	mebibyte (20 bits) `1048576` More...

constexpr std::size_t	GiB = 1024 * MiB
	gibibyte (30 bits) `1073741824` More...

constexpr std::size_t	TiB = 1024 * GiB
	tebibyte (40 bits) `1099511627776` More...

constexpr std::size_t	PiB = 1024 * TiB
	pebibyte (50 bits) `1125899906842624` More...

constexpr std::size_t	EiB = 1024 * PiB
	exbibyte (60 bits) `1152921505606846976` More...

constexpr std::size_t	ZiB = 1024 * EiB
	zebibyte (70 bits) `1180591620717411303424` More...

constexpr std::size_t	YiB = 1024 * ZiB
	yobibyte (80 bits) `1208925819614629174706176` More...


constexpr std::size_t	KB = 1000
	kilobyte `1000` More...

constexpr std::size_t	MB = 1000 * KB
	megabyte `1000000` More...

constexpr std::size_t	GB = 1000 * MB
	gigabyte `1000000000` More...

constexpr std::size_t	TB = 1000 * GB
	terabyte `1000000000000` More...

constexpr std::size_t	PB = 1000 * TB
	petabyte `1000000000000000` More...

constexpr std::size_t	EB = 1000 * PB
	exabyte `1000000000000000000` More...

constexpr std::size_t	ZB = 1000 * EB
	zetabyte `1000000000000000000000` More...

constexpr std::size_t	YB = 1000 * ZB
	yotabyte `1000000000000000000000000` More...


constexpr std::size_t	Seconds = 1

constexpr std::size_t	Minutes = 60 * Seconds

constexpr std::size_t	Hours = 60 * Minutes

constexpr std::size_t	Days = 24 * Hours

constexpr std::size_t	Weeks = 7 * Days

constexpr std::size_t	Years = 365.24 * Days

constexpr std::size_t	Months = Years / 12

Detailed Description

Namespace declaration for all of the Myra Canyon helper functions.

Typedef Documentation

typedef boost::asio::io_service Myra::ASIO_IO_SERVICE

boost::asio's link to the operating system's I/O services.

typedef boost::asio::signal_set Myra::ASIO_SIGNAL_SET

boost::asio's signal handling.

typedef boost::asio::posix::stream_descriptor Myra::ASIO_STREAM

boost::asio's POSIX stream descriptor.

typedef boost::asio::system_timer Myra::ASIO_TIMER

boost::asio's timer support.

typedef boost::system::error_code Myra::EC

boost system error code.

typedef int Myra::FD

Native file descriptor.

typedef uint64_t Myra::FlowHash

64-bit hash calculated by Myra::hash().

typedef std::map<Myra::FlowHash, Myra::Flow> Myra::FlowMap

A std::map of flows, where the key is the flow hash.

See also: Myra::Server::flow_map

typedef std::set<FlowHash> Myra::Hashes

Set of hashes.

typedef std::map<IPv4_ADDR, std::string> Myra::HostLookupMap

Map of address to hostname.

typedef boost::multiprecision::int128_t Myra::int128_t

Signed 128-bit integer.

typedef uint32_t Myra::IPv4_ADDR

32-bit IPv4 addresses.

typedef uint128_t Myra::IPv6_ADDR

128-bit IPv6 addresses.

typedef Json::Value Myra::JSON

Easy to reference JSON object.

typedef uint8_t Myra::MAC[6]

MAC addresses.

See also: L2::Header

typedef std::map<std::string, std::string> Myra::MStr

Map of strings.

typedef std::queue<Pkt> Myra::Packet_Queue

A queue (FIFO) of packets.

typedef uint16_t Myra::PORT

16-bit port.

typedef std::set<std::string> Myra::SStr

Set of strings.

typedef uint64_t Myra::STAT

Type used for statistics.

typedef PORT Myra::TCP_PORT

16-bit TCP port.

typedef std::chrono::high_resolution_clock::time_point Myra::TP

High resolution time point (aka "now").

typedef bool(* Myra::TrafficExamination)(Flow &flow, const Myra::PktInfo &info)

A function that examines a packet as part of a flow.

Returns: true if it needs to be called again with another packet; false if it should never be called again (wrong type of traffic)

typedef std::vector<TrafficExamination> Myra::TrafficExams

A vector of traffic examination functions. Used in Flow to call all relevant examination functions.

typedef PORT Myra::UDP_PORT

16-bit UDP port.

typedef boost::multiprecision::uint128_t Myra::uint128_t

Unsigned 128-bit integer.

typedef std::vector<uint8_t> Myra::VBytes

Vector of bytes. (E.g., to use as a packet buffer.)

typedef std::vector<std::string> Myra::VStr

Vector of strings.

Function Documentation

std::string Myra::approximate_time ( const time_t tt )

Build a text string that tells us approximately when a specific event took place.

Parameters

[in] tt absolute time since the unix epoch (1970-01-01 00:00:00)

Examples of some output this function produces:

23 seconds ago
4 hours ago
in 6 weeks

Here is the call graph for this function:

Here is the caller graph for this function:

std::string Myra::approximate_time ( const TP & when )

Similar to the other approximate_time() function, but takes a TP reference.

Here is the call graph for this function:

std::string Myra::cpp_demangle ( const std::string & str )

Demangle the given C++ name. This is compiler-specific.

Here is the caller graph for this function:

Myra::FD Myra::create_pid_lockfile ( const std::string & name )

Create a PID lock file. This creates and places an exclusive lock on a PID file in /var/run/.

Parameters

[in] name is the name to use when creating the PID lock file (usually the process name)

Returns: the file descriptor for the open PID lock file in /var/run/

See also: Myra::release_pid_lockfile()

Here is the call graph for this function:

Here is the caller graph for this function:

std::string Myra::format_bps	(	const uint64_t &	bytes,
		std::size_t	seconds
	)

Parameters

[in]	bytes	total number of bytes transferred
[in]	seconds	the length of time (in seconds) of the period over which the transferred was measured

Network speed always uses decimal (SI) formatting.

For example:

Value	Seconds	Output
1000	1	`1.00 KBps [8.00 Kbps]`
12345	10	`1.23 KBps [9.88 Kbps]`
987654321	60	`16.46 MBps [131.69 Mbps]`

Here is the call graph for this function:

std::string Myra::format_bytes_IEC ( const uint64_t & bytes )

For example:

Input	Output
1000	`0.98 KiB`
1024	`1.00 KiB`

Here is the call graph for this function:

Here is the caller graph for this function:

std::string Myra::format_bytes_SI ( const uint64_t & bytes )

For example:

Input	Output
1000	`1.00 KB`
1024	`1.02 KB`

Here is the call graph for this function:

std::string Myra::format_SI_or_IEC	(	const long double &	bytes,
		std::string &	postfix,
		const int	divider
	)

}

Format the value similar to KBytes or KBytes/sec.

Here is the caller graph for this function:

Myra::FlowHash Myra::hash ( const Myra::PktInfo & info )

Generate a 64-bit hash value.

Not all packets will generate a hash. Only packets we expect to "follow" will generate a hash. For example, a one-shot ICMP message wont hash, while a TCP or UDP packet will hash.

When a packet cannot hash, a value of zero will be returned.

Here is what we use to generate a network traffic flow:

   addr1:           32 bits
   addr2:           32 bits
   port1:           16 bits
   port2:           16 bits
   ethertype:        8 bits
   protocol:         8 bits
                   --------
   TOTAL:          112 bits
   UNUSED BITS:     16 bits

And even then, the ethertype and protocol fields don't provide that many bits of variability, but we're going to ignore that, or at least reduce the impact by spreading out the fields so the ones with the most variability (port1 and port2) are at opposite ends of the value. The 16 remaining empty/unused bits will be filled with a 2nd copy of port1, which is more helpful than having 16 bits always set to zero.

The 128-bit value to hash is composed like this:

    bits 0-15       [16 bits]:  port 1
    bits 16-23      [ 8 bits]:  protocol
    bits 24-55      [32 bits]:  addr 1
    bits 56-63      [ 8 bits]:  ethertype
    bits 64-95      [32 bits]:  addr 2
    bits 96-111     [16 bits]:  port 2
    bits 112-127    [16 bits]:  port 1 (again)

The actual algorithm which takes the 128-bit value and produces a 64-bit hash comes from the Hash128to64() function in the MIT-licensed FarmHash project, which in turn was Murmur-inspired.

See also: https://code.google.com/p/farmhash/source/browse/trunk/src/farmhash.h

Here is the call graph for this function:

Here is the caller graph for this function:

std::string Myra::hex_out	(	const std::string &	str,
		const int	number_of_rows_to_display = `4`
	)

Produce hex output for the given data.

Here is the caller graph for this function:

std::string Myra::hex_out	(	const uint8_t *	data,
		const size_t	len,
		const int	number_of_rows_to_display = `4`
	)

If number_of_rows_to_display is set to 1 or less, then all rows will be shown without skipping. Otherwise, the middle section of large buffers will be skipped to reduce the number of lines logged, keeping both the start and end of the buffer intact.

void Myra::initialize_file_logging ( const std::string & filename )

Add the specified filename as a log sink.

Here is the caller graph for this function:

void Myra::initialize_logging ( void )

Initialize boost::log.

Here is the caller graph for this function:

Myra::IPv4_ADDR Myra::ipv4_from_string ( const std::string & str )

Convert a number-and-dots text string to a IPv4 address.

Here is the caller graph for this function:

Myra::IPv6_ADDR Myra::ipv6_from_string ( const std::string & str )

bool Myra::is_root ( void )

Determine if we're running with root-level access.

Returns: false if the current user isn't root; true if the user id or effective user id is zero (root)

Here is the caller graph for this function:

void Myra::log_backtrace ( void )

Log a line for every function name in the call stack.

Here is the call graph for this function:

std::string & Myra::ltrim ( std::string & str )

Trim whitespace from string.

This modifies the original string.

See also: http://stackoverflow.com/questions/216823/whats-the-best-way-to-trim-stdstring

Here is the caller graph for this function:

std::string Myra::ltrim ( const std::string & str )

A new string is created, trimmed, and returned. The original string remains untouched.

Here is the call graph for this function:

std::size_t Myra::number_of_processing_units ( void )

Get the number of processing units available. This returns the sum of CPUs, cores, hyperthreads, etc.

See also: nproc(1) for an example

Returns: the number of processing units, or the minimal value of "1" if it cannot be determined

std::string Myra::read_text_file ( const std::string & filename )

Read an entire text file.

Note: This method is meant to read small text files, such as the JSON configuration file. It cannot be used for binary files since std::getline() may alter the "\n" at the end of some lines.

Here is the caller graph for this function:

void Myra::release_pid_lockfile ( FD & pid_fd )

Release the PID lock file. This unlocks the file in /var/run/ created by create_pid_lockfile().

Parameters

[in,out] pid_fd is the open PID file descriptor previously obtained from Myra::create_pid_lockfile(). It is automatically cleared (set to -1) once the PID lock file has been unlocked, closed, and deleted.

Here is the call graph for this function:

Here is the caller graph for this function:

void Myra::reset_logging ( void )

Reset boost::log and disable logging.

Here is the caller graph for this function:

std::string & Myra::rtrim ( std::string & str )

This modifies the original string.

See also: http://stackoverflow.com/questions/216823/whats-the-best-way-to-trim-stdstring

Here is the caller graph for this function:

std::string Myra::rtrim ( const std::string & str )

A new string is created, trimmed, and returned. The original string remains untouched.

Here is the call graph for this function:

std::string Myra::run_command_and_get_string ( const std::string & cmd )

Run a command and get all of the output from STDOUT.

See also: Myra::run_command_get_vector()

Here is the call graph for this function:

Myra::VStr Myra::run_command_and_get_vector ( const std::string & cmd )

Run a command and get all of the output from STDOUT.

See also: Myra::run_command_get_string()

Here is the caller graph for this function:

std::string Myra::to_string ( const MAC & mac )

Format a MAC address for pretty-printing.

Here is the caller graph for this function:

std::string Myra::to_string ( const IPv4_ADDR & addr )

Format a IPv4 address for pretty-printing.

std::string Myra::to_string ( const IPv6_ADDR & addr )

Format a IPv6 address for pretty-printing.

std::string Myra::trim ( const std::string & str )

inline

Trim whitespace from string.

This modifies the original string.

See also: http://stackoverflow.com/questions/216823/whats-the-best-way-to-trim-stdstring

Here is the call graph for this function:

Here is the caller graph for this function:

std::string& Myra::trim ( std::string & str )

inline

Trim whitespace from string.

This modifies the original string.

See also: http://stackoverflow.com/questions/216823/whats-the-best-way-to-trim-stdstring

Here is the call graph for this function:

void Myra::write_file	(	const std::string &	filename,
		const std::string &	text
	)

Write a text file. This writes the file to a temporary location, then renames it to the requested name.

Here is the call graph for this function:

Here is the caller graph for this function:

Variable Documentation

constexpr std::size_t Myra::Days = 24 * Hours

constexpr std::size_t Myra::EB = 1000 * PB

exabyte 1000000000000000000

constexpr std::size_t Myra::EiB = 1024 * PiB

exbibyte (60 bits) 1152921505606846976

constexpr std::size_t Myra::GB = 1000 * MB

gigabyte 1000000000

constexpr std::size_t Myra::GiB = 1024 * MiB

gibibyte (30 bits) 1073741824

constexpr std::size_t Myra::Hours = 60 * Minutes

constexpr std::size_t Myra::KB = 1000

kilobyte 1000

constexpr std::size_t Myra::KiB = 1024

kibibyte (10 bits) 1024

constexpr std::size_t Myra::MB = 1000 * KB

megabyte 1000000

constexpr std::size_t Myra::MiB = 1024 * KiB

mebibyte (20 bits) 1048576

constexpr std::size_t Myra::Minutes = 60 * Seconds

constexpr std::size_t Myra::Months = Years / 12

constexpr std::size_t Myra::PB = 1000 * TB

petabyte 1000000000000000

constexpr std::size_t Myra::PiB = 1024 * TiB

pebibyte (50 bits) 1125899906842624

constexpr std::size_t Myra::Seconds = 1

constexpr std::size_t Myra::TB = 1000 * GB

terabyte 1000000000000

constexpr std::size_t Myra::TiB = 1024 * GiB

tebibyte (40 bits) 1099511627776

constexpr std::size_t Myra::Weeks = 7 * Days

constexpr std::size_t Myra::YB = 1000 * ZB

yotabyte 1000000000000000000000000

Note: This doesn't fit into a 64-bit variable.

constexpr std::size_t Myra::Years = 365.24 * Days

constexpr std::size_t Myra::YiB = 1024 * ZiB

yobibyte (80 bits) 1208925819614629174706176

Note: This doesn't fit into a 64-bit variable.

constexpr std::size_t Myra::ZB = 1000 * EB

zetabyte 1000000000000000000000

Note: This doesn't fit into a 64-bit variable.

constexpr std::size_t Myra::ZiB = 1024 * EiB

zebibyte (70 bits) 1180591620717411303424

Note: This doesn't fit into a 64-bit variable.

Namespaces

Classes

Typedefs

Functions

Variables

Detailed Description

Typedef Documentation

Function Documentation

Variable Documentation