Introduction
The ever-growing presence of Linux, which powers everything from server rooms, embedded devices and Internet of Things (IoT) endpoints, places increasing importance on the ability to manage complex distributed systems. However, managing heterogeneous fleets of systems with vastly different hardware and software is highly difficult. Existing management solutions exhibit several common limitations: they do not scale well across various device classes, they rely on connecting devices to centralized and costly server-side equipment, and often lock most of their functionality behind proprietary models and subscription fees.
mINK equips companies with easily extensible components for handling virtually any system aspect ranging from low-level resources to applications. mINK-enabled systems connect into a peer-to-peer architecture that improves the distributed system’s fault tolerance. At the same time, its plugin-based design enables companies to quickly and flexibly add custom features to the framework and create scalable and versatile management solutions.
Managing Linux-based systems
Linux powers a considerable portion of modern devices. The Linux kernel constitutes the backbone of today’s internet ecosystem, dominates the supercomputer world, and is at the core of the Android OS running on billions of smartphones. Linux is also universally present on embedded systems and the Internet of Things (IoT), powering everything from home energy and industrial automation to network devices, medical equipment, and many more.
The widespread growth in size and diversity of Linux-based systems is closely tied to enterprises’ need to connect and manage them efficiently. However, current solutions for managing Linux-based devices feature several common limitations:
Targeting only a specific class of devices
Some management solutions cater only to certain types of devices, such as resource constrained IoT devices or high-end server equipment. This state forces companies to rely on several management solutions to control different infrastructure segments or leave some components completely unmanaged. The latter is no longer an option for companies looking to take full advantage of their infrastructure and for maintaining and securing their competitive market advantage.
Centralized system architecture
Most management solutions are based on a client-server model, where a central server-side component monitors and manages clients. As the number of endpoints increases, the server needs to handle more intensive data demands and a larger number of client requests, which negatively impacts the server’s availability and performance. Furthermore, the client-server model relies on the server’s constant availability, which means that issues such as server software and hardware failures or network collapses can easily disrupt device management workflows.
Proprietary business models lock-ins
Many solutions for managing Linux-based systems lock all or most of the useful functionalities behind proprietary commercial licenses. Companies using these solutions cannot verify the code they use to monitor and manage their devices, nor can they customize or expand them to address their unique requirements.
mINK Framework
mINK framework is a highly versatile open source solution for controlling and managing Linux- based distributed systems. Inspired by the telecommunications industry, the framework originated as a high-performance SIGTRAN firewall for telecoms and other SS7 service providers. As such, the framework adapts and benefits from several established principles and technologies utilized primarily by high-performance signaling gateways.
System agent service
The system agent service (sysagentd) is responsible for handling the device on which the mINK node is instantiated. This service is tightly connected to the underlying GNU/Linux operating system over extensible modules called plugins. Plugins are selected during the compile time via the pre-build configuration process.
The service’s northbound interface establishes two-way communication with routingd over the GDT protocol. Messages can originate from other sysagentd services on the network or from system components that send and receive messages to and from the sysagentd service over protocol adapters.
The service’s direct communication with the underlying system is dependent on the system’s internals. For example, if the underlying system is a distribution with systemd and dbus support, mINK plugins will rely on the features that these two components provide. On the other hand, the plugins need to be expanded for distributions that use their own ecosystem of tools, such as OpenWrt. The vanilla mINK framework comes with comprehensive OpenWrt support, and developers can extend mINK to support other distributions’ ecosystems.
Architecture
By design, mINK nodes connect into a decentralized peer-to-peer (P2P) architecture. Compared to traditional client-server architectures where one or more clients request services from a specific server, every peer in a P2P network is an equally privileged element that acts as both the client and the server. Participants or peers interact directly without the need for any central decision-making source.
P2P networks can be further divided into unstructured or structured. In unstructured overlays, peers maintain an ad hoc list of neighbors, and there are no predetermined routes for how nodes communicate. In structured networks, each node is equipped with a list of neighboring nodes with which it can communicate.
Every mINK node is a peer of a structured P2P network that connects to other predefined peers. If a single mINK node experiences issues or stops functioning, other nodes continue operating normally. Moreover, a single device can run multiple mINK nodes to increase redundancy. Companies can arbitrarily connect mINK nodes and introduce new or remove existing nodes to address current requirements.
In addition to improving the system’s fault tolerance, mINK’s peer-to-peer architecture also enables significant code reduction and reusability. The framework frees developers and companies from developing and maintaining server-side software and allows them to distribute the same software to connect various devices and system components.
Design principles
The mINK framework is built from the ground up with flexibility in mind. To this end, the framework employs several core design principles of scalability, cross-platform, decoupling and openness.
Scalability
The sysagentd service provides the essential abstractions for interacting with GNU/Linux-based systems. Developers can then use these common abstractions to quickly create independent plugins. The sysagentd service also handles how these plugins are initialized and configured and how they communicate with other services in the node. By resolving complex communication and configuration challenges, this plugin-based approach significantly decreases the time necessary to develop new plugins and allows developers to focus on creating custom new functionalities.
In addition to a faster time-to-market for new services, this approach also provides extensibility and flexibility by enabling companies to quickly add, remove or change the selection of plugins to load.
Cross-platform
mINK nodes can run on any machine that runs Linux. This capability allows companies to run mINK on a large number of Linux-supported computer architectures, including ARM, MIPS, x86_64, i386 and many others. By extension, mINK nodes can run on a wide variety of hardware such as Customer Premises Equipment (CPEs), servers, workstations and specialized embedded devices.
Openness
The mINK framework is fully open source with code hosted on GitHub. Instead of locking features or functionality behind proprietary models, mINK empowers companies with complete visibility into the source code. The mINK framework aims to be truly GNU/Linux-agnostic.
Applicability
The mINK framework can benefit a wide variety of applications across diverse industry sectors. From building solutions for automotive, solar, water and wind industries to advancing transportation infrastructures and machine building, companies can utilize mINK in any domain requiring efficient communication between Linux-based systems.
Local system monitoring
In this use case, a single mINK node is deployed on a single machine. The mINK node’s services are responsible for setting and getting device data and routing this data between node components.To illustrate this better, we will consider two domains of local system monitoring: CPEs and telemetry.
CPE monitoring
A mINK node running on a GNU/Linux-powered CPE can perform various system-wide operations such as setting and getting CPE parameters, collect- ing data and making data-driven operations.
The sysagentd service acts as a system-wide manager whose main purpose is to set and get data from configuration files, run necessary scripts and prepare data for other components that interact with mINK over protocol adapters. One example of such a component is the local Web User Interface, which can communicate with sysagentd by sending JSON-RPC calls over the MQTT protocol plugin.
The mINK node can prepare the data collected in this fashion for other components, such as cloud-based controllers, agents running on different machines in the network, or for sysagentd services running on the same or other devices in the network. Once the data is prepared, mINK can send the data over the network via the GDT protocol or use protocol adapters to send data via other protocols.
Telemetry monitoring
In this use case, the sysagentd service can be ex- tended with one or more plugins for accessing or extracting information from specific hardware components. The plugins can utilize traditional GNU/Linux methods for these tasks, such as shared memory, driver exposure to proc filesystem or device files. The plugins can then pass the extracted data further up on the communication ladder.
The range of accessible hardware is dependent upon many factors, and extracting information is sometimes achieved through equipment such as Digital Signal Processors (DSPs), LCDs and other monitors, or specialized input devices. An example of mINK interfacing directly with the underlying hard- ware is made possible through a mINK plugin for the Bridgetek BT81X EVE4 module. The EVE HMI controller IC appears to the host microcontroller (MCU) as a memory-mapped SPI device. The mINK plugin can access this memory-mapped device and send commands to it to directly access the module. Figure 2 shows a simplified overview of the data flow in this specific use case.
The framework’s decentralized architecture allows sending these commands from any mINK-enabled node, which means that the module can be accessed remotely from anywhere on the network.
Distributed system management
Another mINK framework use case involves deploying and managing multiple mINK nodes over the network. Companies can cluster all or some nodes together to form a single entity for information exchange. One or more instances can also behave as data retention daemons (DRDs) that collect and process data from other nodes.
CPE management
CPE management in the context of mINK is an extension of the framework’s CPE monitoring capabilities. The distributed and decentralized nature of mINK allows two key management benefits:
The same nodes can be deployed on both the CPE and the cloud side.
This benefit abolishes the need to develop the entire cloud side from scratch since the only components requiring development are the sysagent plugins. Furthermore, the communication sequences between CPE and cloud-side mINK nodes are the same as between two locally deployed mINK nodes. This reusability allows companies to significantly speed up the development time for new features while also increasing the system’s robustness and fault tolerance. There is no hardcoded limit regarding the number of nodes that can communicate in this setup. Depending on the desired use case, companies can organize the network into clusters, a server-client-like model or a distributed P2P network topology.
The cloud-side sysagentd service can also be easily extended via plugins to address basic and complex requirements.
Companies can deploy different cloud components or data retention daemons with varying sets of features depending on their requirements. For example, a mINK node can contain plugins for collecting specific data sets from CPEs deployed in customers’ premises and sending this data to observability platforms, such as Grafana for further processing and visualization. Additionally, companies can extend mINK nodes to leverage supported expert systems and AI engines or add support for custom solutions.
Figure 3: Example of mINK CPE Management
Telemetry management
If we extend the telemetry monitoring to management, a mINK node can send collected data to other system components on the network, such as other hardware, applications and mINK nodes. The scenarios below highlight two domains where this capability applies:
IoT
The IoT domain relies on connecting different components and exchanging data over the network. Here, mINK can run on GNU/Linux powered devices that gather information from IoT stacks and relay this data to mINK-enabled cloud components and data retention daemons. Figure 4 shows an example of mINK’s applicability in the IoT domain.
Figure 4: mINK Telemetry management - IoT
Industries
Various industries utilize specialized equipment to increase operational efficiency and develop new revenue models. Examples of such use cases include regulators, which monitor complex systems, perform voltage regulation and control the behavior of different peripheral devices. Companies can adapt mINK to collect telemetry from on-site equipment or remotely from Linux-powered machines with Internet access. The mINK nodes can then send the data through to other mINK nodes for further processing.