Skip to main content
Version: v1.1

Network Manager

Introduction​

The Network Manager is a subcomponent of the Communication Manager (CM) responsible for creating, maintaining, and cleaning up network resources for service providers and instances. It allocates IP subnets and addresses from predefined private network pools, generates VLAN IDs for network isolation, operates a local DNS server for service name resolution, and prepares firewall rules based on service connection policies.

This module ensures that every service instance running on the Unit has a unique IP address, resolvable DNS hostnames, and appropriate network isolation — without requiring manual network configuration from the OEM integrator.

Architecture​

The Network Manager is implemented in the aos::cm::networkmanager namespace and exposes the NetworkManagerItf interface to other CM modules (primarily the launcher, which calls it during instance deployment).

Dependencies​

The Network Manager requires four external interfaces:

InterfacePurpose
StorageItfPersistent storage for network state — networks, hosts, and instance allocations survive restarts
RandomItfCryptographic randomness source for VLAN ID generation
NodeNetworkItfSends network configuration (IP, subnet, VLAN ID) to the SM on each Node, which creates the actual VLAN network interfaces
DNSServerItfManages the local DNS hosts file and restarts the DNS server process

Internal Subsystems​

SubsystemResponsibility
IpSubnetSubnet pool management — tracks available and allocated subnets, provides free IP addresses within each subnet
DNSServerDNS server operation — writes hosts file entries and signals the DNS process to reload
NetPoolNetwork pool generation — splits predefined base CIDR ranges into target-sized subnets

IP Subnet Allocation​

Network Pools​

The Network Manager draws subnets from a set of predefined private network ranges. These base CIDR blocks are split into smaller subnets of a target prefix length:

Base NetworkTarget PrefixResulting Subnets
172.17.0.0/16/161 subnet
172.18.0.0/16/161 subnet
172.19.0.0/16/161 subnet
172.20.0.0/14/164 subnets
172.24.0.0/14/164 subnets
172.28.0.0/14/164 subnets

This yields a total pool of 14 available /16 subnets. Each subnet provides approximately 65,533 usable host addresses (excluding network and broadcast addresses, plus the first address reserved for the gateway).

Allocation Process​

When a new network is needed (triggered by a provider update or instance deployment):

  1. Route overlap check: The allocator queries the system routing table and selects a subnet that does not overlap with any existing routes on the host
  2. Subnet reservation: The selected subnet is removed from the available pool and associated with the requesting network ID
  3. IP assignment: Individual IP addresses are allocated sequentially from the subnet's available address pool (starting from the third address — network + 2)

When a network is no longer needed (all instances removed), the subnet is released back to the available pool.

Instance Migration​

When a service instance migrates from one Node to another within the same network, the Network Manager preserves the instance's IP address and DNS server configuration. This ensures network continuity — other instances communicating with the migrated service do not need to update their connection targets.

VLAN Management​

Each provider network is assigned a unique VLAN ID for Layer 2 isolation between different service networks on the same physical infrastructure.

VLAN ID generation:

  • IDs are generated randomly in the range 0–4095 (standard 802.1Q range)
  • The generator retries up to 4 times if a collision with an existing VLAN ID is detected
  • Generated IDs are persisted in storage and included in the network parameters sent to each Node's SM

The actual VLAN interface creation is not performed by the Network Manager itself — it sends the VLAN ID along with subnet and IP information to the Node's SM via the NodeNetworkItf, and the SM is responsible for creating the VLAN network interface on the Node.

DNS Server Operation​

The Network Manager operates a local DNS server that provides name resolution for service instances. This allows services to communicate with each other using stable hostnames rather than IP addresses that may change across deployments.

Hostname Generation​

For each service instance, the Network Manager generates DNS hostnames based on the instance identity:

PatternExampleCondition
{instance}.{subjectID}.{itemID}0.subject1.service-aAlways generated
{instance}.{subjectID}.{itemID}.{networkID}0.subject1.service-a.net1Always generated
{subjectID}.{itemID}subject1.service-aOnly for instance 0 (primary)
{subjectID}.{itemID}.{networkID}subject1.service-a.net1Only for instance 0 (primary)

The short-form hostnames (without instance number) are only registered for instance 0, providing a stable "default" address for each service regardless of how many replicas are running.

Hosts File Management​

The DNS server uses a hosts file (addnhosts) for name resolution. The Network Manager:

  1. Accumulates hostname-to-IP mappings as instances are prepared
  2. Writes the complete hosts file when RestartDNSServer is called
  3. Signals the DNS server process with SIGHUP to reload the hosts file

The hosts file format follows the standard /etc/hosts convention: one IP address followed by tab-separated hostnames per line.

DNS Server Lifecycle​

The DNS server process is managed externally (not started by the Network Manager). The Network Manager interacts with it through:

  • A PID file to locate the running process
  • A hosts file to provide name resolution data
  • A SIGHUP signal to trigger configuration reload

The DNS server's IP address is provided to each service instance as part of its network parameters, so instances can configure their resolver to use the local DNS.

Firewall Rule Preparation​

The Network Manager prepares firewall rules based on the service's declared allowed connections and exposed ports. These rules control which service instances can communicate with each other across subnet boundaries.

Allowed Connections​

Services declare their allowed connections in the format: {itemID}/{port}/{protocol} (protocol defaults to tcp if omitted). For each allowed connection, the Network Manager:

  1. Locates the target service instance by item ID across all networks
  2. Checks if the target instance exposes the requested port/protocol combination
  3. If the target is in a different subnet, generates a firewall rule permitting the connection
  4. If the target is in the same subnet, no rule is needed (direct communication is allowed)

Firewall Rule Structure​

Each generated rule contains:

FieldDescription
Source IPIP of the requesting instance
Destination IPIP of the target instance
Destination PortPort on the target instance
ProtocolTransport protocol (tcp/udp)

These rules are returned as part of the InstanceNetworkParameters and distributed to the Node's SM, which applies them using the system's packet filtering infrastructure.

Exposed Ports​

Services declare exposed ports in the format: {port}/{protocol} (protocol defaults to tcp if omitted). Exposed ports are stored per instance and used during firewall rule evaluation to verify that a target instance actually accepts connections on the requested port.

Provider Network Lifecycle​

Provider networks represent the network infrastructure for a set of service providers on a given Node. The lifecycle is managed through UpdateProviderNetwork:

  1. Synchronization: The current set of providers is compared against existing networks
  2. Creation: For newly required networks, a subnet is allocated, an IP is assigned, a VLAN ID is generated, and the configuration is sent to the Node's SM
  3. Cleanup: Networks no longer needed (provider removed from the list) have their host entries cleaned up, instances removed, and if the network becomes empty, the subnet is released back to the pool
  4. Persistence: All network state changes are persisted to storage for recovery after restart

Initialization and Recovery​

During initialization, the Network Manager:

  1. Loads all persisted network state from storage (networks, hosts, instances)
  2. Rebuilds the in-memory network state map
  3. Marks all previously allocated subnets and IPs as used in the IpSubnet allocator (preventing double-allocation)
  4. Initializes the subnet pool from predefined private networks

This ensures that after a CM restart, all existing network allocations are preserved and the allocator correctly accounts for already-used addresses.