The network dataplane
Every packet between your workloads, and between them and the internet, is judged by Velrix’s dataplane: small eBPF programs in each server’s kernel, attached where the packet enters. There is no connection-tracking table and no nftables underneath: the programs are the whole packet path for project networks, security groups, floating IPs, gateways and load balancers.
You don’t configure the dataplane itself. Networks, security groups, floating IPs and balancers are what people make (Networking); the dataplane carries them out on every server, the same way on every server.
What it does
| Function | Where it runs | What you get |
|---|---|---|
| Anti-spoofing | each workload’s network interface, as its packets leave it | A workload can only send from its own addresses. Nothing else on its network can pretend to be it. |
| Security groups | tc, where a packet enters a network: from a workload, from another server, or from the server itself | Every packet is judged once, on the server it enters. Rules are lookups, so a hundred rules cost what one does. |
| Replies | a reply cache per network | An answer to a connection a rule allowed comes back without a rule of its own. Each network has its own cache, so one tenant’s traffic can never push out another’s. |
| Floating IPs | the gateway, statelessly | One-to-one address translation with no per-connection state, so connections survive the gateway moving to another server. |
| Outbound NAT | the gateway, per tenant | Workloads without a floating IP reach the internet from the gateway’s address. Each tenant has its own bounded share of ports, so one tenant can’t use them up for everyone. |
| L4 load balancers | XDP on the gateway’s network card | Each balancer has its own Maglev table: a connection keeps going to the same backend, also across a gateway failover, and adding or removing a backend moves as few connections as possible. |
| SYN-flood defence | XDP, before the server’s own network stack | A flood of connection attempts against a floating IP or a balancer is dropped at the card. Real clients still connect. |
Each tenant’s use of the shared tables is bounded and counted, so a busy or hostile tenant reaches its own limit, never the server’s.
Changes never half-apply
When someone changes a rule, a group or a balancer, the network module computes the new tables and each server loads them beside the ones in use, checks them whole, then switches over in one step. A packet sees the old rules or the new ones, never a mix. A set of tables that doesn’t check out is refused, the server keeps the rules it has, and operators are told.
The programs stay attached in the kernel while the platform restarts or updates, so traffic keeps being filtered throughout. A network interface gets its programs before it carries a single packet, and a packet no program judged is dropped. The dataplane fails closed.
What a server needs
- Velrix OS. Its kernel has everything the dataplane needs. On another Linux host, the kernel must be 6.6 or newer with BPF, the BPF JIT, tcx and BTF. A server that can’t load the dataplane carries no project networks; Manager › Hardware names what is missing. There is no fallback to an older packet filter.
- Physical network cards run the XDP programs in their driver (native XDP). Most server cards from the last ten years support it: Intel (ixgbe, i40e, ice), Mellanox/NVIDIA (mlx5), Broadcom (bnxt) and others.
- Virtual servers (a VPS, or Velrix in a VM) run XDP in generic mode. It works on any virtual card and keeps the host’s receive offloads, which native XDP on virtio would turn off. It is slower per packet than native XDP, which matters only at rates a VPS rarely sees.
- Few receive queues. A card with fewer receive queues than the server has CPUs would process every packet on one or a few CPUs. Velrix OS turns on RPS and RFS on such cards, so receive work is spread over the CPUs and lands on the one the receiving workload runs on. Nothing to set.
Reading the counters
Each network’s page in Networks shows its dataplane counters, per server:
- packets passed, and packets dropped by reason (no rule allowed it, spoofed source, flood, a fragment that came before its first part);
- the reply cache: entries in use, and entries pushed out before their time, which means the cache is too small for that network;
- for a gateway: outbound NAT ports in use per tenant, against the tenant’s bound, and each balancer’s backends with their share of connections.
Operators see every network; a project’s members see their own. Drops by rule also show in the network’s flow logs when flow logs are on (Logs).
A rising count of entries pushed out, or of ports at a tenant’s bound, is the signal to raise that limit.
Limits, and where to change them
The sizes and timings are dataplane policies in Policies, under the network group, changed at runtime:
| Policy | Default |
|---|---|
| How long an idle reply stays allowed | TCP 5 days, UDP 120 seconds, ICMP 30 seconds |
| Reply cache size, per network | set per realm |
| Outbound NAT ports per tenant, per gateway | set per realm |
- ICMP errors (unreachable, packet too big, time exceeded) for a connection that was allowed come back like its replies, so path MTU discovery works.
- Fragments: a fragment takes the verdict of its first part. A fragment that arrives before its first part is dropped and counted. A network has one MTU, so fragments are rare.
- A balancer’s tables and a security group’s rules have upper sizes per server. A change that would pass them is refused whole, and the server keeps the rules it has.