Build a Container Network by Hand with ip netns

Every container you have ever started thinks it owns its own network card, its own routing table and its own firewall. It does not. It is wearing a Linux network namespace like a costume, and you can build the whole trick by hand in about ten commands. This guide builds a container-style network from scratch with network namespaces, veth pairs, a bridge and nftables NAT, so you can see exactly what Docker automates for you.

What a network namespace actually is

The network_namespaces(7) man page describes it plainly: a namespace isolates network devices, the IPv4 and IPv6 protocol stacks, IP routing tables, firewall rules and sockets. Newer versions of the page also list the UNIX domain abstract socket namespace. A process inside a namespace sees only the interfaces that were placed there, and nothing else. Using them needs a kernel built with CONFIG_NET_NS, which mainstream distro kernels have.

Isolation alone is useless, because a namespace with no connection to anything is a sealed box. The connector is the veth pair. The veth(4) man page calls it a pipe-like abstraction: whatever goes in one end comes out of the other, and the two ends can live in different namespaces. That is the entire plumbing story for containers.

Step 1: two namespaces and a wire between them

Start with the smallest possible network: two namespaces joined by one veth pair. Run everything as root. The ip netns add command creates a named namespace; per ip-netns(8), a named namespace is an object under /run/netns (older docs say /var/run/netns) that keeps the namespace alive while it exists.

ip netns add red
ip netns add blue

# Create the pipe and push one end into each namespace
ip link add veth-red type veth peer name veth-blue
ip link set veth-red netns red
ip link set veth-blue netns blue

# Address and raise both ends (loopback starts down in a new namespace)
ip -n red addr add 10.10.0.1/24 dev veth-red
ip -n blue addr add 10.10.0.2/24 dev veth-blue
ip -n red link set lo up
ip -n red link set veth-red up
ip -n blue link set lo up
ip -n blue link set veth-blue up

# Talk across the wire
ip netns exec red ping -c 2 10.10.0.2

Two replies means you built a network. Check your host with ip link: veth-red and veth-blue are gone from it, because moving an interface into a namespace removes it from the namespace you left. Run ip netns exec red ip route and you will see only the one connected route. Nothing leaks in from the host.

Step 2: a bridge for more than two

A single veth pair is point to point. To connect many namespaces you need a switch, and Linux has one built in: the bridge. The pattern is one veth pair per container, with one end inside the container and the other end plugged into a bridge on the host. This is the same shape Docker's default bridge network uses, with a bridge on the host and one veth end per container.

ip netns del red; ip netns del blue   # start clean (this also removes the veths)

ip link add br0 type bridge
ip addr add 10.20.0.1/24 dev br0
ip link set br0 up

for n in red blue; do
  ip netns add $n
  ip link add veth-$n type veth peer name eth0-$n
  ip link set eth0-$n netns $n
  ip -n $n link set eth0-$n name eth0      # rename inside the namespace
  ip link set veth-$n master br0           # plug host end into the bridge
  ip link set veth-$n up
  ip -n $n link set lo up
  ip -n $n link set eth0 up
done

ip -n red  addr add 10.20.0.11/24 dev eth0
ip -n blue addr add 10.20.0.12/24 dev eth0

# Default route: send everything via the bridge address on the host
ip -n red  route add default via 10.20.0.1
ip -n blue route add default via 10.20.0.1

ip netns exec red ping -c 2 10.20.0.12   # namespace to namespace
ip netns exec red ping -c 2 10.20.0.1    # namespace to host

Step 3: give the namespaces internet with NAT

Right now red can reach the host but nowhere beyond it. Two things are missing. The host must be willing to forward packets between interfaces, and packets leaving to the internet must have their source rewritten, because 10.20.0.11 is a private address that no upstream router will route back to you. That rewriting is masquerade, source NAT that uses whatever address the outgoing interface currently has.

sysctl -w net.ipv4.ip_forward=1

nft add table ip nat
nft add chain ip nat postrouting '{ type nat hook postrouting priority 100 ; }'
nft add rule ip nat postrouting ip saddr 10.20.0.0/24 oifname != "br0" masquerade

# Test from inside the namespace (replace eth0 mentions if your uplink differs)
ip netns exec red ping -c 2 1.1.1.1

The nftables wiki explains that NAT lives in a chain of type nat, and that adding a NAT rule to a filter-type chain is an error. Postrouting at priority 100 is the conventional spot for source NAT, and the oifname test stops us from rewriting traffic that never leaves the bridge. Only the first packet of a flow is evaluated by the NAT chain; the kernel remembers the binding for the rest of the connection.

DNS inside a namespace

Ping by IP works but ping example.com fails? A namespace shares your filesystem, so it reads the host /etc/resolv.conf, and on systemd-resolved hosts that points at a stub address like 127.0.0.53 which only exists on the host loopback. The ip-netns(8) man page documents the fix: applications that are namespace-aware check /etc/netns/NAME/ first, and ip netns exec creates a mount namespace and bind mounts the files from /etc/netns/NAME over their usual locations in /etc.

mkdir -p /etc/netns/red
echo "nameserver 1.1.1.1" > /etc/netns/red/resolv.conf

ip netns exec red getent hosts example.com

Debugging and cleanup

• ip netns list shows what exists; ip netns exec NAME ip addr and ip route show the view from inside.
• ip netns exec NAME bash gives you a shell in the namespace. Your prompt will not change, so check with ip link.
• Watch the bridge with bridge link and bridge fdb show br br0 to see which MACs it learned and on which port.
• Capture on the host side of a veth with tcpdump -ni veth-red to see exactly what the container sends.
• ip netns del red removes the name; the namespace itself is freed once no process still holds it, and its veth devices go with it.
• Forwarding and nft rules persist until reboot. Delete the table with nft delete table ip nat and remove br0 with ip link del br0 when finished.

Takeaway

A container network is four ordinary Linux pieces: a namespace for isolation, a veth pair for the wire, a bridge for the switch and NAT for the exit door. Once you have typed them yourself, docker network and Kubernetes CNI plugins stop being magic and become tools you can debug with ip, bridge and tcpdump. It pairs nicely with the bufferbloat guide, since you can now put a cake qdisc on a veth and measure latency under load in a sandbox.