Networking troubleshooting

Each entry names a symptom, its cause and what to do. Message ids link to the entry in the error codes.

No address for a server

Cause. The pool has no free IPv4 address or IPv6 block left (error.ip_pool.exhausted). Addresses that are allocated, reserved or in quarantine do not count as free. The server can also have no network adapter on the network of the pool (error.ip_pool.no_nic), or the adapter has a primary address in this pool already (error.ip_pool.primary_exists).

Remedy.

1. Read the free addresses of the pool: kyvernon ip-pools available <pool>.
2. Read what the server has: kyvernon addresses list <vm>.
3. Release addresses that are no longer used, or extend the ranges of the pool: kyvernon ip-pools update <pool> --range <first>-<last>.
4. For a server without a suitable adapter, add one on the network of the pool in the hardware settings of the virtual machine.

Address in use or in quarantine

Cause. Another caller took the address between listing and assigning (error.ip_pool.address_in_use), or the address was released recently and stays in quarantine for 14 days unless the pool sets another period (error.ip_pool.address_quarantined).

Remedy. List the free addresses again with kyvernon ip-pools available <pool> and choose another one, or assign without naming an address to take the first free one. kyvernon ip-pools allocations <pool> shows when a quarantine ends.

Network adapter without backing

Cause. A network adapter is connected to a network that has no host-bridge backing, so it cannot be plugged into a bridge when the virtual machine starts (error.nic_network_unwired). The console shows such a network with the backing Not wired.

Remedy.

1. Give the network a backing: kyvernon network update <id> --body-file network.json, with a backing that names the bridge.
2. Or connect the adapter to a network that has a backing.
3. Power the virtual machine on again.

Proxy mode refused

Cause. The uplink mode proxy needs an Open vSwitch bridge and the name of the uplink interface. The interface is missing, has the name of the bridge, or the network is a quarantine network or a Linux bridge (error.network.proxy_uplink).

Remedy. Set "uplink": {"mode": "proxy", "interface": "eno1"} on an Open vSwitch network, with the real name of the host interface, or use the mode bridged.

Public address does not answer

Cause. The address is allocated, but traffic does not arrive. Common reasons: the bridge or the VLAN on the host does not match the network, the provider does not accept the MAC address of the guest in bridged mode, or the guest has not configured the address. Only the primary IPv4 address arrives by DHCP; extra addresses and the IPv6 block are configured in the guest.

Remedy.

1. Check the bridge, the VLAN and the MTU on the host: kyvernon host network-health <host>. A status other than ok names what differs.
2. Read the addresses of the server: kyvernon addresses list <vm>, and compare them with the configuration inside the guest.
3. When the provider filters MAC addresses, change the uplink to the mode proxy as described in IP pools.
4. A network adapter that sends from an address it was not given is dropped by port security; allocate the address to the adapter first.

Network cannot be deleted

Cause. A virtual machine still has a network adapter on the network. The request fails with a precondition error and lists the virtual machines.

Remedy. Connect those adapters to another network or remove them, then delete the network again: kyvernon network delete <id>.

IP pool cannot be deleted or changed

Cause. The pool still has allocations (error.ip_pool.in_use), a range overlaps the range of another pool on the same network (error.ip_pool.overlap), or the IPv6 prefix and block length cannot change while blocks are allocated or in quarantine (error.ip_pool.ipv6_layout_in_use).

Remedy. Release the addresses with kyvernon addresses release <vm> or kyvernon ip-pools free <pool> <address>, wait for the quarantine to end, then change the pool. Compare the ranges of both pools with kyvernon ip-pools get <pool>.

Reverse DNS stays requested

Cause. In manual mode a name is stored as requested until an operator confirms it. In automatic mode the DNS provider can refuse it: no reverse zone holds the address (error.rdns.no_zone), the name does not resolve back to the address (error.rdns.not_forward_confirmed), or the provider did not accept the change (error.rdns.provider_failed).

Remedy. Apply the name at the provider, then run kyvernon ip-pools ptr-applied <pool> <address>; or reject it with a reason: kyvernon ip-pools ptr-rejected <pool> <address> --reason <text>. For the other causes, create the forward record of the name first, or repeat the request later. See Reverse DNS.

Outgoing mail blocked

Cause. The pool blocks outgoing TCP port 25: IPv4 is allowed and IPv6 is blocked by default, and a server can be unblocked individually.

Remedy. An operator reads and changes the state: kyvernon addresses smtp <vm>, and kyvernon addresses smtp <vm> --unblock --reason <text>. The change is recorded as an event.

Adapter on a distributed port group has no link

Cause. OVN logical networks are not switched on (no [ovn] in controller.toml), the controller has lost its connection to the OVN Northbound database, or the host has not bound the logical port: ovn-controller on the host is not connected to the Southbound database, or a host precondition does not hold.

Remedy.

1. Read the realization with GET /v1/network/realization. connected must be true and realizedGeneration equal to modelGeneration; under hosts, the host of the virtual machine must report applied: true and count the port in portsInstalled. preconditions names what the host lacks.
2. On the host, run sudo bash deploy/server1/ovn-doctor.sh host; on the controller host, sudo bash deploy/server1/ovn-doctor.sh central. Every line marked FAIL names the step to repeat.
3. When the event NetworkNorthboundLostEvent is the newest network event, start the Northbound database again; the controller reconnects and writes every segment and port again.

Related

• Host-bridge networks
• Public addresses
• Traffic for a server that is throttled