--- name: af-xdp description: AF_XDP skill for high-performance XDP sockets. Use when creating AF_XDP sockets, configuring UMEM and XSK rings, XDP_REDIRECT programs, copy vs zero-copy mode, or comparing with DPDK. Activates on queries about AF_XDP, xsk_umem, XDP_REDIRECT, libbpf xsk, or zero-copy XDP. --- # AF_XDP ## Purpose Guide agents through AF_XDP sockets for high-performance packet I/O: socket creation, UMEM setup, fill/completion/RX/TX rings, XDP programs with `XDP_REDIRECT` to XSK, copy vs zero-copy modes, libbpf helpers, performance comparison with DPDK, and production use cases. ## When to Use - Building a userspace packet processor with lower overhead than raw sockets - Redirecting XDP-filtered traffic to userspace without DPDK complexity - Implementing a custom load balancer or IDS dataplane - Comparing zero-copy vs copy mode on your NIC/driver - Integrating with existing libbpf/XDP infrastructure - Need kernel cooperation (firewall rules) plus userspace processing ## Workflow ### 1. Architecture overview ``` NIC → XDP program (BPF) → XDP_REDIRECT → AF_XDP socket → userspace ↓ XDP_DROP/PASS/TX ``` Components: - **UMEM** — shared memory region for frames - **Fill ring** — userspace provides empty frame addresses to kernel - **Completion ring** — kernel returns completed TX frames - **RX ring** — kernel delivers received packets - **TX ring** — userspace submits packets for transmission ### 2. UMEM and XSK socket creation ```c #include #include #include #define NUM_FRAMES 4096 #define FRAME_SIZE XSK_UMEM__DEFAULT_FRAME_SIZE #define RX_BATCH_SIZE 64 struct xsk_umem_info { struct xsk_ring_prod fill; struct xsk_ring_cons comp; struct xsk_umem *umem; void *buffer; }; struct xsk_socket_info { struct xsk_ring_cons rx; struct xsk_ring_prod tx; struct xsk_socket *xsk; }; int xsk_setup(struct xsk_umem_info *umem_info, struct xsk_socket_info *xsk_info, int ifindex, int queue_id, int xsk_flags) { umem_info->buffer = mmap(NULL, NUM_FRAMES * FRAME_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); struct xsk_umem_config umem_cfg = { .fill_size = XSK_RING_PROD__DEFAULT_NUM_DESCS, .comp_size = XSK_RING_CONS__DEFAULT_NUM_DESCS, .frame_size = FRAME_SIZE, .frame_headroom = XSK_UMEM__DEFAULT_FRAME_HEADROOM, .flags = 0, }; int ret = xsk_umem__create(&umem_info->umem, umem_info->buffer, NUM_FRAMES * FRAME_SIZE, &umem_info->fill, &umem_info->comp, &umem_cfg); if (ret) return ret; struct xsk_socket_config xsk_cfg = { .rx_size = XSK_RING_CONS__DEFAULT_NUM_DESCS, .tx_size = XSK_RING_PROD__DEFAULT_NUM_DESCS, .libbpf_flags = XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD, .xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST, .bind_flags = xsk_flags, // XDP_ZEROCOPY or XDP_COPY }; return xsk_socket__create(&xsk_info->xsk, "eth0", queue_id, umem_info->umem, &xsk_info->rx, &xsk_info->tx, &xsk_cfg); } ``` ### 3. Populate fill ring ```c void populate_fill_ring(struct xsk_umem_info *umem) { uint32_t idx; uint32_t ret = xsk_ring_prod__reserve(&umem->fill, RX_BATCH_SIZE, &idx); for (uint32_t i = 0; i < ret; i++) *xsk_ring_prod__fill_addr(&umem->fill, idx + i) = i * FRAME_SIZE; xsk_ring_prod__submit(&umem->fill, ret); } ``` Must keep fill ring stocked — kernel drops packets if no buffers available. ### 4. XDP redirect program ```c // xdp_redirect.c #include #include struct { __uint(type, BPF_MAP_TYPE_XSKMAP); __uint(max_entries, 64); __type(key, int); __type(value, int); } xsks_map SEC(".maps"); SEC("xdp") int xdp_redirect_prog(struct xdp_md *ctx) { int index = ctx->rx_queue_index; return bpf_redirect_map(&xsks_map, index, 0); } ``` ```bash # Load and attach bpftool prog load xdp_redirect.o /sys/fs/bpf/xdp_redirect ip link set dev eth0 xdp obj xdp_redirect.o sec xdp # Pin xsks_map and update with socket fd ``` ### 5. RX processing loop ```c while (running) { uint32_t idx_rx = 0, rcvd; rcvd = xsk_ring_cons__peek(&xsk_info->rx, RX_BATCH_SIZE, &idx_rx); if (!rcvd) continue; for (uint32_t i = 0; i < rcvd; i++) { const struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk_info->rx, idx_rx + i); uint64_t addr = desc->addr; uint32_t len = desc->len; uint8_t *pkt = (uint8_t *)xsk_umem__get_data(umem_info->buffer, addr); process_packet(pkt, len); } xsk_ring_cons__release(&xsk_info->rx, rcvd); // Return frames to fill ring refill_fill_ring(umem_info, rcvd); } ``` ### 6. Copy vs zero-copy | Mode | Flag | Requirements | |------|------|--------------| | Copy | `XDP_COPY` (default) | Any driver; kernel copies to UMEM | | Zero-copy | `XDP_ZEROCOPY` | Driver support (i40e, ixgbe, mlx5, etc.) | ```bash # Check driver ZC support ethtool -i eth0 # Kernel log on bind: dmesg | grep xsk # "Zero-copy enabled" or "Copy mode" ``` Zero-copy: NIC DMAs directly into UMEM frames — lowest latency. Copy mode: safer, universal. ### 7. libbpf xsk.h helpers ```bash # Modern libbpf includes xsk API pkg-config --libs libbpf # -lbpf -lxdp (if separate libxdp installed) ``` Key functions: - `xsk_umem__create` / `xsk_umem__delete` - `xsk_socket__create` / `xsk_socket__delete` - `xsk_umem__get_data` — pointer from frame address - `xsk_socket__fd` — for epoll/poll integration ### 8. Performance vs DPDK | Factor | AF_XDP | DPDK | |--------|--------|------| | Setup complexity | Moderate | High (hugepages, EAL) | | Kernel integration | XDP filter in kernel | Full bypass | | Typical throughput | Near-DPDK with ZC | Highest | | NIC binding | Stays on kernel driver | vfio/uio binding | | Use case fit | Filter + selective userspace | Full dataplane takeover | ### 9. Production patterns ``` Common deployments ├── CDN edge cache — XDP_DROP junk, redirect cacheable to XSK ├── DDoS mitigation — XDP_DROP attack patterns ├── Load balancer — XDP_TX hairpin or redirect to backend XSK └── Observability — mirror subset to XSK for analysis ``` ```bash # Multi-queue: one XSK per RX queue, pinned to CPU taskset -c 2 ./xsk_app --queue 2 ``` ## Common Problems | Symptom | Cause | Fix | |---------|-------|-----| | No packets in RX ring | XDP program not redirecting | Verify xsks_map entry for queue index | | `EBUSY` on socket create | XDP already attached | `ip link set dev eth0 xdp off` first | | Zero-copy fallback to copy | Driver lacks ZC | Check dmesg; use supported NIC | | Packet drops | Fill ring empty | Aggressive refill; increase NUM_FRAMES | | TX not working | Completion ring not polled | Process comp ring to recycle frames | | Permission denied | CAP_NET_RAW needed | Run with appropriate capabilities | ## Related Skills - `skills/observability/ebpf` — XDP/BPF program development - `skills/async-io/dpdk` — full kernel bypass alternative - `skills/async-io/io-uring` — async I/O for non-packet workloads - `skills/observability/ebpf-rust` — Aya for XDP in Rust - `skills/profilers/linux-perf` — profile XDP program CPU usage - `skills/allocators/numa-programming` — NUMA-local UMEM allocation