BPF Updates 12
This is issue 12 of the regular newsletter around BPF written by Alexander Alemayhu. It summarizes ongoing development, presentations, videos and other information related to BPF and XDP. It is released roughly once a week.
The highlights since the last time
- Generic metadata transfer from XDP into skb via new helper function
bpf_xdp_adjust_meta
. bpf_perf_event_read_value
helper function series got merged.- Multiple programs can now be attached to a cgroup.
- A new map type
cpumap
for XDP got merged.
The addition of metadata transfer from XDP allows XDP programs to apply early filters to extract metadata from packets and make it available to BPF programs later in the forwarding chain such as programs attached to clsact qdiscs.
The new values available via the perf helper make it easier to normalize measurements. cpumap
let's you redirect XDP frames to remote CPU's where these can handle the packet representation and fire up the networking stack.
More interesting topics
- More fields added to
bpf_prog_info
. - Netronome bpftool upstreamed.
- LPM performance improvements.
- 32-bit eBPF encoding support.
The BPF tooling is evolving and introspection is getting better.
Presentations
Talks
All Systems Go! 2017 - A gentle introduction to [e]BPF
Beginner friendly overview of BPF.
All Systems Go! 2017 - High-performance Linux monitoring with eBPF
Intro to BPF and short part on how it's being leveraged in weaveworks.
All Systems Go! 2017 - Using BPF in Kubernetes
Nice talk focusing on some of the use cases in the cloud.
Kernel Recipes 2017 - EBPF and XDP - Eric Leblond
Overview of how Suricata uses BPF Nice to see the version number for the various features / work mentioned. The slides also has some code.
Kernel Recipes 2017 - Performance Analysis with BPF - Brendan Gregg
Introduction to BPF focusing on BCC and tracing.
Slides
eBPF cgroup filters for data usage accounting on Android
Interesting work on how to use BPF for wifi data usage.
Cilium – Kernel Native Security & DDOS Mitigation for Microservices with BPF
High level introduction to Cilium with some new ongoing work on Cilium. If you are using / interested in Kafka, worth checking out for the API filtering.
eBPF Implementation for FreeBSD
Interesting slides on eBPF port to FreeBSD.
In case you missed it
Heap Allocation Flamegraphs
Post on using one of the grave tools for tracing JVM processes.
Linux の BPF : (5) eBPF による Linux Kernel Tracing
Going through a tracing example using kernel 4.12.
An intro to using eBPF to filter packets in the Linux kernel
Beginner friendly post with a accompanying example for tracing.
IP Accounting and Access Lists with systemd
Simple cgroup/BPF packet counting and blacklist via systemd.
XDP on Power
XDP on the Power architecture.
Cilium v0.10 & v0.11 Released: Double the Fun - Two Updates in One!
Great post on the recent progress made in the Cilium project.
Projects
A couple of new GitHub projects. Please star if you like the projects ;)
Cilium client API example
Simple example illustrating use of the Cilium API.
ebpf_asm
An assembler for eBPF programs written in an Intel-like assembly syntax.
go-ebpf
eBPF programs without a libbcc dependency
generic-ebpf
Generic eBPF VM. Currently support FreeBSD kernel, FreeBSD userspace, Linux kernel, Linux userspace and MacOSX userspace.
Random cool note
Schedule is out! Dont miss this amazing event. Come to the land of Kimchi and Palaces. Hangout with awesome geeks. https://www.netdevconf.org/2.2/schedule.html
Patches
- Daniel Borkmann, [PATCH net 0/3] Fix for BPF devmap percpu allocation splat
- Richard Weinber, [PATCH 1/3] bpf: Don't check for current being NULL
- Tariq Toukan, [PATCH net-next 0/3] mlx4_en XDP TX improvements
- Chenbo Feng, [PATCH net-next v6 0/5] bpf: security: New file mode and LSM hooks for eBPF object permission control
- [PATCH net-next v6 1/5] bpf: Add file mode configuration into bpf maps
- [PATCH net-next v6 2/5] bpf: Add tests for eBPF file mode
- [PATCH net-next v6 3/5] security: bpf: Add LSM hooks for bpf object related syscall
- [PATCH net-next v6 4/5] selinux: bpf: Add selinux check for eBPF syscall operations
- [PATCH net-next v6 5/5] selinux: bpf: Add addtional check for bpf object file receive
- Jakub Kicinski, [PATCH net-next 0/4] bpf: move context info out of the verifier
- Jakub Kicinski, [PATCH net v2] bpf: disallow arithmetic operations on context pointer
- Jakub Kicinski, [PATCH net-next] tools: bpftool: use more common tag format
- Jakub Kicinski, [PATCH net-next 00/12] nfp: bpf: support direct packet access
- [PATCH net-next 01/12] bpf: verifier: set reg_type on context accesses in second pass
- [PATCH net-next 02/12] nfp: bpf: reorder arguments to
emit_ld_field_any()
- [PATCH net-next 03/12] nfp: bpf: add missing return in jne_imm optimization
- [PATCH net-next 04/12] nfp: bpf: fix compare instructions
- [PATCH net-next 05/12] nfp: bpf: add mov helper
- [PATCH net-next 06/12] nfp: bpf: implement byte swap instruction
- [PATCH net-next 07/12] nfp: bpf: support BPF offload only on little endian
- [PATCH net-next 08/12] nfp: bpf: fix context accesses
- [PATCH net-next 09/12] nfp: bpf: separate I/O from checks for legacy data load
- [PATCH net-next 10/12] nfp: bpf: add support for direct packet access - read
- [PATCH net-next 11/12] nfp: bpf: direct packet access - write
- [PATCH net-next 12/12] nfp: bpf: support direct packet access in TC
- Jesper Dangaard Brouer, [net-next V8 PATCH 0/5] New bpf cpumap type for XDP_REDIRECT
- [net-next V8 PATCH 1/5] bpf: introduce new bpf cpu map type
BPF_MAP_TYPE_CPUMAP
- [net-next V8 PATCH 2/5] bpf: XDP_REDIRECT enable use of cpumap
- [net-next V8 PATCH 3/5] bpf: cpumap xdp_buff to skb conversion and allocation
- [net-next V8 PATCH 4/5] bpf: cpumap add tracepoints
- [net-next V8 PATCH 5/5] samples/bpf: add cpumap sample program
xdp_redirect_cpu
- [net-next V8 PATCH 1/5] bpf: introduce new bpf cpu map type
- Jakub Kicinski, [PATCH net-next v2 0/7] bpf: get rid of global verifier state and reuse instruction printer
- [PATCH net-next v2 1/7] selftests/bpf: add a test for verifier logs
- [PATCH net-next v2 2/7] bpf: encapsulate verifier log state into a structure
- [PATCH net-next v2 3/7] bpf: move global verifier log into verifier environment
- [PATCH net-next v2 4/7] bpf: move instruction printing into a separate file
- [PATCH net-next v2 5/7] tools: bpftool: use the kernel's instruction printer
- [PATCH net-next v2 6/7] bpf: don't rely on the verifier lock for metadata_dst allocation
- [PATCH net-next v2 7/7] bpf: write back the verifier log buffer as it gets filled
- Jiong Wang, [PATCH, bpf-llvm] Fix bug on silently truncating 64-bit immediate
- Richard Weinberger, [PATCH] bpf: devmap: Check attr->max_entries more carefully
- Colin King, [PATCH][bpf-next] bpf: remove redundant variable old_flags
- Steven Rostedt, [PATCH] tracing: bpf: Hide bpf trace events when they are not used
- Jakub Kicinski, [PATCH net-next 00/15] nfp: bpf ABIv2 and multi port
- [PATCH net-next 01/15] nfp: output control messages to
trace_devlink_hwmsg()
- [PATCH net-next 02/15] nfp: bpf: lift the single-port limitation
- [PATCH net-next 03/15] nfp: bpf: use the power of sparse to check we encode registers right
- [PATCH net-next 04/15] nfp: bpf: move software reg helpers and cmd table out of translator
- [PATCH net-next 05/15] nfp: bpf: encode all 64bit shifts
- [PATCH net-next 06/15] nfp: bpf: remove register rename
- [PATCH net-next 07/15] nfp: bpf: remove packet marking support
- [PATCH net-next 08/15] nfp: add more white space to the instruction defines
- [PATCH net-next 09/15] nfp: bpf: encode LMEM accesses
- [PATCH net-next 10/15] nfp: bpf: encode extended LM pointer operands
- [PATCH net-next 11/15] nfp: bpf: move to datapath ABI version 2
- [PATCH net-next 12/15] nfp: bpf: calculate code store ECC
- [PATCH net-next 13/15] nfp: bpf: pad code with valid nops
- [PATCH net-next 14/15] nfp: bpf: byte swap the instructions
- [PATCH net-next 15/15] nfp: bpf: pass dst register to ld_field instruction
- [PATCH net-next 01/15] nfp: output control messages to
- Yonghong Song, [llvm] r315889 - bpf: fix bug on silently truncating 64-bit immediate
- Shmulik Ladkani, [PATCH v2] netfilter:
xt_bpf
: FixXT_BPF_MODE_FD_PINNED
mode of 'xt_bpf_info_v1
' - Christina Jacob, [PATCH v2] XDP Program for Ip forward
- Martin KaFai Lau, [PATCH net-next 0/3] bpf: Misc improvements and a new usage on bpf obj name
- Yonghong Song, [PATCH net-next v7 0/5] bpf: add two helpers to read perf event enabled/running time
- [PATCH net-next v7 1/5] bpf: perf event change needed for subsequent bpf helpers
- [PATCH net-next v7 2/5] bpf: add helper
bpf_perf_event_read_value
for perf event array map - [PATCH net-next v7 3/5] bpf: add a test case for helper
bpf_perf_event_read_value
- [PATCH net-next v7 4/5] bpf: add helper
bpf_perf_prog_read_value
- [PATCH net-next v7 5/5] bpf: add a test case for helper
bpf_perf_prog_read_value
- Craig Gallek, [PATCH net-next v3 0/2] libbpf: support more map options
- Jesper Dangaard Brouer, [net-next PATCH 0/3] Improve xdp_monitor samples/bpf
- Alexei Starovoitov, [PATCH net] bpf: fix liveness marking
- Alexei Starovoitov, [PATCH v2 net-next 0/8] bpf: muli prog support for cgroup-bpf
- [PATCH v2 net-next 1/8] bpf: multi program support for cgroup+bpf
- [PATCH v2 net-next 2/8] bpf: introduce
BPF_PROG_QUERY
command - [PATCH v2 net-next 3/8] bpf: enforce return code for cgroup-bpf programs
- [PATCH v2 net-next 4/8] libbpf: introduce
bpf_prog_detach2()
- [PATCH v2 net-next 5/8] samples/bpf: add multi-prog cgroup test case
- [PATCH v2 net-next 6/8] libbpf: sync bpf.h
- [PATCH v2 net-next 7/8] libbpf: add support for
BPF_PROG_QUERY
- [PATCH v2 net-next 8/8] samples/bpf: use
bpf_prog_query()
interface
- Yonghong Song, [llvm] r314911 - bpf: fix an insn encoding issue for neg insn
- Alexei Starovoitov, [PATCH net] bpf: fix
bpf_tail_call()
x64 JIT - Jakub Kicinski, [PATCH net-next v4 0/3] tools: add bpftool
- Eric Dumazet, [PATCH net] socket, bpf: fix possible use after free
- Stephen Hemminger, [PATCH net-next] samples/bpf: fix warnings in
xdp_monitor_user
- Martin KaFai Lau, [PATCH net-next] bpf: Fix compiler warning on info.map_ids for 32bit platform
- Yonghong Song, [llvm] r314469 - bpf: fix a bug for disassembling ld_pseudo inst
- Paul Chaignon, [PATCH RFC v3] bpf: allow map helpers access to map values directly
- Mark Rutland, EBPF-triggered WARNING at mm/percpu.c:1361 in v4-14-rc2
- Yonghong Song, [llvm] r314376 - bpf: add new insns for
bswap_to_le
and negation - Martin KaFai Lau, [PATCH net-next 0/5] bpf: Extend
bpf_{prog,map}_info
- [PATCH net-next 1/5] bpf: Add name,
load_time
, uid andmap_ids
tobpf_prog_info
- [PATCH net-next 2/5] bpf: Add
map_name
tobpf_map_info
- [PATCH net-next 3/5] bpf: libbpf: Provide basic API support to specify BPF obj name
- [PATCH net-next 4/5] bpf: Swap the order of checking
prog_info
andmap_info
- [PATCH net-next 5/5] bpf: Test new fields in
bpf_attr
andbpf_{prog,map}_info
- [PATCH net-next 1/5] bpf: Add name,
- Matt Redfearn, [PATCH] MIPS: bpf: Fix uninitialised target compiler error
- Edward Cree, [PATCH v2 net-next 0/2] bpf/verifier: disassembly improvements
- Daniel Borkmann, [PATCH net-next 0/6] BPF metadata for direct access
- [PATCH net-next 1/6] bpf: rename
bpf_compute_data_end
intobpf_compute_data_pointers
- [PATCH net-next 2/6] bpf: add meta pointer for direct access
- [PATCH net-next 3/6] bpf: update bpf.h uapi header for tools
- [PATCH net-next 4/6] bpf: improve selftests and add tests for meta pointer
- [PATCH net-next 5/6] bpf, nfp: add meta data support
- [PATCH net-next 6/6] bpf, ixgbe: add meta data support
- [PATCH net-next 1/6] bpf: rename
- Jason Wang, [PATCH net-next] virtio-net: correctly set xdp_xmit for mergeable buffer
- Yonghong Song, [PATCH] bpf: add support for neg insn and change format of bswap insn
- Edward Cree, [PATCH net-next] bpf/verifier: improve disassembly of BPF_END instructions
- Cong Wang, [Patch net-next v2] net_sched: use idr to allocate bpf filter handles
- Jiong Wang, [llvm] r313958 - bpf: refactor inst patterns with more mnemonics
- Jiong Wang, [llvm] r313959 - bpf: refactor inst patterns with better inheritance
- Jiong Wang, [llvm] r313960 - bpf: add 32bit register set
- Jiong Wang, [llvm] r313961 - bpf: initial 32-bit ALU encoding support in assembler
- Craig Gallek, [PATCH net-next v2] bpf: Optimize lpm trie delete
- Joel Fernandes, [PATCH v4 0/4] Add cross-compilation support to eBPF samples
- Daniel Borkmann, [PATCH iproute2 master 0/2] BPF/XDP json follow-up
- Edward Cree, [PATCH net] net: change skb->
mac_header
when Generic XDP callsadjust_head
- Jason Wang [PATCH net-next 1/3] virtio-net: remove unnecessary parameter of
virtnet_xdp_xmit()
- Eric Dumazet, [PATCH net] bpf: do not disable/enable BH in
bpf_map_free_id()
- Jiong Wang, [PATCH RFC 0/4] Initial 32-bit eBPF encoding support
- Yonghong Song, [PATCH net] bpf: one perf event close won't free bpf program attached by another perf event
- Daniel Borkmann, [PATCH net v2] bpf: fix ri->
map_owner
pointer onbpf_prog_realloc
- Yonghong Song, [llvm] r313593 - bpf: add inline-asm support
- Craig Gallek, [PATCH net-next 0/3] Implement delete for BPF LPM trie
- Tobias Klauser, [PATCH] bpf: devmap: pass on return value of
bpf_map_precharge_memlock
- Simon Dardis, [llvm] r313281 - [bpf] Fix test to always use little endian.
- Edward Cree, [PATCH net] bpf/verifier: reject
BPF_ALU64|BPF_END
Please note that netdev receives a lot of patches and the list above is not meant to be comprehensive.