# Learn eBPF Tracing: Tutorial and Examples
URL:: http://www.brendangregg.com/blog/2019-01-01/learn-ebpf-tracing.html
Author:: brendangregg.com
## Highlights
> eBPF should stand for something meaningful, like Virtual Kernel Instruction Set (VKIS), but due to its origins it is extended Berkeley Packet Filter. It can be used for many things: network performance, firewalls, security, tracing, and device drivers. Some of these have plenty of free documentation online, like for tracing, and others not yet. The term tracing refers to performance analysis and observability tools that can produce per-event info. You may have already use a tracer: tcpdump and strace are specialized tracers. ([View Highlight](https://read.readwise.io/read/01fpa3whseqj1s74ydzmnbttrc))
> • Beginner: run [bcc](https://github.com/iovisor/bcc) tools
> • Intermediate: develop [bpftrace](https://github.com/iovisor/bpftrace) tools
> • Advanced: develop [bcc](https://github.com/iovisor/bcc) tools, contribute to bcc & bpftrace ([View Highlight](https://read.readwise.io/read/01fpa3xz2jc38kz5t77bxp91h1))
> **eBPF** does to Linux what JavaScript does to HTML. (Sort of.) So instead of a static HTML website, JavaScript lets you define mini programs that run on events like mouse clicks, which are run in a safe virtual machine in the browser. And with eBPF, instead of a fixed kernel, you can now write mini programs that run on events like disk I/O, which are run in a safe virtual machine in the kernel. In reality, eBPF is more like the v8 virtual machine that runs JavaScript, rather than JavaScript itself. eBPF is part of the Linux kernel. ([View Highlight](https://read.readwise.io/read/01fpa3zhvzw83c594hwcd7wcdh))
> This eBPF-based tool shows completed TCP sessions, with their process ID (PID) and command name (COMM), sent and received bytes (TX_KB, RX_KB), and duration in milliseconds (MS):
> # **tcplife**
> PID COMM LADDR LPORT RADDR RPORT TX_KB RX_KB MS
> 22597 recordProg 127.0.0.1 46644 127.0.0.1 28527 0 0 0.23
> 3277 redis-serv 127.0.0.1 28527 127.0.0.1 46644 0 0 0.28
> 22598 curl 100.66.3.172 61620 52.205.89.26 80 0 1 91.79
> 22604 curl 100.66.3.172 44400 52.204.43.121 80 0 1 121.38
> 22624 recordProg 127.0.0.1 46648 127.0.0.1 28527 0 0 0.22
> 3277 redis-serv 127.0.0.1 28527 127.0.0.1 46648 0 0 0.27
> 22647 recordProg 127.0.0.1 46650 127.0.0.1 28527 0 0 0.21
> 3277 redis-serv 127.0.0.1 28527 127.0.0.1 46650 0 0 0.26
> [...]
> eBPF *did not* make this possible – I can rewrite tcplife to use older kernel technologies. But if I did, we'd never run such a tool in production due to the performance overhead, security issues, or both. What eBPF did was make this tool *practical*: it is efficient and secure. For example, it does not trace every packet like older techniques, which can add too much performance overhead. Instead, it only traces TCP session events, which are much less frequent. This makes the overhead so low we can run this tool in production, 24x7. ([View Highlight](https://read.readwise.io/read/01fsy14jvcwkms6907ebwpb3hd))
> For beginners, try the tools from bcc. See the [bcc install instructions](https://github.com/iovisor/bcc/blob/master/INSTALL.md) for your OS. On Ubuntu, it may be like:
> # **sudo apt-get update**
> # **sudo apt-get install bpfcc-tools**
> # **sudo /usr/share/bcc/tools/opensnoop**
> PID COMM FD ERR PATH
> 25548 gnome-shell 33 0 /proc/self/stat
> 10190 opensnoop -1 2 /usr/lib/python2.7/encodings/ascii.x86_64-linux-gnu.so
> 10190 opensnoop -1 2 /usr/lib/python2.7/encodings/ascii.so
> 10190 opensnoop -1 2 /usr/lib/python2.7/encodings/asciimodule.so
> 10190 opensnoop 18 0 /usr/lib/python2.7/encodings/ascii.py
> 10190 opensnoop 19 0 /usr/lib/python2.7/encodings/ascii.pyc
> 25548 gnome-shell 33 0 /proc/self/stat
> 29588 device poll 4 0 /dev/bus/usb
> ^C
> There I finished by running opensnoop to test that the tools worked. If you get this far, you've used eBPF!
> Companies including Netflix and Facebook have bcc installed on all servers by default, and maybe you'll want to as well. ([View Highlight](https://read.readwise.io/read/01fsy14vtsfxecf9hfjv8wy94f))
> • [bcc Tutorial](https://github.com/iovisor/bcc/blob/master/docs/tutorial.md) ([View Highlight](https://read.readwise.io/read/01fsy15jemtnq56j1c4s9pfv8b))
> [](http://www.brendangregg.com/Perf/bcc_tracing_tools.png) ([View Highlight](https://read.readwise.io/read/01fsy15rp5tmdww1p6cytpmx1b))
---
Title: Learn eBPF Tracing: Tutorial and Examples
Author: brendangregg.com
Tags: readwise, articles
date: 2024-01-30
---
# Learn eBPF Tracing: Tutorial and Examples
URL:: http://www.brendangregg.com/blog/2019-01-01/learn-ebpf-tracing.html
Author:: brendangregg.com
## AI-Generated Summary
Learn eBPF Tracing: Tutorial and Examples
## Highlights
> eBPF should stand for something meaningful, like Virtual Kernel Instruction Set (VKIS), but due to its origins it is extended Berkeley Packet Filter. It can be used for many things: network performance, firewalls, security, tracing, and device drivers. Some of these have plenty of free documentation online, like for tracing, and others not yet. The term tracing refers to performance analysis and observability tools that can produce per-event info. You may have already use a tracer: tcpdump and strace are specialized tracers. ([View Highlight](https://read.readwise.io/read/01fpa3whseqj1s74ydzmnbttrc))
> • Beginner: run [bcc](https://github.com/iovisor/bcc) tools
> • Intermediate: develop [bpftrace](https://github.com/iovisor/bpftrace) tools
> • Advanced: develop [bcc](https://github.com/iovisor/bcc) tools, contribute to bcc & bpftrace ([View Highlight](https://read.readwise.io/read/01fpa3xz2jc38kz5t77bxp91h1))
> **eBPF** does to Linux what JavaScript does to HTML. (Sort of.) So instead of a static HTML website, JavaScript lets you define mini programs that run on events like mouse clicks, which are run in a safe virtual machine in the browser. And with eBPF, instead of a fixed kernel, you can now write mini programs that run on events like disk I/O, which are run in a safe virtual machine in the kernel. In reality, eBPF is more like the v8 virtual machine that runs JavaScript, rather than JavaScript itself. eBPF is part of the Linux kernel. ([View Highlight](https://read.readwise.io/read/01fpa3zhvzw83c594hwcd7wcdh))
> This eBPF-based tool shows completed TCP sessions, with their process ID (PID) and command name (COMM), sent and received bytes (TX_KB, RX_KB), and duration in milliseconds (MS):
> # **tcplife**
> PID COMM LADDR LPORT RADDR RPORT TX_KB RX_KB MS
> 22597 recordProg 127.0.0.1 46644 127.0.0.1 28527 0 0 0.23
> 3277 redis-serv 127.0.0.1 28527 127.0.0.1 46644 0 0 0.28
> 22598 curl 100.66.3.172 61620 52.205.89.26 80 0 1 91.79
> 22604 curl 100.66.3.172 44400 52.204.43.121 80 0 1 121.38
> 22624 recordProg 127.0.0.1 46648 127.0.0.1 28527 0 0 0.22
> 3277 redis-serv 127.0.0.1 28527 127.0.0.1 46648 0 0 0.27
> 22647 recordProg 127.0.0.1 46650 127.0.0.1 28527 0 0 0.21
> 3277 redis-serv 127.0.0.1 28527 127.0.0.1 46650 0 0 0.26
> [...]
> eBPF *did not* make this possible – I can rewrite tcplife to use older kernel technologies. But if I did, we'd never run such a tool in production due to the performance overhead, security issues, or both. What eBPF did was make this tool *practical*: it is efficient and secure. For example, it does not trace every packet like older techniques, which can add too much performance overhead. Instead, it only traces TCP session events, which are much less frequent. This makes the overhead so low we can run this tool in production, 24x7. ([View Highlight](https://read.readwise.io/read/01fsy14jvcwkms6907ebwpb3hd))
> For beginners, try the tools from bcc. See the [bcc install instructions](https://github.com/iovisor/bcc/blob/master/INSTALL.md) for your OS. On Ubuntu, it may be like:
> # **sudo apt-get update**
> # **sudo apt-get install bpfcc-tools**
> # **sudo /usr/share/bcc/tools/opensnoop**
> PID COMM FD ERR PATH
> 25548 gnome-shell 33 0 /proc/self/stat
> 10190 opensnoop -1 2 /usr/lib/python2.7/encodings/ascii.x86_64-linux-gnu.so
> 10190 opensnoop -1 2 /usr/lib/python2.7/encodings/ascii.so
> 10190 opensnoop -1 2 /usr/lib/python2.7/encodings/asciimodule.so
> 10190 opensnoop 18 0 /usr/lib/python2.7/encodings/ascii.py
> 10190 opensnoop 19 0 /usr/lib/python2.7/encodings/ascii.pyc
> 25548 gnome-shell 33 0 /proc/self/stat
> 29588 device poll 4 0 /dev/bus/usb
> ^C
> There I finished by running opensnoop to test that the tools worked. If you get this far, you've used eBPF!
> Companies including Netflix and Facebook have bcc installed on all servers by default, and maybe you'll want to as well. ([View Highlight](https://read.readwise.io/read/01fsy14vtsfxecf9hfjv8wy94f))
> • [bcc Tutorial](https://github.com/iovisor/bcc/blob/master/docs/tutorial.md) ([View Highlight](https://read.readwise.io/read/01fsy15jemtnq56j1c4s9pfv8b))
> [](http://www.brendangregg.com/Perf/bcc_tracing_tools.png) ([View Highlight](https://read.readwise.io/read/01fsy15rp5tmdww1p6cytpmx1b))