%% date:: [[2022-11-17]], [[2023-04-23]], [[2023-11-26]], [[2023-12-18]] %% # [[eBPF]] eBPF stands for extended Berkeley Packet Filter, and it is a technology that allows developers to run programs in an operating system [[Kernel]] without having to change the kernel source code itself or use kernel modules. [^ebpf] eBPF is a form of [[Auto-instrumentation|Auto]]-[[Instrumentation]]. <iframe width="560" height="315" src="https://www.youtube.com/embed/UX5aeL5KeZs" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> ## What's the deal with eBPF? Interestingly, eBPF doesn't add any more functionality compared to traditional forms of [[Instrumentation]]; instead, people are talking about it because what it adds is ease of use. [^odysseas] eBPF is often compared to [[JavaScript]] in that JavaScript revolutionized the web by allowing code to run on the client, rather than on the server. Before JavaScript, web pages were written in pure HTML, and were little more than documents displayed publicly. JavaScript allowed developers to create web pages that were more interactive and outsourced some of the processing to the clients. eBPF does the same thing for kernel code: ![[eBPF monitoring diagram.png]] %%[[eBPF monitoring diagram|🖋 Edit in Excalidraw]]%% ## [[How eBPF is different from traditional instrumentation]] ![[anton-rodriguez-ebpf-diagram.png]] *ebPF by Anton Rodriguez at his presentation in the Kafka Summit* [^rodriguez] Together with my colleagues, [[Artem Charykov]] and [[Daniel González Lopes]], I did a hackathon based on how to implement eBPF for [[Grafana Labs]]. Here's a video I made to summarize what we did. This video also includes an explanation of what eBPF is: <iframe width="560" height="315" src="https://www.youtube.com/embed/bmroQsJxTL8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe> %%[[Hackathon - eBPF Monitoring]]%% ## Advantages of eBPF ### It's event-driven eBPF programs run code in response to events that have occurred, using either pre-defined or custom hooks like system calls, HTTP requests, or network events. [^ebpf] ### It's highly efficient eBPF programs use a JIT (Just In Time) compiler that allow programmability without too much overhead. Some estimate the CPU and memory overhead of eBPF monitoring at 1%. [^netflix] ### It's secure Using eBPF programs is more secure than modifying [[Kernel]] source code. eBPF programs run on top of the source code, so they cannot corrupt the kernel itself. ### It allows [[Continuous Improvement|Continuous Delivery]] eBPF code can be shipped and used without requiring changes to the kernel code or version. ## Usage eBPF programs are used for the following use cases: - Load-balancing and networking - Extracting observability data (metrics, logs, and traces) with low overhead - Enforcing security rules [^ebpf] ## eBPF Tools - BCC - bpftrace - kubectl-trace - [[Cilium]] by [[Isovalent]], in partnership with [[Grafana Labs]] - [[Grafana Beyla]] ### Commercial products using eBPF - [[Pixie]] by [[New Relic]] - [[Datadog]]'s agent [collects data via eBPF](https://docs.datadoghq.com/network_monitoring/performance/setup/?tab=agentlinux) - [[Parca]] by [[Polar Signals]] - [ebpf Exporter](https://github.com/cloudflare/ebpf_exporter) by [[Cloudflare]] is a [[Prometheus]] exporter for custom eBPF metrics ## Companies using eBPF - [[Netflix]] uses eBPF for monitoring network traffic in their [[Microservices|microservices-based]] architecture. [^netflix] [^ebpf]: _What is eBPF? An Introduction and Deep Dive into the eBPF Technology_. Retrieved from https://ebpf.io/what-is-ebpf/ . [[What Is eBPF|My highlights.]] [^rodriguez]: Rodriguez, A. (2021). _Monitoring Kafka Without Instrumentation Using eBPF_. Kafka Summit 2021. [^odysseas]: Lamtzidis, O. (2020). *Using eBPF monitoring to know what to measure and why*. Container Journal. Retrieved from https://containerjournal.com/topics/container-management/using-ebpf-monitoring-to-know-what-to-measure-and-why/ [^netflix]: Tiagri, A. et al. (2021). *How Netflix uses eBPF flow logs at scale for network insight*. Netfix TechBlog. Retrieved from https://netflixtechblog.com/how-netflix-uses-ebpf-flow-logs-at-scale-for-network-insight-e3ea997dca96