eBPF: An Acronym That Doesn’t Have a Meaning, Yet Could Imply Banking
Meta has reported a significant 20% reduction in CPU cycles for its major services, thanks to its Strobelight profiling orchestration suite. This system utilizes the open-source eBPF project, leading to a reduction of about 10% to 20% fewer servers needed for popular platforms like Facebook, Instagram, and WhatsApp, as well as virtual reality experiences.
eBPF, which originally stood for “Extended Berkeley Packet Filter,” has evolved beyond its initial definition. It now encompasses broader functions, allowing for more comprehensive uses within the operating system kernel. This software runs sandboxed programs efficiently and safely, particularly within the Linux kernel. There is also an ongoing effort to adapt it for Windows systems. The real advantage of eBPF is that it allows developers to run custom code within the kernel without needing to create and compile kernel modules or drivers.
Optimizing performance at the kernel level is crucial, especially for large-scale operations. Even small performance bottlenecks can lead to significant issues when they amplify across multiple services. Collecting data consistently across diverse systems and ensuring that it remains interpretable through various versions of a kernel presents considerable challenges.
To tackle these performance enhancements, Meta developed the open-source Strobelight, which manages several profiling applications that leverage eBPF. This system is designed to gather observability data, which includes logs of system events, performance metrics, and traces of network connections. The primary objective of Strobelight is to optimize infrastructure efficiency, which considerably lowers operational costs.
Meta software engineer Jordan Rome emphasized that eBPF facilitates the safe inclusion of custom code into the kernel, which allows for minimal overhead in data collection. This capability greatly expands the possibilities within the observability realm, making the Strobelight tool extremely effective. Strobelight currently includes 42 distinct profiling applications, which assess various factors such as memory usage, function call counts, events across programming languages, GPU utilization for AI tasks, and service request latencies.
A recent case study by the eBPF Foundation highlighted an impressive achievement: Meta saved enough server capacity through a simple one-character code alteration—specifically, one ampersand (&)—which resulted in annual savings equivalent to 15,000 servers. This change was discovered by a performance engineer examining data from Strobelight, enabling them to pinpoint a costly inefficiency arising from an unintended array copy in the C++ programming language.
By replacing an automatic copy operation with a reference through the addition of an ampersand, the engineer avoided unnecessary data duplication, leading to significant cost-efficient benefits. Notably, Rome described how a relatively small adjustment could yield massive savings in server capacity once implemented in production, illustrating the profound impact even minor code changes can have on overall system efficiency.
The ongoing developments and applications of eBPF through tools like Strobelight not only optimize Meta’s services but also highlight the potential of open-source technologies in enhancing operational performance across various computing environments.
