Improving Observability in Cloud Native Environments

Enhancing Observability in Cloud Native Environments with eBPF

By Rory McCune, Cloud Native Security Advocate at Aqua Security

The speed of organisations’ digital transformation initiatives has dramatically increased over the course of the pandemic. The rate of this process can come at a cost, resulting in frequent infrastructure disruptions, troublesome applications, and major gaps in cybersecurity defences. However, with the help of observability tools, DevSecOps teams can identify threats and suspicious behaviours in the early phases of a cyberattack. These solutions provide rapid feedback reports to administrators enabling them to react appropriately to the attack within their network environment.

Observability does not look the same across all cloud native and non-cloud native environments. So maintaining performance whilst capturing the information that security organisations need can be a significant challenge. This is where Extended Berkeley Packet Filter (eBPF) technology comes in – it achieves fast and powerful observability in cloud native environments.

The eBPF solution

BPF, eBPFs predecessor, was designed to capture useful network packets directly from the kernel, eliminating the need to copy them to the user space and then funnel them out through a network tap. BPF runs code in the kernel in order to decide which packets to filter in or out. This results in unnecessary traffic being filtered out without a lengthy manual process.

Building on this, eBPF enables the user to run arbitrary code in the kernel and can be triggered by many different types of events, not just the arrival of network packets. For example, by attaching the program to a “kprobe” event, eBPF code can be triggered to run when a kernel function starts.

eBPF based applications are typically made up of two parts. The first is the kernel space code, which is responsible for capturing relevant events and making them available to the user space application. The second part is the user space program, which reviews the events shared by the eBPF code running in the kernel and can refine and augment this low-level information with additional context. These two parts together enable the identification of unexpected workloads as they appear.

eBPF provides a major advantage of other approaches to the problem which would traditionally have used kernel module code to achieve the same results. Using kernel modules often presents risks to the stability of the overall system, as a bug in the module can crash the whole kernel. eBPF on the other hand uses a verifier to ensure that code is safe and then runs it in a sandbox, giving the performance advantages of kernel level software, without the risks.

As well as eBPF’s incredible speed, it also has exceptional workload monitoring capabilities. eBPF code can run in response to file events to check if they are expected for that workload. This means that it will pick up on any unexpected behaviour in the application and highlight it for review.

Pushing the limits

A container is fundamentally a Linux process, therefore eBPF can interact easily with them. Open source projects can push the limits of using eBPF technology, especially in the case of runtime Linux forensics. It is possible to make open source interoperable for different versions of the Linux kernel so others can create eBPF programs that are easier to use for a mainstream DevOps audience. With this capability eBPF holds the key to powerful observability in cloud native environments.

Projects run by eBPF technology can be used to better appreciate a program’s runtime behaviour and can take on challenges that may prove difficult for other security software solutions. eBPF technology is important to obtain the relevant observability, monitoring, and forensics capabilities for cloud native environments. Improving these components is an absolute must to uphold effective cybersecurity standards.

Nowadays, there are toolkits like BCC (BPF Compiler Collection) that can be used to write eBPF programs in modern day languages like Go and Python, rather than having to manually write an eBPF program in bytecode. With this process simplified, users can better appreciate the ease of use and benefits that come along with eBPF technologies, and the programs run by it within their cloud native environments.

Running applications usually interact with the operating system via system calls (syscalls). One of the key benefits of eBPF is that it allows the running of sandboxed programs in the Linux kernel without changing the kernel source code or loading kernel modules. Therefore, by attaching such programs to Linux Security Module (LSM) hooks using kprobes, it is easy to collect the argument values that were used by the kernel. This dramatically simplifies the process of analysing the behaviour of the running application via syscalls, which in turn make issue resolution much easier.

eBPF programs are a high performance and effective tool for monitoring system software within a cloud native environment. Its advanced observability can alert an administrator around sudden and unusual behaviour. As powerful as it is, only several lines of code are needed to arm your system with maximum observability.