Rustinel: Open-source endpoint detection for Windows and Linux

Open-source endpoint detection has traditionally been divided, with tools primarily catering to Windows environments centered around Sysmon, while Linux solutions have revolved around eBPF or auditd. This division has posed challenges for defenders managing mixed operating systems, necessitating the integration of disparate pipelines, rule sets, and maintenance tasks. Enter Rustinel, a Rust-based endpoint agent designed to unify these efforts into a cohesive codebase.

A single agent across two operating systems

Rustinel adeptly gathers telemetry using ETW on Windows and eBPF on Linux, normalizing the data into a unified model. It evaluates this information against Sigma rules, YARA signatures, and atomic indicators of compromise. Alerts generated by the system are stored in ECS-compatible NDJSON format, facilitating seamless integration into SIEM or log-analysis platforms such as Elastic or Splunk.

The agent’s Windows capabilities encompass a wide range of events, including process creation, image load, network activity, file changes, registry modifications, DNS queries, PowerShell executions, WMI interactions, service management, and scheduled tasks. Meanwhile, Linux support currently covers process, network, file, and DNS telemetry, with plans for broader eBPF support in the future.

Operating in user mode on both platforms, Rustinel can be installed as a service on Windows. For Linux, it requires kernel version 5.8 or newer with BTF support and must run under root or a chosen supervisor. The active response feature is optional and can be configured to operate in a dry-run mode, complete with allowlists to exempt trusted paths.

A user-mode design choice

Unlike most commercial EDR solutions that deploy kernel drivers for enhanced visibility and tamper resistance, Rustinel opts for a different approach. Author Théo Foucher explains that this choice was intentional. “Most commercial EDRs rely on a kernel driver for good reasons: tamper resistance, early visibility, and the ability to observe or block some activity before user-mode components see it. With Rustinel, I made a different design choice: stay as simple, transparent, and stable as possible while still collecting useful host telemetry,” Foucher shared with Help Net Security.

While acknowledging the limitations of a user-mode agent—specifically its inability to match the tamper resistance or deep visibility of a kernel driver—Foucher emphasizes the operational advantages. A bug in a user-mode process is less likely to crash a host compared to one in kernel space, and Rust’s memory safety features inherently mitigate various classes of defects. He also highlights the vulnerabilities inherent in kernel-based protections, noting that techniques like BYOVD and driver-abuse attacks can compromise even the most established commercial EDRs.

Three detection layers

Rustinel employs three detection engines operating in parallel. Sigma is responsible for behavioral matching against the normalized event stream, while YARA scans executables upon process creation. The IOC engine conducts deterministic checks for known hashes, IP ranges, domains, and path patterns.

Foucher explains that this combination was strategically chosen to allow defenders to leverage existing content they already manage. “I arrived at Sigma, YARA, and IOC matching because they solve different parts of the detection problem and they are already widely understood by defenders,” he stated.

However, he candidly addresses the coverage gaps that exist. Memory-only payloads, heavily obfuscated living-off-the-land activities, novel behaviors outside established Sigma rules, and telemetry-tampering techniques are currently beyond the agent’s reach. Additionally, encrypted command-and-control communications over trusted infrastructure may evade IOC matching unless accompanied by behavior that triggers a rule. To enhance coverage against packed and runtime-unpacked payloads, memory scanning with YARA is actively under development.

Rustinel is freely available on GitHub under the Apache 2.0 license.

Must read:
Subscribe to the Help Net Security ad-free monthly newsletter to stay informed on the essential open-source cybersecurity tools. Subscribe here!

Winsage
Rustinel: Open-source endpoint detection for Windows and Linux