Enhancing Linux-based firmware fuzzing: from visual-assisted planning to stateful full-system testing

Modern embedded Linux systems implement device functionality through large collections of interdependent processes—often dozens or even hundreds of running executables that interact through various communication mechanisms. This complexity creates significant challenges for security analysis: analysts must navigate firmware images with many interacting binaries, reason about complex inter-process interactions, and design fuzzing strategies without reliable documentation. Conventional fuzzing methods test binaries in isolation and miss vulnerabilities that only emerge through multi-step interactions across processes, while manual inspection simply cannot cover such extensive and interconnected systems. This dissertation tackles firmware security through two contributions that represent an evolution from human-guided analysis to automated techniques. The work started with building visual analytics to help humans understand the problem, then moved to automated approaches once we understood what made the problem hard. The first contribution is FuzzPlanner, a visual analytics system that helps analysts plan firmware fuzzing campaigns through systematic visualization of firmware behavior and structure. FuzzPlanner lets analysts explore temporal interaction patterns, examine binaries, investigate inter-binary relationships, and make targeting decisions based on quantitative metrics. Case studies on commercial router and range extender firmware show that it works well for identifying promising fuzzing targets among hundreds of binaries. However, working with FuzzPlanner on real firmware exposed some fundamental problems. When firmware has many binaries communicating through different IPC mechanisms, manually picking where to inject fuzzer inputs becomes overwhelming. Tracking how data flows and gets used across multiple processes through IPC mechanisms becomes too complex for visual-assisted approaches—which guide testing of individual binaries in isolation—and thus miss bugs that emerge only through inter-binary interactions in full-system contexts. The second contribution is STAFF, which addresses this problem through the discovery of inter-binary dependencies. STAFF uses whole-system dynamic taint analysis to automatically figure out how network inputs propagate through firmware, then uses that dependency information to guide protocol-aware fuzzing that maintains stateful interactions while exploring multi-process compositions. Testing on fifteen real-world firmware images shows that the approach is effective. In total, STAFF identified 26 likely new bugs (9 discovered exclusively by STAFF), and 4 previously disclosed CVEs (2 found only by STAFF). These results demonstrate that STAFF is able to uncover both new and known vulnerabilities that considered baseline approaches miss, highlighting the importance of understanding inter-process dependencies when analyzing multi-process firmware. The shift from FuzzPlanner to STAFF happened because deploying FuzzPlanner on real systems showed us which problems need automation rather than visual guidance. Together, these contributions move firmware security analysis from ad-hoc manual methods toward systematic, scalable approaches. As embedded devices keep growing in critical infrastructure, these techniques provide practical tools for security assessment today and lay the groundwork for more comprehensive firmware security analysis going forward.