← Back
Haowei Yang Yingjie Cao Swing

Title

IsolatOS: A Fast Kernel TOCTOU Harness for AI Training

Abstract

TOCTOU bugs are a recurring class of kernel race conditions where the kernel reads the same user-space memory location more than once during a trusted operation without ensuring that the data remains stable between reads. Such inconsistencies may lead to denial of service, information disclosure, authorization bypass, or local privilege escalation.

This talk presents IsolatOS, an efficient TOCTOU harness for Linux and RTOS kernel security testing. It combines syscall lifecycle tracking, uaccess event collection, SMAP/PAN state auditing, exception-based cross-boundary access capture, offline consistency analysis, and TOCTOU-specific feedback for AI training.

Unlike traditional harnesses that mainly return crashes, coverage, return codes, or sanitizer output, IsolatOS provides richer signals, such as whether a generated program reached a high-risk entry, produced repeated reads within one lifecycle, widened the gap between validation and use, discovered new caller IPs, or improved candidate confidence. Its goal is to provide a deployable TOCTOU detection workflow for enterprise kernels, automotive Linux platforms, edge gateways, RTOS devices, and third-party driver certification while preserving normal kernel execution semantics.