notes/forensic-residue-hidden-nixos.md 6 min read

Running forensic tools against my own hidden NixOS environment

I ran disk forensic tooling against the hidden NixOS environment from my NAILS thesis and wrote down what leaked and what did not.


Why I pointed the tools at my own machine

I built NAILS as the artifact for my MSc thesis. It is a dual-environment setup for NixOS. One environment is a visible decoy. The other is a hidden environment that lives on encrypted storage and is layered over the decoy with OverlayFS. The point of the design is to keep hidden activity off the persistent decoy disk. A design goal is easy to state and hard to trust. So I ran forensic tooling against my own system and wrote down what I found.

This post is a plain summary of that evaluation. The numbers and the labels come from the thesis. Where the thesis hedges, I hedge here too.

The dual-environment design in one paragraph

The decoy is an ordinary NixOS installation. It uses an impermanent root, which means the root filesystem is a tmpfs and only an explicit set of paths under /persist survives a reboot. The hidden environment is a full NixOS configuration stored on a mounted hidden root, in the tested case a VeraCrypt container. During activation NAILS mounts OverlayFS layers over selected paths such as /home, /etc, /nix, and /var. The lower layer is the visible filesystem. The upper and work directories sit on the hidden root. Writes during a hidden session land on the hidden root, not on the decoy. NAILS does not unlock the container. It assumes the hidden root is already mounted.

The test setup

I ran a VirtualBox VM based on my amnesic reference base, NAILS OS. The VM had a tmpfs root, a persistent /persist, no swap, and a VeraCrypt container as the hidden backend. I captured four disk states: baseline before any hidden activity, active during a hidden session, post-standard-deactivation, and post-emergency-deactivation.

The two deactivation paths differ. Standard deactivation returns to the decoy and reboots at the end to wipe everything not explicitly persisted. Emergency deactivation removes the overlays without a reboot, for the case where a normal reboot is not safe or not available. In the forensic run I still rebooted manually before capturing the post-emergency state, which is why both post states are compared after a reboot. I ran the workflow once per path and compared each result against the same baseline.

The hidden session was a scripted workload meant to approximate a small journalist or activist session. It touched browsers (Brave, Firefox, tor-browser), a messaging app (signal-desktop), a password manager (keepassxc), document and image tools (libreoffice, GIMP), and a wallet marker (electrum). It also ran cowsay through nix-shell as an artificial Nix package canary. The canary is not a real task. It only tests whether temporary Nix package use leaves a visible trace after deactivation.

What I checked, and with what

The disk analysis used The Sleuth Kit. TSK fls -r gave recursive file listings, including deleted entries still present in metadata. I took those listings against /persist and /nix on the visible NAILS OS installation, not against the hidden VeraCrypt container. On top of that I used baseline diffs, recovered-file extraction, raw disk carving, raw-image string scans, a separate EFI and GPT plaintext scan, and ext4 journal plaintext extraction. The journal was checked on its own because journal residue can outlive ordinary file deletion.

The targeted artifact checks covered the classes that usually give a hidden volume away: shell history, recently-used file metadata, browser profiles, SSH state, dconf, thumbnails, system logs, service-path canaries, and mount records.

I fixed the interpretation rules before looking at results, to avoid reading meaning into every keyword hit. An indicator was baseline-equivalent when the same class of entry was already in the baseline or when its count and context did not increase after hidden use. An indicator was newly visible when it appeared only after deactivation or grew in a way I could attribute to the hidden session. An indicator was not identified when the selected checks simply did not recover it. Baseline-equivalent does not mean absent from disk. Not identified does not mean unrecoverable by any tool.

What did not show up

After both deactivation modes, the targeted user-facing checks did not recover the artifacts I looked for. Shell history was absent. Recently-used metadata was absent. Brave and Firefox state was absent. SSH state was absent. Thumbnail metadata was absent. The visible text logs were clean.

The package, mount, raw-string, and canary indicators stayed baseline-equivalent rather than becoming newly visible. Hidden-package name hits sat at 3,752 lines and hidden-store paths at 464 across baseline and both post states. The Nix database check reported 20 matches in all three states. Bulk hits for /mnt/hidden were 0 everywhere. Overlay strings were 1,725 and mount-option hits were 13 across all three states. The signal-desktop and cowsay tokens did appear, but only inside Nix package-definition files that were already on the visible system, not in new installed program files. keepassxc and tor-browser were baseline packages to begin with.

A few flat counts are worth naming. TSK fls -r returned 720,370 lines for /persist and 719,962 lines for /nix in the baseline and both post states. Recovered-file counts held at 3,821 in all three states. EFI and GPT string hits were 0 throughout. The ext4 journal string hits went from 167 at baseline to 113 in both post states, a decrease rather than a new signal.

What did show up

One class was not clean, and I report it as such. Both post states had 832 diff lines under /persist. The rewrites sit under /persist/var/lib/nixos and cover declarative-groups, gid-map, declarative-users, uid-map, and auto-subuid-map. These are the bookkeeping files the current NixOS activation path uses for users, groups, and UID and GID maps. I do not treat this as hidden-session content leakage. I do treat it as an observable effect of running the activation path. Standard and emergency deactivation produced the same diff count and the same rewritten files in this run. That is a result for this workload and this VM, not a general claim that the two paths are equivalent.

What this does not prove

The honest part. This is scoped qualitative work, not a proof of undetectability.

I only ran external disk forensics against the VeraCrypt backend. LUKS with ext4 and plain directory roots are exercised by automated tests, which catch regressions but do not establish forensic equivalence across backends. I did not analyze volatile memory at all. Encryption keys, command state, and application data can persist in RAM while the hidden environment is active, and that surface is untouched here. The run was a single VM on a single host, so firmware logs, vendor journaling, and device-mapper edge cases on real hardware may behave differently. It was one user role with one fixed workload. A different application mix or a longer session may surface artifacts this script did not.

There is also the reading itself. I set the interpretation rules in advance, but I am the single author who classified the output, and no second examiner repeated the work. The selected tools may have missed indicators that were present. A more aggressive forensic procedure might recover things these checks did not.

And there is a limit the design does not try to hide. The evaluation measures whether hidden-session content leaks. It does not disguise the fact that the machine is an impermanence-based NixOS with encrypted storage and no swap. The /persist/var/lib/nixos rewrites stay visible on the decoy, and the forensic tooling I used leaves its own presence on the analysis host. The setup reduces specific persistent artifacts. It does not make the machine look like it was never configured for this.

References

NAILS is research software and still alpha. If you want the code or the full evaluation, both are public.