Skip to content
zoryn/ maintainer-assistant

Hasher on tmpfs

Running the hasher chroot on tmpfs — Linux's in-memory filesystem — makes builds roughly 7× faster than the same hardware on SSD, and, just as importantly, saves the drive: every hasher build would otherwise write gigabytes of short-lived data (unpacked sources, object files, %install trees, RPMs, apt caches) that gets deleted minutes later. Keeping that churn in RAM is a straight extension of SSD/HDD lifetime for anyone who builds packages regularly. The cost is RAM: the chroot, the unpacked source, every intermediate %install root and any RPMs produced during the build all live in memory until hasher cleanup runs.

The zoryn-specific convenience is the part that makes this practical day-to-day: if hasher_dir is a symlink pointing at a directory that doesn't exist, zoryn creates the target before every build. So a symlink into /tmp "heals itself" after every reboot — no fstab entry, no mkdir in your .profile, nothing to remember.

Why tmpfs

Two reasons, both important:

Speed. A hasher build is small-write heavy: thousands of %install files, hundreds of configure probes, constant apt install/uninstall for BuildRequires. On SSD each of those is bound by IOPS and fsync latency; on tmpfs they are memcpys. The end-to-end speedup on typical ALT packages (shell, python, small C libs) is around 7×. Compile-bound heavyweights (chromium, kernel, LLVM) see a much smaller gain — but even there the %install and cleanup stages alone can shave minutes off every build.

Drive wear. Every hasher build writes several gigabytes and then deletes almost all of them: the chroot is unpacked, used, and torn down; %install trees are built and packed into RPMs; apt caches get rewritten on each config change. On an SSD that's tens of GB of TBW per day for an active maintainer, and on a spinning disk it's hours of head seek. Moving all of it to RAM means the only data that ever touches persistent storage is the finished SRPM/RPM you actually want to keep — drive endurance stops being a function of how often you build.

Trade-offs

  • RAM budget. A sisyphus x86_64 hasher base is ~1.2 GB installed; add sources, object files, the %install tree and result RPMs and a medium package's working set is 3–5 GB. You need that much free RAM on top of the build process's own memory. Memory doesn't reclaim itself between builds — see Reclaiming RAM below.
  • Huge packages won't fit. Chromium, LLVM, webkit, libreoffice — anything that produces tens of GB of intermediate output — will OOM on tmpfs. The clean answer is to keep a second, disk-backed builder specifically for the heavyweights and build those packages on it: you get to keep the tmpfs builder as your default for the other 95% of packages, and send the occasional giant to the disk slot explicitly. See Two-tier setup below.
  • State doesn't survive reboot. /tmp is wiped on boot. Harmless for the chroot itself (hasher rebuilds it from scratch anyway), but any cached apt archive or previously-built RPMs in the hasher repo are gone. zoryn's apt config sync handles the apt side automatically.

Setup

1. Make sure /tmp is tmpfs

On a modern ALT install this is the default — /tmp is mounted from systemd's tmp.mount (or /etc/fstab) as tmpfs with roughly half of RAM as the size limit. Check:

mount | grep ' /tmp '
# tmpfs on /tmp type tmpfs (rw,nosuid,nodev,…,size=…)

If it isn't, enable it:

sudo systemctl enable --now tmp.mount

The default size cap is usually enough. If you know your working set, bump it in /etc/systemd/system/tmp.mount.d/override.conf — Options=size=16G for example.

ALT's convention for per-user private tmp is /tmp/.private/$USER/ — a 0700 directory created automatically on login and wiped on reboot. That's the right parent for the hasher root:

ln -s /tmp/.private/$USER/hasher ~/hasher

$ ls -l ~/hasher
lrwxrwxrwx 1 user user 26 … /home/user/hasher -> /tmp/.private/user/hasher

You don't need to mkdir -p /tmp/.private/$USER/hasher yourself — that's the whole point of the next step.

3. Let zoryn handle post-reboot re-creation

After a reboot /tmp — and with it /tmp/.private/$USER/hasher — is gone, and ~/hasher becomes a dangling symlink.

Do nothing. Before every build, zoryn build / zoryn builder status / any command that needs the hasher dir runs an "ensure" step that:

  1. Notices ~/hasher is a symlink.
  2. Resolves the target (/tmp/.private/$USER/hasher).
  3. mkdir -p on the resolved target.

The same logic runs over SSH for remote builders — shipped in the same round-trip as the busy-check. So any builder whose hasher_dir is a symlink into a tmpfs path is self-healing across reboots on both sides.

4. Configure the builder (optional)

The built-in local builder already uses ~/hasher, so if you only build locally there's nothing to configure — step 2 is the full setup.

For extra local slots or remote builders, keep ~/hasher (or a numbered sibling) as the hasher directory. Example with four parallel local slots on tmpfs:

ln -s /tmp/.private/$USER/hasher1 ~/hasher_1
ln -s /tmp/.private/$USER/hasher2 ~/hasher_2
ln -s /tmp/.private/$USER/hasher3 ~/hasher_3
ln -s /tmp/.private/$USER/hasher4 ~/hasher_4

zoryn builder add --name local --type local --multi-add 4 \
  --hasher-dir='~/hasher_{hasher_number}' \
  --repo /srv/repo/sisyphus -y

The {hasher_number} template expands per slot; each symlink lives under the same tmpfs parent, so all four share /tmp's size cap instead of competing for a fixed per-slot budget.

If you prefer the symlink indirection to go away, point --hasher-dir straight at the tmpfs path — there's no functional difference, just one fewer readlink:

zoryn builder add --name tmpfs --type local \
  --hasher-dir=/tmp/.private/$USER/hasher -y

Two-tier: tmpfs by default, disk for heavyweights

The pattern that works well in practice is two local builders: the default one on tmpfs (fast, every-day), and a second one on the SSD/HDD for the few packages that don't fit in RAM. Picking which to use is then an explicit -b flag on those occasions, not something you have to think about for the other 95% of builds.

Set it up once:

# Default tmpfs builder (the regular `local` — replace its hasher_dir with
# a symlink into tmpfs per step 2 above, nothing else to configure).

# Second local builder backed by the real disk:
mkdir -p ~/hasher_disk
zoryn builder add --name disk --type local --number 2 \
  --hasher-dir=~/hasher_disk \
  --repo /srv/repo/sisyphus -y

Two separate hasher_number values mean both slots can coexist; ~/hasher_disk is a plain directory on your home filesystem, so it's not affected by the /tmp size cap and doesn't eat RAM.

Then, day-to-day:

zoryn build              # tmpfs — fast path for ~every package
zoryn build -b disk      # the occasional chromium / LLVM / webkit

Tip: if you keep a known-heavy list, a tiny shell alias is enough — there's no need to encode it in zoryn config. The builder setup is the only permanent change.

Reclaiming RAM

On tmpfs, "disk space used by the chroot" is RAM held by your process. With hsh --lazy-cleanup (the zoryn default) chroots are kept between builds to warm up apt caches and speed up re-runs; aborted or crashed builds also leave their scratch tree behind. After a session of builds it's normal to see several GB of /tmp still occupied.

Free it with:

zoryn builder clean --all             # every builder, local and remote — the right default
zoryn builder clean --all --dry-run   # preview before wiping
zoryn builder clean local             # a specific one, if you only want to clear that slot

--all covers every configured builder, local and remote; on tmpfs only the local slots reclaim RAM, but cleaning remote chroots in the same pass is harmless and keeps remote /tmp (or SSD) in shape too — there's no reason not to.

Each clean removes the hasher chroot, aux dirs and any leftover tarballs, and re-creates the (now empty) hasher directory — including the dangling-symlink auto-create from step 3, so ~/hasher is ready for the next build immediately.

Make it a habit: run zoryn builder clean --all at the end of a build session, or whenever free -h shows RAM dropping below your next build's working set. There is no automatic eviction — the kernel won't reclaim files under a live mount on its own.

Diagnostics

  • Failed to create hasher_dir. The symlink's target parent doesn't exist and zoryn can't create it. Check ls -ld /tmp/.private/$USER — it must exist and be owned by you. If missing, log out and back in (PAM re-creates it) or sudo install -d -m 0700 -o "$USER" /tmp/.private/$USER.
  • Chroot survives a reboot. You pointed the symlink at a non-tmpfs path (e.g. /home/...). Verify with mount | grep tmpfs and readlink -f ~/hasher.
  • OOM in the middle of a big build. The package's working set exceeds your tmpfs size. Don't fight it — retry the build on a disk-backed builder (zoryn build -b disk from the two-tier setup above). Raising /tmp's size= is a last resort: it competes with everything else under /tmp, and the pattern stops working as soon as the next package is even bigger.

See also