tests: give the interactive VM enough RAM to not OOM under real use

Root-caused a "logo upload hangs, connection refused in browser
console" report: the qemu-vm module's defaults (1 core, 1024MB) are
too tight for this stack (MySQL + Redis + PHP-FPM + chromium for PDF
generation, each PDF request spawning a full multi-process browser
instance) under real concurrent load, and this VM boots with
panic_on_oom - so exhausting memory doesn't degrade gracefully, the
whole VM goes down and every connection starts refusing.

Reproduced directly: at the 1024MB default, even 2 concurrent PDF
generation requests triggered `redis-server invoked oom-killer` and a
kernel panic. Bumped to 2 cores / 2560MB, which comfortably survives
that same load (~1.4GB still free afterward) - verified via the exact
concurrent-request scenario that crashed the 1024MB config, not just
absence of errors.

virtualisation.cores/memorySize aren't options on a plain nixosSystem
in this nixpkgs (only nixosTest nodes get the qemu-vm module
automatically), so this also adds an explicit import of it.

Documented the underlying resource cost (chromium is a full browser
per PDF request, pm.max_children=8 means up to 8 can run concurrently)
in CLAUDE.md as an operational note for real deployments, not just
this test VM.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-26 16:36:22 -04:00
co-authored by Claude Sonnet 5
parent ccf4d01a34
commit b1fe86e9d5
2 changed files with 24 additions and 2 deletions
+7 -2
View File
@@ -39,7 +39,8 @@ This project uses `just` (see `justfile`) and Nix directly.
and disables `boot.loader.grub` purely to satisfy NixOS's built-in assertions when `nix flake check` evaluates
`config.system.build.toplevel` directly — the qemu-vm module overrides both with a tmpfs root and direct kernel
boot when actually building `config.system.build.vm` (what `just build-vm` uses), so these values are otherwise
unused.
unused. Also raises `virtualisation.cores`/`memorySize` to 2/2560 (qemu-vm's defaults, 1 core/1024MB, are too
tight for real interactive use — see the module architecture notes below on PDF generation's memory cost).
- `nixos-test.nix` — the automated test (`pkgs.testers.nixosTest`, exposed as `checks.<system>.invoiceninja` in
`flake.nix`). Builds its own `pkgs` (with `allowUnfree` baked in — `nixosTest` forbids setting `nixpkgs.config`
on a node once `pkgs` is externally supplied) and boots two nodes from `common.nix` — one with
@@ -126,7 +127,11 @@ expect Nix to report the correct hash on a failed build (or use `nix-prefetch`/`
`127.0.0.1:80`/`[::1]:80` too — otherwise visiting the VM via the IP literal instead of `localhost` matches no
vhost and Caddy silently returns an empty `200 OK` (a blank page with no error anywhere).
- Chromium is required at runtime for PDF generation (`PDF_GENERATOR=snappdf`, `SNAPPDF_CHROMIUM_PATH`) and is
added to the PHP-FPM and data-setup services' `path`.
added to the PHP-FPM and data-setup services' `path`. Each PDF request launches a full chromium instance (not
a lightweight renderer), and PHP-FPM's default `pm.max_children = 8` means up to 8 of these can run
concurrently — on a memory-constrained deployment this can exhaust RAM (confirmed: even a couple of
concurrent PDF requests OOM-killed a 1024MB VM). Size deployments accordingly, or lower
`phpfpm.settings."pm.max_children"`.
### `flake.nix`
+17
View File
@@ -4,6 +4,9 @@
{
imports = [
(modulesPath + "/profiles/qemu-guest.nix")
# Not imported by default for a plain nixosSystem in this nixpkgs (only nixosTest nodes get it
# automatically) - needed explicitly so virtualisation.cores/memorySize below are real options.
(modulesPath + "/virtualisation/qemu-vm.nix")
(import ./common.nix { inherit proxyServer; })
];
@@ -26,4 +29,18 @@
};
services.resolved.enable = true;
# The qemu-vm module's defaults (1 core, 1024MB) are too tight for real interactive use: MySQL +
# Redis + PHP-FPM + chromium (PDF generation, spawned per request - each is a full multi-process
# browser, not a lightweight renderer) running concurrently can exhaust 1GB fast, and this VM
# boots with panic_on_oom - so it's not a graceful slowdown, the whole VM goes down. Confirmed by
# reproducing a "logo upload hangs" report this way: at the 1024MB default, even a couple of
# concurrent PDF-generation requests triggered `redis-server invoked oom-killer` and a kernel
# panic. 2 cores / 2560MB comfortably handles 2 concurrent PDF generations alongside normal
# traffic (verified: ~1.4GB still free afterward); a deliberate flood of many concurrent PDF
# requests can still exhaust even this, since each is a full chromium instance - if that's a
# concern for your deployment, consider capping `phpfpm.settings."pm.max_children"` below its
# default of 8, or provisioning more RAM.
virtualisation.cores = 2;
virtualisation.memorySize = 2560;
}