TARDIS Project

​

This VM is intended to function mainly as a Nix build server. Because of Nix's sandboxing, it's possible to have it upload your source code (and Nix instructions), have the other server build it, and then copy the result back down (if you want to).

Currently, this is only available to admins, because remote builds require trusted-user and this is equivalent to root. Once this issue is dealt with, we hope to make it more available.

To try this out, first setup passwordless SSH to ''nixbuild.internal.tardisproject.uk''. Because of how the Nix daemon runs, you'll need to add this to /etc/ssh/ssh_config as well (programs.ssh.extraConfig on NixOS).

Once you have that, check connectivity with nix store ping --store ssh-ng://nixbuild.internal.tardisproject.uk - it should return a version number.

You can now run nix builds remotely, for example:

$ nix build --impure --expr '(import <nixpkgs> {}).hello' `# Build something from nixpkgs` \
    --builders 'ssh://nixbuild.internal.tardisproject.uk x86_64-linux  - 10 1 kvm,benchmark' `# Our builder` \
    --rebuild --max-jobs 0 `# Force it to happen, and remotely`

Running this, you might notice two things:

1. Each dependency is downloaded locally, then uploaded to the build server, possibly resulting in a long time before anything gets built.
2. We download the end result, and all of its runtime dependencies

(1) is easily solved by adding builders-use-substitutes = true to your /etc/nix/nix.conf (nix.extraOptions on NixOS).

(2) happens because Nix assumes we want to use the thing we've just built, but if we just want it to be in a binary cache then this might not be the case. To fix this, we have to make Nix use our server as a remote store rather than just a remote build server:

$ nix build --impure --expr '(import <nixpkgs> {}).hello' `# Build something from nixpkgs` \
    --builders 'ssh://nixbuild.internal.tardisproject.uk x86_64-linux  - 10 1 kvm,benchmark' `# Our builder` \
    --eval-store auto --store 'ssh-ng://nixbuild.internal.tardisproject.uk' `# Note ssh-ng instead of ssh` \
    --rebuild --max-jobs 0 `# Force it to happen, and remotely`

Later you can use nix copy or the binary cache to get the build wherever you want.

This page explains the details and differences in much more detail (such as why --eval-store auto is necessary)

If you're only going to build Nix stuff using this, you can add ssh://nixbuild.internal.tardisproject.uk x86_64-linux - 10 1 kvm,benchmark to /etc/nix/machines.

You can also set the environment variable NIX_REMOTE=ssh-ng://nixbuild.internal.tardisproject.uk if you'd like to also use it as a store, however you'll still need to pass --eval-store auto in order for things to work properly (due to bugs in Nix). Consider putting this in a .envrc if there's a particular project you never need to build locally (such as NixOS configs).