Nix Build Server
Name | Nix Build Server |
---|---|
Ports | 22 (tcp) |
View in NetBox. |
This VM is intended to function mainly as a Nix build server. Because of Nix's sandboxing, it's possible to have it upload your source code (and Nix instructions), have the other server build it, and then copy the result back down (if you want to).
Currently, this is only available to admins, because remote builds require trusted-user and this is equivalent to root. Once this issue is dealt with, we hope to make it more available.
Usage
To try this out, first setup passwordless SSH to ''nixbuild.internal.tardisproject.uk''. Because of how the Nix daemon runs, you'll need to add this to /etc/ssh/ssh_config
as well (programs.ssh.extraConfig
on NixOS).
Once you have that, check connectivity with nix store ping --store ssh-ng://nixbuild.internal.tardisproject.uk
- it should return a version number.
You can now run nix builds remotely, for example:
$ nix build --impure --expr '(import <nixpkgs> {}).hello' `# Build something from nixpkgs` \ --builders 'ssh://nixbuild.internal.tardisproject.uk x86_64-linux - 10 1 kvm,benchmark' `# Our builder` \ --rebuild --max-jobs 0 `# Force it to happen, and remotely`
Remote builds vs remote stores
Running this, you might notice two things:
- Each dependency is downloaded locally, then uploaded to the build server, possibly resulting in a long time before anything gets built.
- We download the end result, and all of its runtime dependencies
(1) is easily solved by adding builders-use-substitutes = true
to your /etc/nix/nix.conf
(nix.extraOptions
on NixOS).
(2) happens because Nix assumes we want to use the thing we've just built, but if we just want it to be in a binary cache then this might not be the case. To fix this, we have to make Nix use our server as a remote store rather than just a remote build server:
$ nix build --impure --expr '(import <nixpkgs> {}).hello' `# Build something from nixpkgs` \ --builders 'ssh://nixbuild.internal.tardisproject.uk x86_64-linux - 10 1 kvm,benchmark' `# Our builder` \ --eval-store auto --store 'ssh-ng://nixbuild.internal.tardisproject.uk' `# Note ssh-ng instead of ssh` \ --rebuild --max-jobs 0 `# Force it to happen, and remotely`
Later you can use nix copy
or the binary cache to get the build wherever you want.
This page explains the details and differences in much more detail (such as why --eval-store auto
is necessary)
Saving details
If you're only going to build Nix stuff using this, you can add ssh://nixbuild.internal.tardisproject.uk x86_64-linux - 10 1 kvm,benchmark
to /etc/nix/machines
.
You can also set the environment variable NIX_REMOTE=ssh-ng://nixbuild.internal.tardisproject.uk
if you'd like to also use it as a store, however you'll still need to pass --eval-store auto
in order for things to work properly (due to bugs in Nix). Consider putting this in a .envrc
if there's a particular project you never need to build locally (such as NixOS configs).