Differences

This shows you the differences between two versions of the page.

--- hosts:virtual_machines:nixbuild:nix_build_server [2023/03/10 02:39] – tcmal
+++ hosts:virtual_machines:nixbuild:nix_build_server [2023/08/25 23:11] (current) – removed tcmal
@@ Line 1: / Line 1: @@
-[[meta:autogen:start|]]
-===== Nix Build Server =====
-^ Name | Nix Build Server |
-^ Ports | 22 (tcp) |
-^  | [[https://netbox.tardisproject.uk/ipam/services/37/|View in NetBox.]] |
-[[meta:autogen:end|]]
-This VM is intended to function mainly as a Nix build server. Because of Nix's sandboxing, it's possible to have it upload your source code (and Nix instructions), have the other server build it, and then copy the result back down (if you want to).
-Currently, this is only available to admins, because remote builds require trusted-user and this is equivalent to root. Once [[https://github.com/NixOS/nix/issues/2789|this issue]] is dealt with, we hope to make it more available.
-==== Usage ====
-To try this out, first setup [[howto::ssh_proxy|passwordless SSH]] to %%''nixbuild.internal.tardisproject.uk''%%. **Because of how the Nix daemon runs, you'll need to add this to ''/etc/ssh/ssh_config'' as well (''programs.ssh.extraConfig'' on NixOS).**
-Once you have that, check connectivity with ''%%nix store ping --store ssh-ng://nixbuild.internal.tardisproject.uk%%'' - it should return a version number.
-You can now run nix builds remotely, for example:
-<code>
-$ nix build --impure --expr '(import <nixpkgs> {}).hello' `# Build something from nixpkgs` \
-    --builders 'ssh://nixbuild.internal.tardisproject.uk x86_64-linux  - 10 1 kvm,benchmark' `# Our builder` \
-    --rebuild --max-jobs 0 `# Force it to happen, and remotely`
-</code>
-=== Remote builds vs remote stores ===
-Running this, you might notice two things:
-  - Each dependency is downloaded locally, then uploaded to the build server, possibly resulting in a long time before anything gets built.
-  - We download the end result, and all of its runtime dependencies
-(1) is easily solved by adding ''builders-use-substitutes = true'' to your ''/etc/nix/nix.conf'' (''nix.extraOptions'' on NixOS).
-(2) happens because Nix assumes we want to use the thing we've just built, but if we just want it to be in a binary cache then this might not be the case. To fix this, we have to make Nix use our server as a //remote store// rather than just a remote build server:
-<code>
-$ nix build --impure --expr '(import <nixpkgs> {}).hello' `# Build something from nixpkgs` \
-    --builders 'ssh://nixbuild.internal.tardisproject.uk x86_64-linux  - 10 1 kvm,benchmark' `# Our builder` \
-    --eval-store auto --store 'ssh-ng://nixbuild.internal.tardisproject.uk' `# Note ssh-ng instead of ssh` \
-    --rebuild --max-jobs 0 `# Force it to happen, and remotely`
-</code>
-Later you can use ''nix copy'' or the [[hosts:virtual_machines:nixbuild:nix_binary_cache|binary cache]] to get the build wherever you want.
-[[https://docs.nixbuild.net/remote-builds/|This page]] explains the details and differences in much more detail (such as why ''%%--eval-store auto%%'' is necessary)
-=== Saving details ===
-If you're only going to build Nix stuff using this, you can add ''%%ssh://nixbuild.internal.tardisproject.uk x86_64-linux  - 10 1 kvm,benchmark%%'' to ''/etc/nix/machines''.
-You can also set the environment variable ''%%NIX_REMOTE=ssh-ng://nixbuild.internal.tardisproject.uk%%'' if you'd like to also use it as a store, however you'll still need to pass ''%%--eval-store auto%%'' in order for things to work properly (due to bugs in Nix). Consider putting this in a ''.envrc'' if there's a particular project you never need to build locally (such as NixOS configs).