A few months ago, I converted a server I’m administering to run Guix System. I wouldn’t recommend you do the same at this point (the subject of a long and tragic story yet to be told), but I did learn a lot along the way; in particular, how to get an Elm web app built and up and running, which I want to talk about here.

Aims:

document the state of my Elm packaging work, so others can build on it
show how to bypass Elm’s “highly opinionated” packaging infrastructure
provide an introduction to guix packaging

I’ve split this into a series of articles:

packaging the Elm compiler
developing Guix infrastructure for building Elm apps, and using this to package the Elm reactor, and the full Elm tool
deploying an Elm web application on Guix System

In this first part, we’ll go over the Guix packaging of the Elm compiler proper, a Haskell project. This work has already made it to Guix upstream:

$ guix package --show=elm-compiler
name: elm-compiler
version: 0.19.0
outputs: out
systems: x86_64-linux i686-linux
dependencies: ghc-ansi-terminal@0.8.0.4 ghc-ansi-wl-pprint@0.6.8.2 ghc-edit-distance@0.2.2.1
+ ghc-file-embed@0.0.10.1 ghc-http-client-tls@0.3.5.3 ghc-http-client@0.5.13.1 ghc-http-types@0.12.3
+ ghc-http@4000.3.12 ghc-language-glsl@0.3.0 ghc-logict@0.6.0.2 ghc-network@2.6.3.6
+ ghc-raw-strings-qq@1.1 ghc-scientific@0.3.6.2 ghc-sha@1.6.4.4 ghc-snap-core@1.0.3.2
+ ghc-snap-server@1.1.0.0 ghc-unordered-containers@0.2.9.0 ghc-utf8-string@1.0.1.1 ghc-vector@0.12.0.1
+ ghc-zip-archive@0.3.3
location: gnu/packages/elm.scm:40:2
homepage: https://elm-lang.org
license: Modified BSD
synopsis: The `elm' command line interface, without {elm reactor  
description: This includes commands like `elm make', `elm repl', and many others for helping make Elm
+ developers happy and productive.

An overview of Elm tooling

The elm tool (sources at github.com/elm/compiler) is a command line program that provides a variety of subcommands related to building Elm applications. It’s written in Haskell, and our first step will be to build this tool with Guix (or enough of this tool to be able to run elm make successfully). The sources are at [github.com/elm

It will be useful below to have a rough idea of how the elm tool works. Elm projects come in two forms: packages and applications. Packages are distributed via package.elm-lang.org, and can be used as libraries from other packages and applications. Applications on the other hand are the “end product”, compiling to Javascript to be served to a web browser.

Both such projects are defined by a file elm.json. The elm tool is then run from a directory containing such a file. It provides a couple of commands; the relevant ones for us are:

elm make: Download package dependencies to ~/.elm from package.elm-lang.org. Compile dependencies within ~/.elm to an internal format. Compile Elm modules within ./elm-stuff to the internal format. If this is an application, compile the internal format to Javascript.
elm reactor: This serves a web app to interactively debug your Elm application. It is itself implemented as an Elm application.

We’ll face issues with both of these: elm make’s all-in-one design is great for ease of use if you’re doing things the way they’re meant to be done. But when building in a sandboxed environment without internet access, we’ll need to trick it to believe it actually built its package database by talking to package.elm-lang.org. Then it’ll grudgingly fall back to an offline mode and work for us.

Concretely, we’ll construct a package database in ~/.elm/0.19.0/package/ by unpacking dependency sources there and generating a versions.dat database.

The issue with elm reactor isn’t with its use, but with its build: Since it is in Elm application itself, we need to have elm make (or its logic) available to build it. The clean way to do this is with a two-stage build, where we first build the Elm compiler, then use it to build the Elm reactor. Which is all good, except the upstream build solves this is a different and rather hacky way: It calls out to itself via Template Haskell, generating the web application Javascript during the build of the elm tool.

This one we’ll address by tearing the build apart: In a first stage, we’ll patch out the reactor and build a version of the elm tool that primarily supports elm make. In a second stage we’ll re-enable the reactor, providing Javascript files built using the first stage elm make tool.

Packaging the elm compiler (minus reactor)

The steps we’ll follow to package elm make are:

make sure the Haskell project builds with stack and the right LTS version
generate a package definition using guix import hackage -s < elm.cabal
import unpackaged Haskell dependencies
replace the source field to refer to the github release
apply patches

We’ll go through this process explicitly below, not least to share the pain inherent in Guix packaging work.

The first step is easy enough. I found things built fine with a GHC 8.4 stackage release after applying the patch elm-compiler-relax-glsl-bound.patch to relax version bounds.

guix import hackage converts a cabal file into an expression of the form

(package
  (name "ghc-elm")
  (version "0.19.0")
  (source
    (origin
      (method url-fetch)
      (uri (string-append
             "https://hackage.haskell.org/package/elm/elm-"
             version
             ".tar.gz"))
      (sha256
        (base32 "failed to download tar archive"))))
  (build-system haskell-build-system)
  (inputs
   `(("ghc-ansi-terminal" ,ghc-ansi-terminal)
     ...
     ("ghc-zip-archive" ,ghc-zip-archive)))
  (home-page "https://elm-lang.org")
  (synopsis "The `elm` command line interface")
  (description
   "This includes commands like `elm make`, `elm repl`, and many others
for helping make Elm developers happy and productive.")
  (license bsd-3))

This is a scheme record, with a number of named fields. The important ones are

source: This declares the package source archive.
inputs: This declares a list of dependencies. Typically, those are other packages.
build-system: This declares the build system that’s used. It’s pretty much up to the build system to define what the source and input fields actually are.

The dependencies in inputs are given as an association list of pairs ("package-name" ,package), where the name appears to be mostly irrelevant, while the package itself is a package record like we’re defining.

Let’s try to build this package definition: We’ll write it to a file called elm.scm, and call guix build:

$ guix build -f elm.scm
ice-9/eval.scm:223:20: In procedure proc:
error: package: unbound variable
hint: Did you forget `(use-modules (guix packages))'?

Indeed, putting the hinted line at the top of the file helps.

The Guile module (guix packages) is defined in guix/packages.scm within the guix git repository. Generally, the guix tooling itself and packaging infrastructure is defined within guix/, while the packages themselves are defined under gnu/.

The next failure relates to the broken source field:

$ guix build -f elm.scm
guix build: error: exception thrown: #<condition &invalid-base32-character [character: #\e string: "failed to download tar archive"] 29c9f00>

Unsurprisingly, "failed to download tar archive" isn’t a valid base32 hash. This happened because elm isn’t actually published on Hackage, so guix import failed to download the source archive and compute a hash. To fix this, we edit the file to refer to a correct URI for the elm compiler sources, put in some arbitray valid hash copied from another package definition, and eventually fix the hash to the correct version.

$ guix build -f elm.scm
ice-9/eval.scm:223:20: In procedure proc:
error: url-fetch: unbound variable
hint: Did you forget `(use-modules (guix build download))'?

Well, not “forget” as such, but indeed: adding the import gets us further.

$ guix build -f elm.scm
ice-9/eval.scm:223:20: In procedure proc:
error: haskell-build-system: unbound variable
hint: Did you forget a `use-modules' form?

Yes, we also need (use-modules (guix build-system haskell)).

$ guix build -f elm.scm
ice-9/eval.scm:223:20: In procedure proc:
error: bsd-3: unbound variable
hint: Did you forget `(use-modules (guix licenses))'?

Oh no. We did.

$ guix build -f elm.scm
/home/rob/blog-test/elm.scm:17:3: In procedure inputs:
error: ghc-ansi-terminal: unbound variable
hint: Did you forget a `use-modules' form?

Oh, our dependencies aren’t yet imported. We can find this one:

$ guix search ghc-ansi-terminal
name: ghc-ansi-terminal
version: 0.8.0.4
outputs: out
systems: x86_64-linux i686-linux
dependencies: ghc-colour@2.3.4
location: gnu/packages/haskell-xyz.scm:288:2
homepage: https://github.com/feuerbach/ansi-terminal
license: Modified BSD
synopsis: ANSI terminal support for Haskell  
description: This package provides ANSI terminal support for Haskell.  It allows cursor movement, screen clearing, color
+ output showing or hiding the cursor, and changing the title.
relevance: 20

It’s defined in the module (gnu packages haskell-xyz), so we’ll add an import for that. There’s a few more similar missing imports, which we fix by also importing (gnu packages haskell-check) and (gnu packages haskell-web).

The Elm dependencies have by now all been packaged, but usually at this point you’d expect to find a few libraries that are still missing. Then we’d have to package those first. guix import has a recursive mode that can help with this.

$ guix build -f elm.scm
Backtrace:
In guix/store.scm:
   623:10 19 (call-with-store _)
In guix/scripts/build.scm:
   914:26 18 (_ #<store-connection 256.99 e32ae0>)
In ice-9/boot-9.scm:
    829:9 17 (catch _ _ #<procedure 1bf1800 at ice-9/boot-9.scm:104…> …)
In guix/ui.scm:
    415:6 16 (_)
In guix/scripts/build.scm:
    879:5 15 (_)
In srfi/srfi-1.scm:
   679:15 14 (append-map _ _ . _)
   592:17 13 (map1 ("x86_64-linux"))
   679:15 12 (append-map _ _ . _)
   592:17 11 (map1 (#<package elm@0.19.0 /home/rob/blog-test/elm.sc…>))
In guix/scripts/build.scm:
   840:18 10 (_ _)
In guix/packages.scm:
   936:16  9 (cache! #<weak-table 358/443> #<package elm@0.19.0 /ho…> …)
  1255:22  8 (thunk)
  1188:25  7 (bag->derivation #<store-connection 256.99 e32ae0> #<<…> …)
In srfi/srfi-1.scm:
   592:29  6 (map1 (("haskell" #<package ghc@8.4.3 gnu/packages…>) …))
   592:17  5 (map1 (("source" #<origin "https://github.com/elm/…>) …))
In ice-9/boot-9.scm:
    829:9  4 (catch srfi-34 #<procedure 3743870 at guix/packages.sc…> …)
In guix/packages.scm:
  1003:18  3 (_)
In guix/store.scm:
  1803:24  2 (run-with-store #<store-connection 256.99 e32ae0> _ # _ …)
  1673:13  1 (_ _)
In guix/build/download.scm:
    741:0  0 (url-fetch _ _ #:timeout _ #:verify-certificate? _ # _ # …)

guix/build/download.scm:741:0: In procedure url-fetch:
Invalid keyword: #vu8(73 77 243 55 36 34 67 7 214 226 180 208 179 66 68 140 201 39 144 20 131 56 78 228 248 207 238 44 179 142 153 60)

Oops. I’ve never seen this error before going through these steps again while writing the article. It turns out that the helpful hint to import (guix build download) was misleading: We want to import the url-fetch from (guix download) instead.

$ guix build -f elm.scm
[...]
downloading from https://github.com/elm/compiler/archive/0.19.0.tar.gz...
 0.19.0.tar.gz                                                                               608KiB/s 00:01 | 439KiB transferred
sha256 hash mismatch for /gnu/store/pcxygjzvmals9yramliz2bzznz1lh9i1-elm-0.19.0.tar.gz:
  expected hash: 1111111111111111111111111111111111111111111111111111
  actual hash:   0g4risrjrvngz3j4wf432j82gjcc8i1b7l5lwbb0fhr24hvz6ka9
hash mismatch for store item '/gnu/store/pcxygjzvmals9yramliz2bzznz1lh9i1-elm-0.19.0.tar.gz'
[...]

Progress! We copy and paste the actual hash into elm.scm, and we’re actually trying to build. Though as expected, we run into trouble in the Haskell configure phase:

$ guix build -f elm.scm
[...]
starting phase `configure'
running "runhaskell Setup.hs" with command "configure" [...]
[...]
Configuring elm-0.19.0...
Setup.hs: Encountered missing dependencies:
language-glsl >=0.0.2 && <0.3
[...]

This is where we need to apply the patch elm-compiler-relax-glsl-bound.patch. We store it in the current directory, and add it to the source definition:

  (source
    (origin
      (method url-fetch)
      (file-name "elm-0.19.0.tar.gz")
      (uri "https://github.com/elm/compiler/archive/0.19.0.tar.gz")
      (sha256
        (base32 "0g4risrjrvngz3j4wf432j82gjcc8i1b7l5lwbb0fhr24hvz6ka9"))
      (patches
       `("elm-compiler-relax-glsl-bound.patch"))))

$ guix build -f elm.scm
[...]
ui/terminal/src/Develop/StaticFiles.hs:92:3: error:
    • Exception when trying to run compile-time code:
        /homeless-shelter: createDirectory: permission denied (Permission denied)
      Code: bsToExp =<< runIO Build.compile
    • In the untyped splice: $(bsToExp =<< runIO Build.compile)
   |
92 |   $(bsToExp =<< runIO Build.compile)
   |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[...]

Finally, we run into trouble with the convoluted Elm build: It’s trying to run the compiler itself while building, which tries to write to $HOME, which is read-only inside the guix sandbox. (We could get around this by providing a writable home directory, but then we’d fail soon after when it tries to access the network.)

We fix this by adding another patch, elm-compiler-disable-reactor.patch.

$ guix build -f elm.scm
[...]
successfully built /gnu/store/8xn1irs377mk6n17bcrxr293qpzr6224-ghc-elm-0.19.0.drv
/gnu/store/542im7drgr0p37ymaiqkh31k5ff4ghaj-ghc-elm-0.19.0
$ /gnu/store/542im7drgr0p37ymaiqkh31k5ff4ghaj-ghc-elm-0.19.0/bin/elm
Hi, thank you for trying out Elm 0.19.0. I, Evan, hope you like it!
[...]

Success! Wrapping this up as a package for Guix upstream requires a little bit more work:

Bind the package definition to a name, and place it in a module within the (gnu packages) hierarchy.
Move the patches to gnu/packages/patches/, and add the files to the list in gnu/local.mk.
Rewrite the guix import-generated synopsis and description to fit the Guix style and markup format.

You can look at the complete patch in the guix repository.

In the next article, we’ll see how to use the Elm compiler within Guix to package Elm applications.

Guix and Elm, part 1: Packaging the Elm compiler

An overview of Elm tooling

Packaging the elm compiler (minus reactor)