Nix & NixOS: Packaging applications for Nix

If you have ever had to debug a configuration problem and wanted to quickly share a command’s output or a file with someone, you’ll know how valuable pastebins are. While there are a lot of options, most of them require a working browser to copy-paste text and create a paste.

Fiche is the software powering termbin.com, a pastebin service you can use from the command line. It has minimal dependencies and can be even used on a bare-bones TTY without root permissions to install custom clients.

It accepts input on a given port and returns a URL to the paste, that’s it. Netcat (nc) should be installed on most Linux systems and can be used to send pipe output of file to the service right at the CLI.

$ echo "(☞゚ヮ゚)☞" | nc termbin.com 9999         # Send text
https://termbin.com/4twt

$ cat file.txt | nc termbin.com 9999            # Send file
http://termbin.com/x7dw

$ ip addr | nc termbin.com 9999                 # Send a command's output
http://termbin.com/k9sh

Fiche can be also hosted on your own server, since the code is open source. At the time of this writing, it wasn’t packaged for the NixOS distribution, so I decided to do it myself. There is documentation available on how to do that, but I had trouble finding a simple yet complete example.

The following describes the all steps necessary to getting a new package included in nixpgks.

Get nixpgks

The Nix packages collection (nixpkgs) is hosted on GitHub, it includes all packages included in NixOS. Start by creating and cloning a fork, like you would for any other open source project you want to contribute to.

Add a branch to track the upstream master

There is a lot of activity on the nixpkgs repository. It’s a good idea to add a second remote to your cloned repository and a branch to track changes on the upstream’s master branch.

$ git remote add upstream https://github.com/NixOS/nixpkgs.git
$ git fetch upstream
$ git checkout -b upstream-master upstream/master
$ git pull

You will probably want to create a feature branch as well, to do your work

$ git checkout -b pinpox-init-fiche

Package definition

Start by deciding where your package should go. Inside the repository’s pkgs directory are subdirectories for different classes of applications.

For fiche, I decided the server (/pkgs/servers/fiche) category would be the best-fitting. Create a directory accordingly.

At this point it’s a good idea to look at the application you want to package in more detail if you haven’t done so already and determine what dependencies it has, where it’s hosted and how it is normally (i.e. manually) installed.

Similar packages

If you’re not packaging something too obscure, chances are good you’ll be able to find similar applications already packaged in nixpkgs. Try to find something in the same programming language or framework, possibly something with the same hosting service.

For fiche, I went with the package definition of the [bspwm window manager(https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/window-managers/bspwm/default.nix) as it’s hosted on GitHub like fiche and also written in C.

Start by copying its contents to a file named default.nix in your new directory.

Define the source

The source from which the application should be downloaded is surely something you will want to change. For GitHub there is a helper function fetchFromGitHub that makes getting a release easy. Include the helper at the top of your default.nix. You might have also noted the stdenv in the inputs, which provides basic stuff you will almost always want to include.

You will also have to define the two attributes pname (The name of the package) and version (The, you guessed it, version).

# /pkgs/servers/fiche/default.nix
{ stdenv, fetchFromGitHub }:

stdenv.mkDerivation rec {
  pname = "fiche";
  version = "0.9.1";

  src = fetchFromGitHub {
    owner = "solusipse";
    repo = "fiche";
    rev = version;
    sha256 = "1102r39xw17kip7mjp987jy8na333gw9vxv31f7v8q05cr7d7kfb";
  };

Most parameters in the src = { } definition should be self-explanatory except the sha256. I’m sure there are more “professional” ways to get the correct hash of the version, but a simple way is to just write something random in there, try to build and copy the correct hash from the error message.

It will fail with the hash in the correct format, so you can just copy-paste it from there and build again.

Install Phase

The install phase defines what has to be done to install the application. In case of fiche, that means compiling the C code and putting the binary to the correct location with the correct permissions. Consulting the original Makefile of the application, shows that only one command is needed. You can and should use Nix’s $out variable, that will be the output path in the Nix store. Don’t forget to create subfolders needed in it.

  installPhase = ''
    mkdir -p $out/bin
    install -m 0755 fiche $out/bin
  '';

Package Metadata

The new package should also have some basic metadata containing descriptions, license and upstream links as well as supported platforms and a maintainer of the package (probably you).

The copied template already contains everything required, just change the values accordingly.

  meta = with stdenv.lib; {
    description = "Command line pastebin for sharing terminal output";
    longDescription = ''
      Fiche is a command line pastebin server for sharing terminal output.
      It can be used without any graphical tools from a TTY and has minimal requirements.
      A live instance can be found at https://termbin.com.

      Example usage:
      echo just testing! | nc termbin.com 9999
    '';

    homepage = "https://github.com/solusipse/fiche";
    changelog = "https://github.com/solusipse/fiche/releases/tag/${version}";
    license = licenses.mit;
    maintainers = [ maintainers.pinpox ];
    platforms = platforms.all;
  };
}

Make your package known

To actually tell Nix about your new package, you will have to include it in /pkgs/top-level/all-packages.nix. Add a definition like the ones already present with the path and name of your package.

# /pkgs/top-level/all-packages.nix
fiche = callPackage ../servers/fiche { };

Add yourself as a package maintainer

If you try to build your package at this point it will fail, if it is your first package because you specified a maintainer that is not known to Nix. All maintainers are specified in /maintainers/maintainer-list.nix. The list is alphabetical, so find the correct place to put your nickname and add an entry.

It is encouraged to use your GitHub username as nickname in that file. You will also have to specify the githubId of your profile.

The easiest way to find it is to use GitHub’s API. Either use your browser or curl to GET it.

curl https://api.github.com/users/pinpox
{
  "login": "pinpox",
  "id": 1719781,
  "node_id": "MDQ6VXNlcjE3MTk3ODE=",
  "avatar_url": "https://avatars1.githubusercontent.com/u/1719781?v=4",
  "gravatar_id": "",
  "url": "https://api.github.com/users/pinpox",
  "html_url": "https://github.com/pinpox",
  
  ...
}

If you use GPG to sign off your commits (which you should), you can also include your key’s ID and fingerprint. Get it using gpg -K and finally put everything together.

# +++ b/maintainers/maintainer-list.nix
pinpox = {
  email = "[email protected]";
  github = "pinpox";
  githubId = 1719781;
  name = "Pablo Ovelleiro Corral";
  keys = [{
    longkeyid = "sa4096/0x823A6154426408D3";
    fingerprint = "D03B 218C AE77 1F77 D7F9  20D9 823A 6154 4264 08D3";
  }];
};

Test your package locally

At this point you are mostly done with the package. If everything is set up correctly, proceed to installing it locally to test if it works.

Create a temporary directory somewhere to build the package, it will create some files while building. To use your clone of the nixpkgs repository when building the package, you can set the $NIXPKGS environment variable to your working repository.

More information on how to build and debug new packages can be found here.

$ mkdir -p ~/tmpdev && cd ~/tmpdev
$ export NIXPKGS=~/path/to/nixpkgs
$ ls $NIXPKGS

With the above setup, install the package as you normally would. After that you should be able to run the binary, if you are packaging a executable program.

$ nix-env -f $NIXPKGS -iA fiche

$ fiche
[Fiche][STATUS] Starting fiche on Fri Oct 23 18:08:13 2020...
[Fiche][STATUS] Domain set to: http://example.com.
[Fiche][STATUS] Server started listening on port: 9999.
============================================================
[Fiche][STATUS] Fri Oct 23 18:08:37 2020
[Fiche][STATUS] Incoming connection from: 127.0.0.1 (localhost).
[Fiche][STATUS] Received 26461 bytes, saved to: k9sh.
============================================================

If everything works: Great! If not, make changes to your package and repeat. To further test the fiche application, I submitted a paste to localhost in a second shell. You can see logging on the server terminal and get a URL back as expected on the client.

$ cat gpg-key | nc localhost 9999
http://example.com/k9sh

Submit your work

Once you are happy with the result, be sure to submit your precious contribution to the official nixpkgs repository, so others can profit from it.

Squash your branch

If you have been using git along the way, your branch will already have multiple commits. While this is totally fine, for the pull request to nixpkgs it seems to be common convention to squash your branch to just one commit, named like the pull request itself: <package name>: init at <version>.

$ git reset $(git merge-base master pinpox-init-fiche)
$ git add ... # Add all the files you changed or created
$ git commit -m "fiche: init at 0.9.1"

Create a pull request

Push your changes to your fork then open a pull request via the GitHub page.

$ git push -f -u origin pinpox-init-fiche

You might have not gotten everything right, so you might get suggestions for changes. In any case wait for feedback from reviewers. Your pull request will have to get reviewed before it can be merged. The repository is set up with all sorts of continuous integration as well, so make sure all tests pass.

If your pull request is not getting seen after a few days, It might be a good idea to ask in the #nixos IRC channel on freenode. Be aware though, that the reviewers are quite overloaded with pull requests, so be patient.

The fiche package

If you want to see my complete code, have a look at the pull request I submitted for fiche. I had to make a few changes and it is currently waiting to be merged in the next few days.

Also, if you run into problems the community has always been very welcoming of new contributors in my experience.