hackage2nix: Correctly flag broken Haskell packages

[dschrempf]

I added pkgs.haskell.packages.ghc901.haskell-language-server to the available packages in my Nix configuration, and get a build error (see below). Afterwards, I did some research and found that haskell-language-server is not yet available for GHC 9.0.1, so pkgs.haskell.packages.ghc901.haskell-language-server is correctly broken but not labeled as broken.

I am not familiar with the Haskell pipeline of Nixpkgs, but I wanted to ask if it is possible to label broken Haskell packages accordingly? In this case, it would have saved me, and I guess many others, a lot of time.

Please let me know, what you think.

The specific error message:

error: builder for '/nix/store/6wg48vjfrzhpb1k2b6aqvbwnbw2ifml4-ghc-api-compat-8.6.drv' failed with exit code 1;
       last 10 log lines:
       >       - The module 'GHC.Iface.Utils'
       >         is not exported by any suitable package.
       >         It occurs in neither the 'exposed-modules' of this package,
       >         nor any of its 'build-depends' dependencies.
       >       - The module 'GHC.Driver.Packages'
       >         is not exported by any suitable package.
       >         It occurs in neither the 'exposed-modules' of this package,
       >         nor any of its 'build-depends' dependencies.
       >     In the stanza library
       >
       For full logs, run 'nix log /nix/store/6wg48vjfrzhpb1k2b6aqvbwnbw2ifml4-ghc-api-compat-8.6.drv'.
error: 1 dependencies of derivation '/nix/store/5rfswq3vf8dyqkw441y5ywx7qb334zzn-haskell-language-server-1.2.0.0.drv' failed to build
error: 1 dependencies of derivation '/nix/store/8g67r8yzcl2azhz48y8vwmnrbb8arf0j-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ylys98ijpysjii2pxcw4kj28lz9dv845-home-manager-generation.drv' failed to build

[cdepillabout]

@dschrempf We have a "main" Haskell package set with a single version of GHC.

This is what the top-level set haskellPackages corresponds to. This is currently 8.10.4. Here's a couple ways you could confirm this with the repl:

$ nix repl /some/path/to/nixpkgs
nix-repl> ghc
«derivation /nix/store/wmwqap28fh3imv8cqbfb9rzpr9203mir-ghc-8.10.4.drv»
nix-repl> haskellPackages.ghc
«derivation /nix/store/wmwqap28fh3imv8cqbfb9rzpr9203mir-ghc-8.10.4.drv»
nix-repl> haskellPackages.lens.passthru.compiler
«derivation /nix/store/wmwqap28fh3imv8cqbfb9rzpr9203mir-ghc-8.10.4.drv»

This compiler is available at a more specific attribute as well:

nix-repl> haskell.compiler.ghc8104
«derivation /nix/store/wmwqap28fh3imv8cqbfb9rzpr9203mir-ghc-8.10.4.drv»

The haskellPackages package set is available at a more specific attribute as well:

nix-repl> haskellPackages.lens    
«derivation /nix/store/379k1a1y7gx131mqf6dzbcd100hka6fj-lens-4.19.2.drv»
nix-repl> haskell.packages.ghc8104.lens
«derivation /nix/store/379k1a1y7gx131mqf6dzbcd100hka6fj-lens-4.19.2.drv»

We also have other compilers and corresponding package sets available (as you're well aware):

nix-repl> haskell.compiler.ghc901      
«derivation /nix/store/vpw154v4d72i3m1n9lkrvcncy3nv9wxh-ghc-9.0.1.drv»
nix-repl> haskell.packages.ghc901.lens
«derivation /nix/store/c4ghvmy03qs0s6xc4p0xiwszjnsly5xn-lens-5.0.1.drv»
nix-repl> haskell.packages.ghc901.lens.passthru.compiler
«derivation /nix/store/vpw154v4d72i3m1n9lkrvcncy3nv9wxh-ghc-9.0.1.drv»

The Haskell infrastructure in Nixpkgs is mostly concerned with the main package set and main compiler. In this package set, we generally track Stackage (sometimes the latest LTS, sometimes the latest Nightly). If you're interested, you can see some of the work we do in PRs like NixOS/nixpkgs#130424.

In Nixpkgs, we only have very little support for Haskell package sets other than the main one. Right now, the main package set is using ghc-8.10.4, so many packages just don't work on the other compilers. We also don't build packages for many other compilers on Hydra, so you often can't get Haskell packages from the cache with other compilers.

In theory, it would be possible for us to get many packages working on all (or at least multiple) packages sets. The main reason we are not doing this is that we just don't have enough man-power. There are four main members of the Haskell team, but mostly have our hands full keeping just the main package set working.

So that means any time you're taking things from the non-main Haskell package set, you're mostly on your own to get things working.

In the Haskell infrastructure in Nixpkgs, we only have a single definition for all the packages we provide:

https://github.com/NixOS/nixpkgs/blob/8a3580c2f60ab204ea35562eaae8313e0fe0ce12/pkgs/development/haskell-modules/hackage-packages.nix

We currently don't have any automatic/semi-automatic way of marking packages broken for different package sets.

We do have separate configuration files for different package sets (for instance, https://github.com/NixOS/nixpkgs/blob/8a3580c2f60ab204ea35562eaae8313e0fe0ce12/pkgs/development/haskell-modules/configuration-ghc-9.0.x.nix for ghc-9.0.1), so in theory we could manually mark things as broken. However, like I said above, we don't really have the bandwidth to do this for all packages for all compilers. We also get very few PRs for anything outside of the main package set.

[dschrempf]

Thank you very much for your detailed explanation. I was not aware of the fact that other package sets than the main one are only minimally supported. Maybe we should add a paragraph about this to the Haskell section of the Nixpkgs manual, I guess this would be https://haskell4nix.readthedocs.io/?

[cdepillabout]

Maybe we should add a paragraph about this to the Haskell section of the Nixpkgs manual

Yes, this definitely should go in the manual!

Unfortunately, we're sort of in a weird state with the Haskell documentation for Nixpkgs. It is in the process of being moved from https://haskell4nix.readthedocs.io/ back into nixpkgs: NixOS/nixpkgs#121403

[teto]

also maybe broken could be self explanatory "broken for ghcXXX" with NixOS/nixpkgs#109407. As a matter of fact, that PR is very relevant for haskell as that's where you are more likely to hit a broken package in nixpkgs.

[sternenseemann]

My current idea for a next step is to only mark packages in the default set as broken in the way we are doing currently. As a result no packages in the non-default sets would be marked as broken automatically which hopefully clarifies that they don't get the same level of service in that regard.

[stale]

I marked this as stale due to inactivity. → More info

[dschrempf]

I think we are still waiting for the new section about Haskell in the NixOS manual.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/packages-marked-as-broken-should-come-with-an-explanation/19187/9

[dschrempf]

Dito.

[sternenseemann]

Dito.

What are you replying to, exactly?

As for this thread, the relevant detail from my infinitely long discourse post is that I still plan on fixing this problem in hackage2nix, hopefully soon even :)

[dschrempf]

Oh :-), I thought this was a stale bot message. Sorry!

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/creating-a-derivation-that-depends-on-all-non-broken-haskell-packages/30838/9