I will be delegating responsibility and authority over managing the NixPkgs soft fork to the core team here. The goals during this phase are:
@sig_core should now organize to best collaborate in order to achieve these goals.
I probably sound like a broken record, but my proposal:
Pretty much we can get the best of both world. Stay up to date with nixpkgs while also being able to dramatically change the format of package organization.
Yeah I’ve been thinking about this and it is sounding like a pretty reasonable idea. I’d like for us to start with getting used to keeping our link to upstream up to date and then we can evaluate another solution like this in a week or so. It should give us some time to consider it and come up with ideas for what we want the structure to look like.
I’ve created GitHub - auxolotl/packages as a place to tests new package structures. For now, it just includes nixpkgs lib and associated tests. If people have ideas of structures they’d like to test please let me know!
I’m thinking include some core packages in that repo, but split packages managed by different SIGs into separate repos.
I’m currently thinking about how to structure the Javascript SIG’s work - It seems to me like at the moment, we would create a dedicated repo to hold everything that we work on (eg. derivations for Javascript runtimes, package sets for user-facing NPM packages, tools for building Javascript projects). That repo would hold a dependency on core, and then there would be a second “top-level” repository that exports core and all individual repos. Am I understanding the proposed structure here correctly?
EDIT: Here’s a graph I whipped up to try and convey my understanding a little clearer
That lines up with what I am thinking as well. Thank you for the sketch!
Where would aux’s nix fork lie in this graph? Would it be a part of core? And if it is, is it a flake dependency or does it not depend on the flake input, and is built with fetchFromGitHub the way they do in nixpkgs?
Here’s my take on what a graph would look like.
Rationale: Imagine some rust package depends on a python package. If the rust package group depends on the python one, now someone calling aux top level will potentially have two different version of the same python package. If you route it through top level, then there’s only one version, with the downside being the version in each package group repos lock file will not be the same as the version being served by toplevel
I’m imagining it would lie in core in a similar manner to nix in nixpkgs, but open to discussion
I need to go dig it up but theres a flake with a pure-nix lib (e.g. no stdenv or derivations) and it would be great to depend on that as much as possible over the full nixpkgs.lib
I have created a POC of this structure in the Github org. Relevant repos:
As a demonstration - you can run nix build github:auxolotl/top-level#javascript.test. The POC consists of 3 repos in the arrangement in @VlinkZ’s diagram.
top-level consumes both packages and javascript as inputs, and javascript only consumes top-level.javascript’s output contains a function that accepts packages as input, and exports the hello derivation from top-level.core (ie. the packages flake).top-level’s output contains both the Nixpkgs set from packages, as well as the function from javascript, with the packages flake input passed to it to construct javascript’s package set.The flakes are pretty barebones, and don’t follow the proper output schema - I just wanted to use this as a quick and dirty POC for now. Obviously, we’d be constructing our flakes a lot nicer than this 
In any case, feel free to take a look and let me know your thoughts in the structure!
Just to add to this discussion a bit. It was brought to my attention that we have not made any unstable releases yet so we are still providing 2-3 days out of date packages.
So my proposal is due to the limited amount of people we have is to create releases for unstable everyday at 7PM UTC. This way releases are consistent and if i’ve read everyones timezones correct is a sweet spot for creating releases whilst the majority of us are awake.
I think that’s a fair strategy!