On the future of our nixpkgs fork

following The Future of NixCPP: Lix and the amount of support for adopting it over our cppnix fork, i think now may be a good time to reflect on some of the other forks originally laid out in the roadmap. namely, nixpkgs

now, this is a major part of our roadmap. just like our cppnix fork, it is actually in the first sentence

We will fork and maintain Nix, NixPkgs, and NixOS.

i can understand the hesistancy around any changes here, but following the discussions in the thread mentioned above, i have hope that we can have a good discussion here as well and find a way to give a better experience to users, while also lessening the burden on contributors

Our current state of affairs

as of now, our nixpkgs fork isnā€™t in the best shape. for the most part, we are always lagging behind upstream and simply pulling in there changes every hour. with the exception of ~4 PRs, we havenā€™t made any real changes. maintaining this creates a few issues

Unclear instability and security risk for our early users

judging by some posts iā€™ve seen here and communications iā€™ve had elsewhere, a lot of the recent drama has actually introduced new people to both nix and aux, and these people are of course looking for a way to start using one or the other. there are also many pre-existing users who are looking to move to an alternative better aligned with their values. if either of these groups choose aux, they will be using our nixpkgs fork ā€“ which as stated before, doesnā€™t provide the best experience right now

the biggest issue here is that only master is synced. if an existing user simply does a s/nixos/auxolotl/ on their nixpkgs input and are using anything besides master (which will be basically everyone), they are getting an out of date and possibly insecure version of nixpkgs. this is currently not made clear anywhere

for new users, they will most likely use github:auxolotl/nixpkgs. this inherits many of the issues of nixpkgs master such as less binaries and less testing, and is then compounded by any changes we make (that also arenā€™t currently being tested). once again, this is also not really stated anywhere, and new users are unlikely to understand the differences between master, nixos-unstable, and nixos-23.11/24.05

if anything is to come out of this post, this must be resolved. we should be actively discouraging the use of this until then, especially as there is no real advantage to using this fork as of now. bad experiences using this fork like this can easily result in driving new users away from both aux and nix

A weird relationship with upstream

as said before, we regularly sync with upstream nixpkgs while also accepting some contributors. this creates a few problems, such as needing to worry about our changes possibly making the binary cache unusable. more important long term though, it creates an odd situation for contributors

for this example, lets say i want to add a new package called auxcord (like in your car, not discord). i can take a few paths here

  1. contribute directly to auxā€™s fork: i use aux, so my first instinct would be to contribute to the aux repository iā€™m using. all aux users benefit!
  2. contribute directly to nixpkgs: i use aux, but i also know many donā€™t. i also might have other packages already in nixpkgsā€¦so what if i contribute there instead? in this case not just aux benefits, but also the rest of the community!

as someone who does maintain some packages and plans on adding more, the obvious choice seems to be the latter. after all i want my packages benefitting as many people as possible, and aux will be pulling the change from nixpkgs anyways. what reason would i have to contribute to aux?

now trying to put myself in the shoes of someone who only uses aux, i still canā€™t really find a reason to contribute to the fork. at the very least, contributing to nixpkgs directly instead will (mostly, assuming no changes to dependencies in aux or other situations) guarantee i will have my contribution cached. the only reason iā€™ve thought of is one out of principal in the case you disagree with the foundationā€¦but as we are still pulling in their changes regularly, i would argue this stance doesnā€™t really do much at current time

some other important points to note here is how easily this can lead to literally line-for-line duplicated effort (this has already happened. see factorio: 1.1.104 -> 1.1.107 by Daholli Ā· Pull Request #2 Ā· auxolotl/nixpkgs Ā· GitHub and factorio: 1.1.104 -> 1.1.107 by WooDyDooKs Ā· Pull Request #305320 Ā· NixOS/nixpkgs Ā· GitHub) and a situation where the aux project is directly taking without giving anything back (this is also happening and will continue to with PRs made to the fork directly). as a contributor to nixpkgs, this isnā€™t the best feeling ā€“ even if i do align myself much more with auxā€™s values

A huge burden on us

following discussion in Musings on a monorepo versus developer oriented distribution, SIG Repos: How should they work?, and some PR threads, the decision has been made (and work has begun) on splitting the monorepo structure of nixpkgs into individual repositories, each with a narrow focus. as some of you may already know, i am a huge supporter of this and i think one of the best technical ideas weā€™ve had here, and as such i think it should be our focus

regardless of my feelings on this subject though, we have a clear issue: we are splitting maintenance between both all of nixpkgs, and individual repositories.

nixpkgs alone is a massive undertaking for even the current upstream contributor base. adding onto this with repositories meant to eventually replace it is something i donā€™t think any project could properly handle ā€“ at least not without sacrifices being made on both ends, resulting in a worse final product

A solution?

i propose we drop our nixpkgs fork

given the risks associated with using it currently, the little reason for anyone to directly contribute, the long term maintenance burden, and the thin spreading of our little resources i do not think maintaining all of these is a viable goal. we should instead focus solely on our individual repository structure.

this would of course involve moving some of nixpkgs into these repositories, but it seems this is already happening. i believe this would also be much more beneficial in the long term maintenance wise and give us more opportunities to fix issues we may have with nixpkgs ā€“ rather than go back and attempt to make these change

Some issues

the biggest one here is that this will stall the process of users actually getting aux in their hands, as without nixpkgs we would need to wait on PRs like the stdenv one linked above to be merged. iā€™d argue this doesnā€™t make a big difference, though

as said before, we arenā€™t really making changes to nixpkgs right now. for all intents and purposes, auxā€™s nixpkgs fork and upstream nixpkgs are the exact same, except in name. given this and the previous risks mentioned, is anyone really getting anything out of aux currently? can we even really consider or fork a fork and not just a set of string changes? i think @isabel summed this up well in a conversation we had last night

tbh we donā€™t even make changes to nixpkgs fork [so] why does it even exist

itā€™s also important to note that moving efforts from the nixpkgs fork would also directly give us more resources in making the development of our individual repositories faster

A plan of action

if this proposal were to be accepted, i think we should continue as follows:

  1. complete and merge add stdenv by vlinkz Ā· Pull Request #2 Ā· auxolotl/core Ā· GitHub. this will allow us to work independently of nixpkgs itself, just at a smaller scale for the time being
  2. decide on exactly what will be considered ā€œcoreā€ and add that to core. some prior art can be referenced here, such as arch linuxā€™s core repository section and packages required by the nixos minimal iso. i prefer the latter, as it is a more tangible deliverable and would help us in tracking when this initial work in core is complete for the time being
  3. mark core (and thus top-level) as usable, but alpha/beta. we should also be able to build a minimal ā€œauxOSā€ install iso at this point, and we could release that alongside this announcement. i think this is a good way to not keep aux out of the hands of people for too long, while also giving them a product that isnā€™t just nixpkgs without any real changes. this is partially inspired by Lixā€™s approach to their beta announcement, as they have already provided a distinct product for people to try
  4. with the foundation in core built, we should now be able to flesh language specific/broader SIG repositories. assuming the model described in SIG Repos: How should they work? - #15 by VlinkZ and flake: use core over top-level by getchoo Ā· Pull Request #1 Ā· auxolotl/javascript Ā· GitHub, repositories like python and javscript can now be built up. the intention here shouldnā€™t be to reach full parity with python/node/etcPackages, but be enough to lay the grounds for repositories that rely on them. a good benchmark of the work here being complete would be dependencies for gnome and kde plasma being satisfied
  5. with our language specific repositories fleshed out, we should now be able to move onto more narrow, but complex SIG repositories (i.e., plasma and gnome) as well as top-level. SIGs that have already completely their initial work could be much more rapid in their expansion ā€“ as long as it doesnā€™t affect the newly growing repositories ā€“ and we should get more testers involved to avoid any unforeseen issues. a good benchmark of completion here would be plasma and gnome isos being able to be built
  6. now that we have core, language-specific, and even DE SIG repositories at a good enough stage, we should be able to build a full install iso. i believe this would be a good mark for a beta or even initial full release

Summary

by dropping our fork of nixpkgs itself and instead moving some of it into our individual repositories, i believe we can not only help guarantee the long term viability of our maintenance load, but also give people a true ethical and technical alternative to the existing nix ecosystem. we really have something here with this concept, and i would love to see it to fruition without worrying about the tech debt of one of, if not the largest monorepos in the world layered on top

28 Likes

I heavily agree here. I care a lot about some of the things that Aux can bring, such as the documentation and security improvements, but I believe that this can be possible without trying to maintain nixpkgs.

I think Aux, and therefore the roadmap, started in a panicked time when we genuinely thought that the foundationā€™s response would cause a decent amount of nixpkgs contributors to pack up and leave there. If they had done so and had come to Aux, the roadmap as written would look a lot more reasonable. That didnā€™t happen, so we should adapt our approach.

14 Likes

A beautiful write up @getchoo, thanks for this.

Just to go over some points that I found unclear or rather would be a result of this:

  • We delete the nix, nixpkgs and nixpkgs-review forks
  • We start rebranding as an alternate nixpkgs provider and thus os
  • We rewrite a clearer roadmap with our new set of goals

I understand to some this might be disheartening but I really do agree with getchoo and think this is a good way forward. And I completely agree that dropping our forks would really allow us to focus on these better goals.

Not to mention how good it would be for clearing up peoples understanding of what we are doing. Over the course of the last 2 days Iā€™ve heard people think we are a forced system where you must use our nix fork, but of course you donā€™t have to. We got compared to a NUR as wellā€¦ So I think this would be a really good break.

13 Likes

Strongly agree. But weā€™d still have the synchronization issue with nixpkgs even after that, right? Instead of blindly pulling the entirety of nixpkgs into a second repo, weā€™d instead have to piecemeal updates from certain packages in nixpkgs into the appropriate aux repo. And, possibly, do the same thing in reverse, dumping any aux repo changes into nixpkgs.

  • Will the development of this sorting tool hold up usability of package availability?
  • Will new packages in nixpkgs remain unavailable until an appropriate aux repo has been selected for it?
  • Would aux benefit from a more gradual change, where packages slowly get migrated out of auxolotl into individual repos?
  • Could even do something like AUR in Arch, where the entirety of nixpkgs is in a ā€œcommunityā€ repo; not a lot of maintenance, but it can be used if what you need isnā€™t available yet in the aux repos.
2 Likes

(Iā€™m maybe off topic here, but ā€¦) I expect almost all the people who are leaving nixpkgs will be having a good long break. THEN maybe theyā€™ll start contributing to auxolotl instead.

3 Likes

Iā€™m okay with this, as the author of one of the PRs to auxolotl/nixpkgs. In fact, reviews came in on the NixOS counterpart that will force me to rethink my code anyway.

4 Likes

Iā€™m in full agreement here. Iā€™d honestly go as far as to say that the ā€œsoft forkā€ phase of the roadmap should be entirely scrapped, and we should instead focus on building the hard fork. Maintaining the soft fork has proven to be a near impossible task that achieve near 0 benefit - I see no reason to continue doing it. Iā€™m 100% on board with pivoting to a ā€œNixpkgs alternativeā€ that approaches things in completely new ways (like the split repos, for example, but I wouldnā€™t be opposed to more radical changes in codebase as well).

Iā€™m unsure how much we want to deviate, though. Do we want to be ā€œNixpkgs compatibleā€? Or do we want to be our own thing entirely? Personally, I donā€™t think striving for compatibility is a useful goal, since Nixpkgs is a decently fast moving target. Less fast on versioned releases, sure, but itā€™s still a target that we donā€™t have control over, and will very heavily influence our decisions if we choose to target it. Plus, if we are just a Nixpkgs-compatible package set, then we need to ask ā€œWhy should someone choose us over Nixpkgsā€? We need to have some distinguishing factor from Nixpkgs that people look at and say ā€œOh, thatā€™s interesting, I want to try this insteadā€. Iā€™m not sure we have a lot of wiggle room to do that while maintaining compatibility with Nixpkgs at the same time.

Hereā€™s a general overview of what my ideal roadmap would look like:

  • Drop the Nixpkgs repo (as well as nixpkgs-review and nix)
  • Bring in Nixpkgs as a dependency of Core, and use that to allow other SIGs to bootstrap their own work
  • Plan and build out a Core library set, taking prior art from Nixpkgs and changing things however se see fit
  • SIG repos begin adopting parts of Core, and removing their dependency on Nixpkgs
  • Eventually, Core drops their nixpkgs dependency, and we become our own completely isolated project and package set
14 Likes

Iā€™m also in complete agreement with this post. As I mentioned before:

As linked by @getchoo, Iā€™ve been working on splitting stdenv from the rest of nixpkgs as a base for our core repo. This may take a little bit longer, but in the meantime it is completely feasible to start building using coreā€™s nixpkgs export understanding that switching over to using core exclusively may require additional effort.

I think that by starting small and not burdening ourselves with all of nixpkgs, we can all focus our efforts on creating a better package structure rather than getting crushed under the weight of nixpkgs.

I agree 100%, given the current situation, I think a soft fork would just hinder further development.

I also believe that in the time being, even once we have a functioning fully aux system, having a nixpkgs export from core may be a good idea (although using flakes a user could just add the input themselves).

Finally I agree that the roadmap itself needs to be seriously reevaluated. I feel like as some time has passed, the original roadmap does not reflect the reality of the direction of this project. Some part I think are no longer valid:

  • Soft Fork: I believe that trying to maintain a soft fork currently is a bad idea as outlined above
  • Fork CppNix: I think given the recent unveiling of Lix, the Cli SIG should discuss what the plan should be moving forward
  • Infrastructure: If we go in the direction proposed and build up our package set gradually rather than by forking nixpkgs, I believe that work on infrastructure such as CI and a binary cache can start a lot sooner given the smaller scale

As of right now, I agree with @getchooā€™s plan of actions, with the addition that infrastructure work start in parallel with work on core and SIG repos.

14 Likes

This makes me a little sad, because independence from nixpkgs is still a big goal of mine (Iā€™m not currently super optimistic about its future). That said, the elephant in the room was always that the only way to fork nixpkgs and keep maintaining it, was going to be to match its structure and conventions exactly so that pulling in upstream remained possibleā€¦

ā€¦ and that prevents us from doing any of the more interesting things until weā€™ve fully hard-forked anyway, except weā€™ll then also have a giant diverged nixpkgs ball to maintain while trying to build new things.

Iā€™m not looking forward to rebuilding nixpkgs from the ground up, but if thatā€™s the long-term goal anyway, and nixpkgs endures despite my fearsā€¦ then yeah, no sense in creating a clone of all that work and then also try to do new things, we should do just the new things.

Thank you for the write-up :slight_smile:

8 Likes

As a currently non-contributor to any aux repo I donā€™t have any say, but I will anyway :stuck_out_tongue:
I think the initiall roadmap itself makes a lot of sense.
Itā€™s just a few days now and despite the soft-fork being more effort, less reward than the hard-fork, itā€™s a statement. Right now I think itā€™s much more important to stay on a slower tracl track than only carry what weā€™re able to do hold long term.

Itā€™s still very unclear what will happen in the nix community and much too early to give it up IMO. People wonā€™t expect a perfect environment and depending on how the nix governance decisions turn out (which will have to include compromises not everyone likes) there will surely be more waves of contributors deciding to check out the alternative. I think it makes sense to ā€˜wait for themā€™ and try to flesh out the hard fork topics more and other related stuff like brand, etc.

4 Likes

Not that I disagree (and not just that re-translating the roadmap would be a little annoying :stuck_out_tongue:), a couple of points to consider:

  • ā€œWe will fork nixpkgsā€ is a pretty simple, and therefore very flexible, goal of which the current execution is one interpretation.
  • The majority of the mentioned issues with the fork are tied to the execution - not to the goal.
  • As an example, a fork which only provides current stable branch, synced at some frequency actually sustainable for this project, is still an entirely valid fork. I actually thought this was how things were set up now.
  • I understand the desire to always be the objectively superior choice to anything nix, but given differences in resources and level of establishment, that is obviously not an immediately achievable thing. The context requires compromise and that is ok.
  • ā€œFaak it - letā€™s drop itā€ is a valid compromise, but as given in one example above, not the only one.
  • The mentioned communication issues are ofc. very real, but the response to them does not have to be radical change to project direction. That said, there can ofc. be other reasons for still wanting that change.
2 Likes

Oh, and on contributing: If aux goes copyleft then Iā€™d just not be contributing, but outside of that I donā€™t see much point in contributing to upstream. Itā€™s nix - if people want what we got they can just import it.

5 Likes

wellā€¦this has got a lot of activity in the last day :laughing:

i would rather not spam a ton of replies, so this is just going to be one big response to most of the feedback here. bear with me

i share a similar sentiment. a lot of the ideas being thrown around feel like a knee jerk reaction to the drama, and not exactly a calculated response. i think we can learn from lix here

i think this is a really good way to put this, and it could work well in our future collaboration with lix. we could easily have them be the alternative cppnix implementation, while we provide the alternative nix{pkgs,os}

to an extent, yes. i think this will be lessened though, as we should ever attempt to reach the same size as nixpkgs. personally, i imagine that us kinda building a ā€œnewā€ nixpkgs from the ground up can give us a good opportunity to see whatā€™s really needed in the repository. this would gives us an easier time in maintenance, allow for faster iteration with things like staging merges, less duplicated work for niche packages, and in general a better experience for users with more packages being better maintained. some prior art in this aspect would be a distro i use to work on a bit, solus. they have (or had? i sadly havenā€™t kept up with them in a while) an interesting policy of new package inclusions also requiring a justification for it being added

now assuming we donā€™t try to keep consistent parity with nixpkgs (and have less packages in general), i donā€™t imagine would be that big of a deal. i would also find the duplicated effort here a bit more worth it, as weā€™re doing something completely new ā€“ not just giving users yet another monorepo

this is again assuming we will just pull all changes from nixpkgs. i think this would be a misstep

for changes we do pull? i think many could be manual, or just done with a r-ryantm equivalent

this again assumes all new packages added to nixpkgs will likewise be added to auxā€™s repositories, so in assuming they arenā€™t as i said before, this would in a way be true

for packages that are accepted to aux, i donā€™t think choosing an appropriate repo would take much time. something in pythonPackages for example will just go into auxolotl/python 9 times out of 10

i donā€™t think this is a very good idea, as if we do this for long enough i would be concerned about the effort required to move away from it ā€“ especially as new packages are added. this could easily create an ever moving goalpost where each time we move packages, we will have more added that will need to be moved later on as well

Could even do something like AUR in Arch, where the entirety of nixpkgs is in a ā€œcommunityā€ repo

i have a lot of opinions on the aur and many of itā€™s issues (as an arch user for many years and still an active aur maintainer), but for our purposes the only one i will mention is moderation. i would much rather be proactive in avoiding potential issues with poor quality or genuinely malicious packages

i would much rather see the current nixpkgs approach taken in each repository, where anyone can contribute or maintain a package, but these changes do need to be approved by a committer (a SIG member in our case) ahead of time

i completely agree here. i would also like to note that even if we arenā€™t directly nixpkgs compatible, itā€™s not like packages from either could be mixed and matched with overrides or package options in modules

i am a bit iffy on this, as it could easily put a pretty massive burden on core if dependent SIGs are overzealous in what they use from nixpkgs. i can get how the current synchronous approach of core ā†’ language SIGs ā†’ broader/DE SIGs ā†’ top-level can be annoying for those who need to just wait on getting their work start, though. maybe a good solution here is to adopt nixpkgs as a dependency like you said, but ensure that all packages used must be in the build closure of the standard install iso ā€“ as based on the roadmap i originally proposed here, that should be the real benchmark of a beta release for example

but yeah assuming this happens, i would be find with nixpkgs being a dependency for a bit. i would just make sure this can easily happen ofc

i would be against this in color simply because it would mean nixpkgs itself would become a dependency of all other repositories, which would one again kinda defeat the point of moving away from a monorepo imo. maybe we could do this with top-level as that should be the repo people on an auxOS should be usingā€¦but like you said, users can just add the input themselves (with flakes or something like npins)

yeah, especially with us using a pure nixpkgs as a temporary base while core is worked on

this is still my goal here honestly. even though nixpkgs may be used as a temporary bootstrap tool, i would very much rather it not become a permanent dependency on our infrastructure ā€“ because then how are we really an alternative? :stuck_out_tongue:

and this will be hard, definitely; i think this is a good spot to learn from some of the possible mistakes of nixpkgs though. as i said before, creating a more curated experience with our repositories might be able to make a better experience for both users and developers. the aux i see in the future isnā€™t another 90,000 package large repository, but something more sustainable (and comprehensible) for a smaller community

statements are important, i agree. where i disagree though is that a fork of nixpkgs itself is required for this. if anything, i find moving away from a monorepo entirely and separating concerns into smaller, but more independent communities (away from the control of a central group of committers) to be a much more impactful

i would argue much of the issues are actually inherit to the goal itself. having both a nixpkgs fork and individual repositories will always be a massive undertaking. likewise, directly inheriting work from nixpkgs upstream while also accepting PRs to our fork will always create a weird situation for contributors

valid? definitely. but useful? you would have a hard time convincing me at least

only having the current stable (23.11) is an instant no go for me, as i use unstable more often on my devices. this also removes the more fast tracked -small branches for those willing to possibly compile more, or the regular nixpkgs-unstable branch used by a vast majority of non-nixos consumers. having a singular branch that is still basically (only that branch of) nixpkgs but slightly outdated isnā€™t a very appealing offer

this was never my goal honestly. even though i am very interested in the idea of individual repos for example, i wouldnā€™t consider it objectively better for a user at the end of the day. my desire is to be an alternative to nixpkgs that some may prefer, not a definite upgrade


and finally, thank you all for your feedback here! i'll admit i was a bit nervous about the response i may receive due to how drastic this change was -- and purposefully waited a bit for things to settle -- but the discussions here have been great. keep it up guys :)
11 Likes

Honestly, thank you for such a monumental response! It definitely gave me personally a lot to chew on, and Iā€™m sure the more technically-minded folk will appreciate the many ideas you gave.

Thereā€™s actually one thing that I canā€™t get out of my mind looking through your response, and itā€™s your comment on the nature of the eventual repo(s):

This coupled with your mention of Solus got me to thinking. Trying to keep up with the pace of nixpkgs-unstable is a losing battle for a much smaller, community-led and -focused distro (which is what I eventually see this being), but sticking only with a single stable branch also seemsā€¦ unappealing for a number of reasons. A happy compromise would be maybe having a stable branch but also having a rolling branch with a Solus-esque release cadence of once a week, which would be a fairly manageable pace.

I dunno, just a thought I figured Iā€™d throw out there. I love the idea of something more curated and focused on individual people rather than enterprise uses, and I think that could give aux the unique value proposition that it needs.

8 Likes

Also, if we make our own PKGs repo, I wonder if we can have x86_64_v2 as a default.

3 Likes

Prior discussion (discussion of bumping up the x86 microarchitecture level) here:

4 Likes

I donā€™t know where to put this but I think it would be a good idea to use this opportunuty to separate nixpkgs (or our equivalent) fro nixpkgs.lib and deduplicate the builtins with the lib

Take a read of this Musings on a monorepo versus developer oriented distribution, which out lines what we might do.