Forge software/host

Also, important to mention for those who don’t know, Forgejo is working on federation, which means in the future you could host your own instance and use a federated account.

1 Like

RE: Forgejo repo size with forks

This assumes everyone is required to fork the project on your hosted forge. Forgefed could alleviate this & has been in the works (unsure of current status). Sending either a raw patchset or a pull request from a different remote repository on a mailing list or other means do not have these limitations; this is how some of these forges operate like cgit, SourceHut, the aforementioned Ayllu, et al. Alternatively, Radicle (the forge, don’t want to hear about the independent cryptocurrency project) approaches distribution differently akin to Bitorrent–where the D in distributed version control system (DVCS) starts to mean something.

This is a problem with these forges looking to clone the MS GitHub user experience, including the requirement of forks necessarily needing to come from the domain, where the for-profit platform with social media features absolutely wants to keep you engaged on their specific platform. Unfortunately the GitLabs/Giteas/Forgejos of the world aren’t looking to solve some core issues of centralization favoring that familiarity & migration over something that might rock the boat with a different approach (or traditional approach in the case of mailing lists).

(note: this is more for generating discussion than offering a specific suggestion)

2 Likes

I’m not to worried about the storage. Like I said, we can experiment with strategies that general purpose forges cannot, ie. use specific file system features like de-duplication or snapshots which might give us an initial fork for close to zero storage. Remind people to delete branches they no longer need and git should be quite accommodating when it comes to storage.

Also, once we enter phase 2 (hard fork with multi-repo) the history likely goes out the window anyways.

There are so many other ways both Gitea and Forgejo would crumble under the load.

Likely true, but I’d rather spend my time working on those than working on GitHub automation features that we would also surely need. Work in FOSS to improve FOSS.

EDIT: Also, a TB of storage on Hetzner Cloud is roundabout ~53€ up to 10 TB, should be croundfundable in my opinion (disclaimer: I work at Hetzner Cloud).

5 Likes

Our concerns right now are around cost and reliability. Unfortunately the only options that cover both of those well would be GitHub and GitLab. Considering the larger network of GitHub, including its built-in audience of nixpkgs contributors, I think it makes sense for us to choose GitHub right now.

11 Likes

I think it makes sense for us to choose GitHub right now.

Can I take this as your final answer and lock this thread until later, or would you prefer to leave it open for further debate?

As much as I don’t want to, I think I agree with the GitHub direction, only because I just don’t see any other viable option right now. Nixpkgs is an absolute behemoth, and if we’re going to be soft-forking it to begin with, I just don’t see any FOSS forges being able to handle an entire copy of it for every contributor (or for everyone who just decided to click the fork button for whatever reason), nor the community being able to reliably secure the funding for that much storage in the short term.

I agree that we should look for ways to ease the load on a forge, and also that contributing improvements to a forge is a productive use of time. But that’s all going to take time - something we don’t really have at the moment.

8 Likes

I think we can keep the thread open. I’m going to get things set up on GitHub once my day starts, but it can be helpful to have insights into other forge options. Let’s use this thread to catalog some of the pros/cons of forges other than GitHub, especially in regards to cost and reliability (approachability is also important to consider).

3 Likes

This is fair, I think the only options at present are GitHub or things that don’t copy the whole of Nixpkgs on every fork.

From the Forgejo contributor here that strikes out it and Gitea (+codeberg)

Gerrit works in a different way so is still plausible, but there is the community issue. Some people in the thread here have expressed a love for it but it really is very different.

I’m unsure about GitLab but again the community is very much on GitHub.

It’s hard to move. It will be hard forever, I guess.

1 Like

I understand the necessity of using either GitHub or GitLab in order to handle nikpkgs repo size at first but what about things that aren’t nixpkgs?

If we commit ourselves to try to eventually move to a better solution why not just have nixpkgs on GH and the rest somewhere else?

I’m aware this is not exactly comfortable and having two separate forges could get annoying but still.

1 Like

Our solution right now doesn’t have to be perfect, we just have to get started. We could spend a whole bunch of time spinning our wheels on a lot of things, that’s why I published Aux’s values, goals, and roadmap first. We can all align on those pieces to complete the roadmap at which point democratic governance can begin and future decisions can be taken by the responsible body rather than an ad-hoc forum thread with a couple polls.

7 Likes

In Nix history as well, the question of whether Github is a suitable host came up multiple time. I remember that this became more hotly discussed when one of our fellow contributors, “volth” — seemingly associated with archive.today —, had their github account completely memoryholed with no communication from Github. This showed Nix at the time that we were at the mercy of Github corporate and had no way to influence who is allowed on the platform. Just some food for thought.

8 Likes

Disclaimer #1: I’ve lived in the world of large monorepos that use a patch-based approach and my end-user experience felt much nicer than any git ever did. Hence I’m a proponent of this design :wink:

Disclaimer #2: This post is about the long-term forge question, I agree with the notion that getting things of the ground ASAP is more important than the best design possible/ feasible.

One aspect the article seems to not talk about much is that this CVS implementation uses a patch-based storage engine but also has a git user interface, a kind of hybrid approach.

IMO this separation allows for some nice possibilities:

  • seperate line protocol/ UI (or multiple) to enable choice of prefererred client and/or remote (cache) syncs/ backups/ … E.g. to host a copy on Github for discoverability/ interoperability
  • choose a storage engine that fits the size and problem scope of the repository
  • allow usage of alternative clients for those who dislike the git flow

From a quick (non-exhaustive) search, I could find the Jujutsu which gives more details to the idea. This is not meant as a recommendation or endorsement, but an example for the direction my thought goes.

2 Likes

One last shower thought RE: audience talk

There is a lot of talk about MS GitHub having an existing audience & wanting to meet them for the project… the type of user that doesn’t want to use another platform & does not want to create a account (OAuth2 aside) on a new platform, is this the type of user that would be attracted to a new fork of Nix …or would they prefer to stick to the old Nix & not learn a new thing? Meanwhile, there are multiple users already in this brand new forum already expressing their needs not being met for not wanting to use a nonfree forge for their free software, but either feeling forced to or left out of the Nix community with no viable way to contribe code. Is this audience worth ignoring for the other, the innovators for the laggards? :thinking:

4 Likes

This is fair. My preference is for Gerrit+self-hosting, if we have the support here. I think it’s much easier and nicer to review code than in a GitHub-style workflow.

I would be a strong proponent of something like TVL’s depot (a monorepo where you can clone down specific directories easily: e.g. git clone https://code.tvl.fyi/depot.git:workspace=views/tvix.git) or even Gerrit with separate repositories. I will stand behind whatever the eventual decision is, though: I’m not quitting if we decide to use GitHub.

I wonder if our thoughts on this should change in light of Eelco stepping down - even if it doesn’t change our goals it would likely slow down the “laggards” who might previously have been jumping on more now.

2 Likes

This would be a needle-mover for me. If the forge is still MS GitHub, the chat is still Matrix, & the forums are still Discourse, I feel Aux becomes purely a political/structural/governance fork which doesn’t push me personally in a direction & feels like more of the same–just new folks & new rules. A fresh start to do things differently is a heavy motivator for me.

4 Likes

I think it’s worth looking into a agit workflow for those not willing to use GitHub Agit-Flow and git-repo – git-repo: a git wrapper from Alibaba

4 Likes

That looks interesting, thanks! I’ll look at it.

My immediate concern is that things which purportedly use a gerrit-like workflow without using the commit msg hook or while using GitHub pull requests tend to use forcepushes, which GitHub handles extremely poorly (clobbering old commits if you forcepush multiple times, losing comments, etc.). I’ll be interested to dig into the technical details of this more later.

Right now I tend to use StGit+forcepush but this is not ideal for the reasons above

4 Likes

codeberg.org is a good host

We’ve mentioned forgejo-based things several times, but consider them impractical due to the scale of nixpkgs and the way they operate (e.g. with forks being copies, etc.)

2 Likes

I feel like if the move to a different forge does not happen by phase 2, then it won’t happen.
Its not just that we’d need to move all the data and workflows from GitHub to another forge.
People will start building things around this project, third party flakes will reference it, blog posts and stackoverflow will point to it, etc.
Yeah, can build real-only replicas of the source tree, but I always find that to be very bad UX when a search engine points me to a GitHub repo thats actually just a mirror.
And even with our best efforts: things will break

5 Likes