Used to run a few storj nodes on their “official satellites” (their lingo for the coordinating nodes on their blockchain), and there’s a tricky piece to their design.
The storj protocol is proprietary and you need a custom client to use it. Their S3 compatibility is implemented as a Gateway it’s not native to the storj protocol.
If your goal is to implement a distributed cache you either have use their SDKs to make your tools speak storj natively or use the S3 Gateway which can become a single point (of failure. performance, access, …)
I was also wondering if approaching companies for sponsorship might be a way to go with this.
One possibility I see (technically speaking) is Cloudflare as they are already in the business of providing S3 cloud storage with 0 egress fess. They have an open-source sponsorship program one can apply to.
How much that aligns with Aux values and vision I have no idea.
That’s a little surprising, given it’s a DetSys project? It would presumably have the advantage of simplicity though. Was the offer just to support development, or potentially for an open public cache as well?
I did think one of the possible goals for Aux - especially if NixOS itself resolves its governance problems - might be to create a more general model for forks, derived distros, etc. Apart from Guix, NixOS is pretty much unique in Linux world in not having yet been forked, and I tend to see diversity as a good thing. Exploring decentralized storage might be a part of that … but the temptation of being able to get up and running quickly with a more conventional approach is definitely appealing too.
I wondered that too, but OTOH NixOS itself seems to have been struggling a bit with getting storage funded so I don’t know what our odds of success would be?
Some product/company I’ve looked at in the last few days either in the CI/CD or storage spheres appeared to use nix behind the scenes, which made me wonder if they would be worth approaching. Annoyingly though I can’t remember who/what, and I don’t appear to have bookmarked it
I should have added to my previous post:
The distributed cache solution sounds and feels rly nice to me, but it looks like it requires engineering and testing effort. Hence might not fit the requirement of a fast solution.
Offers like sponsorship might be able to solve the more immediate need.
So maybe what sets the 2 possibilities apart is just the timline
For those blissfully unaware of what work I’ve been doing, I have been working on reducing the need for us to create a binary cache at least for the time being. So this should mean that we have longer to discuss potential options. And thus I really want to weigh in but I really have no clue where to start.
I am very happy to work with attic. I have done a lot of work with it but by no mean is it easy. Though this is where I would like to start. Whilst continuing to bootstrap off of cache.nixos.org. Though I have no idea how the backend would work.
This is awesome. But I thought we were trying to move away from DetSys.
If we can confirm that the cache is not proprietary then I don’t mind using it, at least not until Aux is such a runaway success that we can easily host our own cache.
Putting my cards on the table: I think we should be friendly with other parts of the ecosystem, but not put up with any BS.
i really think this is the obvious choice. i’m sure many here have their opinions on them and coporations in general, but i take this as a pretty clear sign of good will – after all, a binary cache at the scale of the nixpkgs is no small expense at ~$9000 USD/month for the foundation
i also don’t see a way we would be able to do this without any corporate sponsors or products. at the very least, basically any alternative would still probably end up giving money to faceless corporations for Simple Storage Solutions (if you get what i mean ). at least with detsys, we know the people behind it and seem to be personally supportive
this is also not to mention the trust issues with even a single binary cache. a distributed cache would only compound this and add a much larger attack service. i would highly recommend against this option until trustix (or similar) is production ready and can be mass adopted
To be transparent I think its worth mentioning that unless @jakehamilton got explicitly told that they are going to have free access for the long run its not worth considering. See below image for context.
I think the simple fact about it is that there is no way without some major sponsor. Which is why I’ve been trying so hard to keep using nixpkgs cache.
why not? even as a temporary solution, this would do wonders in making actual changes to nixpkgs rather than continuing to be what’s honestly just the same thing, but slightly less up to date
it could also give us a huge jumpstart on fresh repos like core, javascript, etc
). at least with detsys, we know the people behind it and seem to be personally supportive