Agenda thread:
SIG Core 2024-05-18
Attendees:
Agenda
- Introduction since first meeting, some new folks around
Take stock of current progress
- @srxl: What do we currently have in Core?
- @srxl: Building both Nix and Lix on linux
- @srxl: Core doesn’t need everything, may want to trim down…
- @srxl: e.g. fetchers, updating logic, etc.
- @srxl: Some language infrastructure
- @srxl: maybe we should see about moving it out?Remove flake dependency on nixpkgs/core by tcmal · Pull Request #2 · auxolotl/lib · GitHub
Core direction
- @dfh: How does core fit into the “big picture?”
- What should it contain?
- @srxl: currently we’ve duplicated nixpkgs’ standard environment, is there anything else we need?
- @dfh: some common build dependencies, or will they be handled out of core?
- @srxl: coreutils should stay in core, it’s needed in stdenv… things like build-essential could be in core, possibly some other build tools like meson, cmake
- @srxl: I’m not decided on openssl, zlib, etc. What do we think?
- @dfh: This is why I asked, do we want it minimal or do we want it to be something that can serve every build process?
- @liketechnik: We could say “everything needed to build lix or nix should go in core”, that’s a clear criteria and won’t snowball
- @srxl: That makes sense, it’s about bootstrapping something that nix/lix will consume…
- @srxl: Will there be some other repo that has these other libraries? Who maintains the repo? Core? Another SIG?
- @dfh: If you have core, which is needed to build lix and nix, and other languages… can you construct the build environment only using core? e.g. languages would have their own build toolchain dependencies
- @dfh: some common build dependencies, or will they be handled out of core?
- @srxl: currently we’ve duplicated nixpkgs’ standard environment, is there anything else we need?
- How should it be used by other SIGs?
Moving lang infrastructure out of Core
- @srxl: what happens if bits of core need to use other languages to create the stdenv? We don’t want circular dependencies…
- @liketechnik: @jeff on the forums had some great ideas… have you had time & spoons to check that out?
- @srxl: Could be worth going back to the thread and rereading it
- @liketechnik: Gist was that everything is tied together in another place, and you can ensure that there are no circular dependencies in that place… e.g. “you will get something to build rust packages and you do not care what it is”
- @liketechnik: It requires a lot of coordination and effort to maintain two-way interfaces or test each repository, unfortunately
- @srxl: This seems like something we need to sort out soon. It’ll be a blocker to getting SIG Language groups up-and-started without Core having decided this. This should be prioritized
- @srxl: Makes language repos too hard to consume individually. They’re kind of out of luck without more stuff, which is sort of like pulling in all of nixpkgs. We should probably think about an architecture that makes this possible
- @srxl: This seems the most concrete, maybe not most optimal
- @minion: I care a lot about not having to eat everything to consume python
- @srxl: Yea, definitely this is important
- @aria: aria (she/it) says:if you’re not using flakes and you’re ok with IFD, it would be possible to have that with registry right?
- @liketechnik: @jeff on the forums had some great ideas… have you had time & spoons to check that out?
Moving lib into its own repo
- @srxl: I wasn’t super sure about this, lib isn’t wholly pure, there are tests, etc. which depend on core
- @aria: Remove flake dependency on nixpkgs/core by tcmal · Pull Request #2 · auxolotl/lib · GitHub removes core dependency…
- @srxl: Ahh, that’s interesting, if we can get rid of that completely then splitting it out is good! Not pulling in nixpkgs to use lib is awesome!
- @aria: There is also GitHub - nix-community/nixpkgs.lib: nixpkgs lib for cheap instantiation [maintainer=@github-action] (with initial help from @blaggacao)
- @srxl: I don’t know if it’s useful to do something similar if we have multi-repo stuff anyway
- @liketechnik: Indeed, it’s not up for discussion on whether we should split them, it’s just the how
- @liketechnik: I think lib does many things, it’s some pure nix stuff, some build stuff and some of the module system. I wonder if we should split things into more repos
- @srxl: Maybe these are getting too small. Perhaps we should split the module system, but by this point some components may be only 1 file
- @srxl: I don’t think there’s any builders, it’s mostly pure nix code…
- @dfh: If we intend to support the module system for writing packages, it feels confusing to have modules in a repo other than core/lib.
- @srxl: Module system being in lib is a little weird in nixpkgs, I think keeping things similar is a good bootstrapping decision though…
Working with Security Committee
- @srxl: We had a lot of dependabot alerts, they seemed to be in tests (so not particularly relevant)
- @dfh: 16/25 are in tests, the rest are outside of tests…
- @dfh: You need two kinds of checks
- “Am I committing known security vulnerabilities?”
- A scanner, to check that you are handling known stuff over time…
- It’s looking at dependencies rather than your code, things that are fixable by providing updates
- @srxl: Ok, so we’d have some tool, and if we get an alert we’d address it
- @srxl: When it comes to vulnerabilities that don’t have fixes yet, do we prefer to pull it (and break stuff)
- @minion: nixpkgs has a way to indicate insecure packages, and everyone then has to agree to them
- @srxl: then that would break everything for everyone in core…
- @dfh: Don’t make it impossible to insecure software, but make it hard for people to do accidentally. The insecure flag does really well
- @dfh: If Aux has old versions that may become even more important to mark things consistently as insecure, but it gives more possibilities to play with secure/insecure bits
- @srxl: Can we make sure that language repos don’t depend on insecure package versions?
- @srxl: Has nixpkgs ever had part of stdenv marked insecure?
- @dfh: in that case we need to coordinate with other SIG repos to ensure other things pull the fix…
- @dfh: You need an “audit my system right now” feature. It needs a list that doesn’t depend on the package repo. You can’t ask the question about a specific package set, because it might change as security vulnerabilities are discovered…
- @dfh: I want us to inspect this against a database, or similar… If we start tracking security vulnerabilities it needs to both end up in the database and be marked as insecure in packages. That stops people installing something fresh and insecure, and lets people run a checker on old stuff
- @srxl: Yes, this needs to not be locked… I don’t know that we can do that in core directly
- @dfh: Yes, I agree, we need to work on “the audit thingy” in security. It needs to stand vertical to everything else.
- @minion: nixpkgs has a way to indicate insecure packages, and everyone then has to agree to them
- @srxl: Can we automate some of these things, e.g. pull requests for marking vulnerable?
- @dfh: There’s not a single central vulnerability database. We need some pipeline where we can ingest from a bunch of sources, review, etc. It’s a huge project. Automation would be very great, but challenging…
- @srxl: This sounds like a good process, but not particularly easy, so not a short-term goal
- @srxl: So, the immediate process is to mark insecure as soon as we get a vulnerability notice.
- @dfh: Ok, can we start with a smaller targer, e.g. look at things similar to the
npm auditoutput and start to apply it to the core repo manually on a regular schedule…
- @dfh: Ok, can we start with a smaller targer, e.g. look at things similar to the
Any other business
- @minion can we change the URL to https://meet.jit.si/AuxSigCore?
- @srxl: Sure
Next meeting - when? how often?
- @srxl: Core can have meetings more ad-hoc… we haven’t been blocked by meetings, and since as @vlinkz and @jeff are not present it’s hard to know how important it is. If anyone in core wants to raise a meeting, please feel free to post something on the forums
- @srxl: We can revisit this later, if we like
Decisions/Action items
- @coded Update Wiki page (https://wiki.auxolotl.org/en/community/core/aux-sig-core)
- @minion Discuss Lix/Nix compatibility with steering
- @srxl Update time and link for meeting on wiki (Unauthorized | Auxolotl Wiki)
- @srxl We need to prioritize figuring out how circular dependencies work/language build tools in core
- @dfh Think/ discuss cross repo coordination between core/ langSIGs when it comes to patching vulns
- @dfh Think/ discuss about recommendations for language specific sec/ vuln scanners or dependabot alternative that works with forgejo
- @dfh Send a PR for dependabot-pr to core repo (Automating Dependabot with GitHub Actions - GitHub Docs)
