Skip to content

Fix running multiple nix-toolbox containers simultaneously#37

Open
thrix-bot wants to merge 6 commits intomainfrom
fix-multi-container
Open

Fix running multiple nix-toolbox containers simultaneously#37
thrix-bot wants to merge 6 commits intomainfrom
fix-multi-container

Conversation

@thrix-bot
Copy link
Copy Markdown
Collaborator

@thrix-bot thrix-bot commented Apr 1, 2026

Summary

Fixes nix-env vs nix profile format conflict when running multiple nix-toolbox containers that share the same Nix store (~/.local/share/nix).

Previously, each new container checked [ ! -e "/nix" ] to decide whether to install Nix. Since /nix doesn't persist across containers, every new container attempted to re-run the installer against the shared store, hitting:

error: profile "/var/home/thrix/.local/state/nix/profiles/profile" is incompatible with 'nix-env'; please use 'nix profile' instead

Now the script:

  • Mounts the shared Nix store first, before checking installation state
  • Detects existing Nix installation via the profile script (~/.nix-profile/etc/profile.d/nix.sh) rather than the /nix mount point
  • Makes nix.conf creation idempotent

Assisted-by: Claude Code

Summary by Sourcery

Ensure Nix is installed and configured based on the shared user profile rather than the /nix mount, so multiple nix-toolbox containers can safely share a persistent Nix store.

Bug Fixes:

  • Prevent repeated Nix installations and profile format conflicts when multiple containers share the same Nix store.

Enhancements:

  • Bind-mount the persistent Nix store to /nix on startup and validate that /nix is a proper directory and mountpoint.
  • Make Nix flakes configuration in /etc/nix/nix.conf idempotent and resilient to pre-existing content.

Previously, each new container checked `[ ! -e "/nix" ]` to decide
whether to install Nix. Since `/nix` is a mount point that doesn't
persist across containers, every new container attempted to re-run the
installer against the shared Nix store, causing profile format conflicts
(`nix-env` vs `nix profile`).

Now the script:
- Mounts the shared Nix store first, before checking installation state
- Detects existing Nix installation via the profile script rather than
  the `/nix` mount point
- Makes `nix.conf` creation idempotent

Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
@sourcery-ai
Copy link
Copy Markdown

sourcery-ai bot commented Apr 1, 2026

Reviewer's Guide

Ensures the Nix store is always bind-mounted from persistent storage, detects Nix installation via the user profile instead of /nix, and makes flake configuration in nix.conf idempotent so multiple containers can safely share the same store.

Sequence diagram for container startup with shared Nix store

sequenceDiagram
    participant Container
    participant nix_sh as nix_sh_script
    participant FS as Host_filesystem
    participant NixInstaller

    Container->>nix_sh: Run nix.sh on startup

    nix_sh->>FS: Ensure $XDG_DATA_HOME/nix exists
    nix_sh->>FS: Check /nix existence and type
    alt /nix exists but not directory
        nix_sh-->>Container: Print error and exit 1
    else /nix dir ok
        nix_sh->>FS: Create /nix if missing (sudo)
        nix_sh->>FS: Bind-mount $XDG_DATA_HOME/nix to /nix if not mountpoint
        nix_sh->>FS: Ensure /etc/nix/nix.conf enables flakes idempotently
        nix_sh->>FS: Check for $HOME/.nix-profile/etc/profile.d/nix.sh
        alt Profile script missing
            nix_sh->>NixInstaller: Invoke single-user install
            NixInstaller->>FS: Install Nix into shared store
        end
        nix_sh->>FS: Source profile script if present
        nix_sh-->>Container: Nix environment ready
    end
Loading

Flow diagram for updated Nix initialization and installation logic

graph TD
    A_start["Start nix.sh"] --> B_ensureDataDir["Ensure $XDG_DATA_HOME/nix exists"]
    B_ensureDataDir --> C_checkNixExists["Does /nix exist?"]

    C_checkNixExists -->|yes| D_isDir["Is /nix a directory?"]
    C_checkNixExists -->|no| F_createNixDir

    D_isDir -->|no| E_errorExit["ERROR: /nix exists but is not a directory; exit 1"]
    D_isDir -->|yes| G_checkMountpoint

    F_createNixDir["Create /nix directory with sudo"] --> G_checkMountpoint["Is /nix a mountpoint?"]

    G_checkMountpoint -->|yes| H_flakesConfig
    G_checkMountpoint -->|no| I_bindMount["Bind-mount $XDG_DATA_HOME/nix to /nix (with sudo)"] --> H_flakesConfig

    H_flakesConfig["Ensure /etc/nix/nix.conf enables flakes idempotently"] --> J_checkProfile["Does $HOME/.nix-profile/etc/profile.d/nix.sh exist?"]

    J_checkProfile -->|no| K_installNix["Run Nix installer in single-user mode"] --> L_sourceEnv
    J_checkProfile -->|yes| L_sourceEnv["Source nix.sh from user profile if present"]

    L_sourceEnv --> M_end["Nix environment ready in container"]
Loading

File-Level Changes

Change Details Files
Bind-mount /nix from the persistent XDG data Nix store on startup and validate it is a proper directory and mount point.
  • Create the persistent Nix store directory under $XDG_DATA_HOME/nix if it does not exist.
  • Validate that /nix is either absent or a directory, failing fast if it exists as a non-directory path.
  • Ensure /nix exists and is bind-mounted from $XDG_DATA_HOME/nix only if it is not already a mount point, avoiding redundant mounts.
nix.sh
Make Nix flake configuration in /etc/nix/nix.conf idempotent.
  • Ensure /etc/nix exists before writing configuration.
  • Create /etc/nix/nix.conf with experimental-features = nix-command flakes when it does not exist.
  • Append experimental-features = nix-command flakes only when the existing configuration is missing either nix-command or flakes, avoiding duplicate entries.
nix.sh
Detect existing Nix installation via the user profile script instead of /nix and only run the installer when necessary.
  • Change the installation guard from checking for /nix existence to checking for $HOME/.nix-profile/etc/profile.d/nix.sh.
  • Keep the user-facing installation messaging and single-user Nix install invocation the same, but gate it on the new profile-based check.
  • Remove the post-install remount logic that tried to mount /nix after restart, since mounting is now handled up front.
nix.sh

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The mount | grep -q /nix check can produce false positives if another mount path contains /nix; consider matching the mountpoint more strictly (e.g. grep -q ' on /nix ' or using findmnt -M /nix if available).
  • The pattern if [ ! -e "/nix" ]; then sudo mkdir /nix; fi can still race between simultaneous containers; using sudo mkdir -p /nix and ignoring the error if it already exists would be more robust for concurrent startup.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `mount | grep -q /nix` check can produce false positives if another mount path contains `/nix`; consider matching the mountpoint more strictly (e.g. `grep -q ' on /nix '` or using `findmnt -M /nix` if available).
- The pattern `if [ ! -e "/nix" ]; then sudo mkdir /nix; fi` can still race between simultaneous containers; using `sudo mkdir -p /nix` and ignoring the error if it already exists would be more robust for concurrent startup.

## Individual Comments

### Comment 1
<location path="nix.sh" line_range="37-42" />
<code_context>
-# Install nix with flakes if no Nix store found
+# Ensure /nix is bind-mounted from persistent storage
+mkdir -p "$XDG_DATA_HOME/nix"
 if [ ! -e "/nix" ]; then
+    sudo mkdir /nix
+fi
+if ! mount | grep -q /nix; then
</code_context>
<issue_to_address>
**suggestion:** Consider handling the case where `/nix` exists but is not a directory.

`-e` will skip `mkdir` even if `/nix` is a regular file or symlink, leading to a confusing bind-mount failure. Use a directory-specific check like `if [ ! -d /nix ]; then` and explicitly error if `/nix` exists but is not a directory to make failures clearer.

```suggestion
# Ensure /nix is bind-mounted from persistent storage
mkdir -p "$XDG_DATA_HOME/nix"

# Ensure /nix exists as a directory; fail fast if it's some other type
if [ -e "/nix" ] && [ ! -d "/nix" ]; then
    echo "Error: /nix exists but is not a directory; cannot bind-mount persistent Nix store." >&2
    exit 1
fi

if [ ! -d "/nix" ]; then
    sudo mkdir -p /nix
fi

if ! mount | grep -q /nix; then
```
</issue_to_address>

### Comment 2
<location path="nix.sh" line_range="42" />
<code_context>
 if [ ! -e "/nix" ]; then
+    sudo mkdir /nix
+fi
+if ! mount | grep -q /nix; then
+    sudo mount --bind "$XDG_DATA_HOME/nix" /nix
+fi
+
</code_context>
<issue_to_address>
**suggestion (bug_risk):** The mount check via `grep -q /nix` can produce false positives.

Without anchoring, this can treat paths like `/nix-backup` as if `/nix` were mounted and skip the bind. Consider using `mountpoint -q /nix` or a stricter pattern like `mount | grep -qE ' on /nix( |$)'` to reliably detect the `/nix` mount.

```suggestion
if ! mountpoint -q /nix; then
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

- Check `/nix` is a directory, not just that it exists; error out if
  it's a file or symlink
- Use `sudo mkdir -p` to avoid race between concurrent containers
- Use `mountpoint -q /nix` instead of `mount | grep -q /nix` to avoid
  false positives from paths like `/nix-backup`

Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
@thrix
Copy link
Copy Markdown
Owner

thrix commented Apr 2, 2026

@sourcery-ai review

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The nix.conf handling only writes the file when it does not exist, so if an existing config is missing experimental-features = nix-command flakes it will never be corrected; consider appending/updating that option when the file exists instead of skipping it entirely.
  • The /nix bind mount assumes $XDG_DATA_HOME is set and valid; it may be safer to ensure a default (e.g., $HOME/.local/share) or fail with a clear error if it is unset before using it for the persistent store.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The nix.conf handling only writes the file when it does not exist, so if an existing config is missing `experimental-features = nix-command flakes` it will never be corrected; consider appending/updating that option when the file exists instead of skipping it entirely.
- The `/nix` bind mount assumes `$XDG_DATA_HOME` is set and valid; it may be safer to ensure a default (e.g., `$HOME/.local/share`) or fail with a clear error if it is unset before using it for the persistent store.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Check for the `nix-command` feature in existing `nix.conf` and rewrite
it if missing, rather than skipping when the file exists.

The `$XDG_DATA_HOME` concern from the review is already handled —
it is defaulted to `$HOME/.local/share` on line 20 before any mount
logic runs.

Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
@thrix-bot
Copy link
Copy Markdown
Collaborator Author

@sourcery-ai review

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The nix.conf handling is no longer idempotent with respect to existing user configuration: when the file exists but lacks nix-command, the script overwrites it entirely; consider appending or editing just the experimental-features line so other settings are preserved.
  • Before bind-mounting $XDG_DATA_HOME/nix, it may be worth validating that the directory exists and is writable by the current user (and perhaps ensuring a sensible default if XDG_DATA_HOME is unset) to avoid subtle permission or env-related failures across containers.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The nix.conf handling is no longer idempotent with respect to existing user configuration: when the file exists but lacks `nix-command`, the script overwrites it entirely; consider appending or editing just the `experimental-features` line so other settings are preserved.
- Before bind-mounting `$XDG_DATA_HOME/nix`, it may be worth validating that the directory exists and is writable by the current user (and perhaps ensuring a sensible default if `XDG_DATA_HOME` is unset) to avoid subtle permission or env-related failures across containers.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

When `/etc/nix/nix.conf` already exists but lacks `nix-command`,
append the experimental-features line instead of overwriting the
file to preserve any existing user configuration.

Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
@thrix-bot
Copy link
Copy Markdown
Collaborator Author

@sourcery-ai review

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The nix.conf handling only checks for nix-command and may leave a profile without flakes enabled if nix-command is present but flakes is not; consider explicitly ensuring both flags are present in the experimental-features line.
  • The bind-mount logic for /nix can still race if multiple containers start simultaneously (both might pass mountpoint -q /nix and then try to mount); consider making the mount operation tolerant of an 'already mounted' error or re-checking the mountpoint after a failed mount.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `nix.conf` handling only checks for `nix-command` and may leave a profile without `flakes` enabled if `nix-command` is present but `flakes` is not; consider explicitly ensuring both flags are present in the `experimental-features` line.
- The bind-mount logic for `/nix` can still race if multiple containers start simultaneously (both might pass `mountpoint -q /nix` and then try to mount); consider making the mount operation tolerant of an 'already mounted' error or re-checking the mountpoint after a failed mount.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

- Tolerate mount race between concurrent containers: if `mount` fails,
  verify `/nix` is already a mountpoint before erroring out
- Check for both `nix-command` and `flakes` in existing `nix.conf`
  to avoid leaving a partial configuration

Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
@thrix-bot
Copy link
Copy Markdown
Collaborator Author

All feedback addressed in 2eef746. @sourcery-ai review — please do a thorough review of the full diff.

@thrix-bot
Copy link
Copy Markdown
Collaborator Author

@sourcery-ai review

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The bind-mount logic for /nix currently swallows failures (sudo mount --bind ... || mountpoint -q /nix); it would be clearer and safer to fail fast with an explicit error if the mount command returns non-zero instead of just re-checking the mountpoint.
  • When appending the experimental-features line to /etc/nix/nix.conf, you might want to ensure you don’t end up with multiple such lines over time (e.g., by normalizing or editing the existing line rather than always appending when the words are missing).
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The bind-mount logic for /nix currently swallows failures (`sudo mount --bind ... || mountpoint -q /nix`); it would be clearer and safer to fail fast with an explicit error if the mount command returns non-zero instead of just re-checking the mountpoint.
- When appending the `experimental-features` line to `/etc/nix/nix.conf`, you might want to ensure you don’t end up with multiple such lines over time (e.g., by normalizing or editing the existing line rather than always appending when the words are missing).

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

- Replace silent mount fallback with explicit error and exit
- Remove existing `experimental-features` line before appending to
  prevent duplicates accumulating over time

Assisted-by: Claude Code
Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>
@thrix-bot
Copy link
Copy Markdown
Collaborator Author

Both comments addressed in 9c2c816:

  • Mount now fails fast with an explicit error instead of silently re-checking
  • Existing experimental-features line is removed via sed before appending to prevent duplicates

@sourcery-ai review

Assisted-by: Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants