AZ-Intec-GmbH/AZ-NIX-SERVERS
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a3d4bd7ab515433bf53c7c2046e442a994f2fa1b
AZ-NIX-SERVERS/docs/superpowers/plans/2026-06-09-frappe-bench-infrastructure.md
T

21 KiB
Raw Blame History

Frappe Bench Infrastructure Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Convert the current Frappe LMS container setup into a production-style, multisite-capable Frappe Bench backed by host MariaDB, host Redis, and agenix-managed secrets.

Architecture: MariaDB and Redis become host infrastructure modules under hosts/AZ-CLD-1/services/. The Frappe runtime becomes one shared Bench module under hosts/AZ-CLD-1/services/containers/frappe.nix, with separate backend, frontend, websocket, worker, and scheduler containers. Secrets are provided through agenix and consumed via config.age.secrets.frappe-env.path.

Tech Stack: NixOS modules, Podman OCI containers, MariaDB, Redis, Frappe Bench, Traefik, agenix, alejandra, nixos-rebuild build.

Execution constraints: Do not run nixos-rebuild switch. Do not run git push. Build-only validation is allowed.

File Structure

  • Create: hosts/AZ-CLD-1/services/mariadb.nix
    • Owns host MariaDB configuration, Frappe DB admin bootstrap, firewall rules, and MariaDB dump timer.
  • Create: hosts/AZ-CLD-1/services/redis.nix
    • Owns host Redis configuration and firewall rules.
  • Rename/create: hosts/AZ-CLD-1/services/containers/frappe.nix
    • Replaces frappe-lms.nix as the shared Bench runtime.
    • Owns Frappe directories, Bench config, site/app initialization, containers, backup timer, Traefik route, and CLI helper.
  • Modify: hosts/AZ-CLD-1/services/default.nix
    • Imports mariadb.nix and redis.nix.
  • Modify: hosts/AZ-CLD-1/services/containers/default.nix
    • Imports frappe.nix instead of the old LMS-specific file.
  • Modify: hosts/common/ports.nix
    • Renames the internal service port key from frappe-lms to frappe while preserving port 3052.
  • Modify: hosts/AZ-CLD-1/secrets.nix
    • Adds the host-local agenix secret reference frappe-env.
  • Modify: secrets.nix
    • Adds secrets/frappe-env.age to the public key registry.
  • Create: secrets/frappe-env.age
    • Encrypted agenix env file containing FRAPPE_DB_ADMIN_PASSWORD and FRAPPE_ADMIN_PASSWORD.

Task 1: Add agenix Secret Plumbing

Files:

  • Create: secrets/frappe-env.age

  • Modify: secrets.nix

  • Modify: hosts/AZ-CLD-1/secrets.nix

  • Step 1: Generate and encrypt the Frappe env secret

Run:

set -euo pipefail
db_secret_value="$(${openssl:-openssl} rand -base64 48 | tr -d '\n')"
admin_secret_value="$(${openssl:-openssl} rand -base64 32 | tr -d '\n')"
tmp_secret="$(mktemp)"
cat > "$tmp_secret" <<EOF_SECRET
FRAPPE_DB_ADMIN_PASSWORD=$db_secret_value
FRAPPE_ADMIN_PASSWORD=$admin_secret_value
EOF_SECRET
export tmp_secret
EDITOR='sh -c '\''cat "$tmp_secret" > "$1"'\'' sh' agenix -e secrets/frappe-env.age
rm -f "$tmp_secret"

Expected: secrets/frappe-env.age exists and contains encrypted data.

If openssl is not in PATH, run inside the dev shell:

nix develop
  • Step 2: Register the encrypted secret in the global agenix registry

In secrets.nix, add this line near the other secrets/*.age entries:

  "secrets/frappe-env.age".publicKeys = systems ++ users;
  • Step 3: Register the secret for AZ-CLD-1

In hosts/AZ-CLD-1/secrets.nix, add this attribute inside age.secrets:

      frappe-env = {
        file = ../../secrets/frappe-env.age;
      };
  • Step 4: Format the changed Nix files

Run:

alejandra secrets.nix hosts/AZ-CLD-1/secrets.nix

Expected: both files format successfully.

  • Step 5: Verify the secret path is visible in the host config

Run:

nix eval --raw .#nixosConfigurations.AZ-CLD-1.config.age.secrets.frappe-env.file

Expected output ends with:

secrets/frappe-env.age
  • Step 6: Commit the secret plumbing

Run:

git add secrets/frappe-env.age secrets.nix hosts/AZ-CLD-1/secrets.nix
git commit -m "secrets: add frappe env secret"

Task 2: Add Host MariaDB Infrastructure

Files:

  • Create: hosts/AZ-CLD-1/services/mariadb.nix

  • Modify: hosts/AZ-CLD-1/services/default.nix

  • Step 1: Create the MariaDB host module

Create hosts/AZ-CLD-1/services/mariadb.nix with this content:

{
  config,
  pkgs,
  ...
}: let
  frappeEnvFile = config.age.secrets.frappe-env.path;
  frappeDbAdminUser = "frappe_admin";

  waitForMariaDb = ''
    mariadb_ready=0
    for attempt in $(${pkgs.coreutils}/bin/seq 1 60); do
      if ${pkgs.mariadb_118}/bin/mysql -u root -N -e 'SELECT 1' >/dev/null 2>&1; then
        mariadb_ready=1
        break
      fi
      ${pkgs.coreutils}/bin/sleep 2
    done

    if [ "$mariadb_ready" != "1" ]; then
      echo "MariaDB did not become ready in time" >&2
      exit 1
    fi
  '';
in {
  services.mysql = {
    enable = true;
    package = pkgs.mariadb_118;
    settings.mysqld = {
      bind-address = "0.0.0.0";
      character-set-server = "utf8mb4";
      collation-server = "utf8mb4_unicode_ci";
      skip-character-set-client-handshake = true;
      skip-innodb-read-only-compressed = true;
    };
  };

  networking.firewall.extraCommands = ''
    iptables -A INPUT -p tcp -s 127.0.0.1 --dport 3306 -j ACCEPT
    iptables -A INPUT -p tcp -s 10.89.0.0/24 --dport 3306 -j ACCEPT
  '';

  systemd.services.mariadb-frappe-admin-init = {
    description = "Create shared Frappe MariaDB admin user";
    wantedBy = ["multi-user.target"];
    after = ["mysql.service"];
    requires = ["mysql.service"];

    serviceConfig = {
      Type = "oneshot";
      RemainAfterExit = true;
      User = "root";
      Group = "root";
    };

    script = ''
      set -euo pipefail

      set -a
      source ${frappeEnvFile}
      set +a

      : "''${FRAPPE_DB_ADMIN_PASSWORD:?FRAPPE_DB_ADMIN_PASSWORD is missing in ${frappeEnvFile}}"

      ${waitForMariaDb}

      escaped_db_admin_password=$(${pkgs.coreutils}/bin/printf '%s' "$FRAPPE_DB_ADMIN_PASSWORD" | ${pkgs.python3}/bin/python3 -c 'import sys; print(sys.stdin.read().replace(chr(39), chr(39) * 2), end="")')

      ${pkgs.mariadb_118}/bin/mysql -u root <<SQL
CREATE USER IF NOT EXISTS '${frappeDbAdminUser}'@'10.89.0.%' IDENTIFIED BY '$escaped_db_admin_password';
ALTER USER '${frappeDbAdminUser}'@'10.89.0.%' IDENTIFIED BY '$escaped_db_admin_password';
GRANT ALL PRIVILEGES ON *.* TO '${frappeDbAdminUser}'@'10.89.0.%' WITH GRANT OPTION;
FLUSH PRIVILEGES;
SQL
    '';
  };

  systemd.services.mariadb-backup-all = {
    description = "Create full MariaDB dump backup";

    serviceConfig = {
      Type = "oneshot";
      User = "root";
      Group = "root";
    };

    script = ''
      set -euo pipefail

      backup_dir=/var/backup/mariadb
      timestamp="$(${pkgs.coreutils}/bin/date +%Y%m%d-%H%M%S)"
      ${pkgs.coreutils}/bin/mkdir -p "$backup_dir"
      ${pkgs.coreutils}/bin/chown root:root "$backup_dir"
      ${pkgs.coreutils}/bin/chmod 0750 "$backup_dir"

      ${pkgs.mariadb_118}/bin/mariadb-dump --all-databases --single-transaction --quick --lock-tables=false \
        | ${pkgs.gzip}/bin/gzip -c > "$backup_dir/all-$timestamp.sql.gz"

      ${pkgs.findutils}/bin/find "$backup_dir" -type f -name 'all-*.sql.gz' -mtime +14 -delete
    '';
  };

  systemd.timers.mariadb-backup-all = {
    wantedBy = ["timers.target"];
    timerConfig = {
      OnCalendar = "*-*-* 03:20:00";
      RandomizedDelaySec = "30m";
      Persistent = true;
    };
  };
}
  • Step 2: Import the MariaDB module

In hosts/AZ-CLD-1/services/default.nix, add ./mariadb.nix near ./postgres.nix:

    ./mariadb.nix
    ./postgres.nix
  • Step 3: Format and evaluate MariaDB config

Run:

alejandra hosts/AZ-CLD-1/services/mariadb.nix hosts/AZ-CLD-1/services/default.nix
nix eval .#nixosConfigurations.AZ-CLD-1.config.services.mysql.enable

Expected output:

true
  • Step 4: Verify MariaDB is no longer planned to live in Frappe

Run:

rg -n "services\.mysql|mariadb-frappe-admin-init" hosts/AZ-CLD-1/services

Expected:

  • services.mysql appears in hosts/AZ-CLD-1/services/mariadb.nix only.

  • mariadb-frappe-admin-init appears in mariadb.nix and future Frappe ordering references only.

  • Step 5: Commit MariaDB infrastructure

Run:

git add hosts/AZ-CLD-1/services/mariadb.nix hosts/AZ-CLD-1/services/default.nix
git commit -m "services: add host mariadb for frappe"

Task 3: Add Host Redis Infrastructure

Files:

  • Create: hosts/AZ-CLD-1/services/redis.nix

  • Modify: hosts/AZ-CLD-1/services/default.nix

  • Step 1: Create the Redis host module

Create hosts/AZ-CLD-1/services/redis.nix with this content:

{...}: {
  services.redis.servers."" = {
    enable = true;
    port = 6379;
    bind = null;
    databases = 16;
    appendOnly = true;
    appendFsync = "everysec";
    settings = {
      protected-mode = "no";
    };
  };

  networking.firewall.extraCommands = ''
    iptables -A INPUT -p tcp -s 127.0.0.1 --dport 6379 -j ACCEPT
    iptables -A INPUT -p tcp -s 10.89.0.0/24 --dport 6379 -j ACCEPT
  '';
}
  • Step 2: Import the Redis module

In hosts/AZ-CLD-1/services/default.nix, add ./redis.nix near the other infrastructure services:

    ./redis.nix
  • Step 3: Format and evaluate Redis config

Run:

alejandra hosts/AZ-CLD-1/services/redis.nix hosts/AZ-CLD-1/services/default.nix
nix eval .#nixosConfigurations.AZ-CLD-1.config.services.redis.servers."".enable

Expected output:

true
  • Step 4: Commit Redis infrastructure

Run:

git add hosts/AZ-CLD-1/services/redis.nix hosts/AZ-CLD-1/services/default.nix
git commit -m "services: add host redis"

Task 4: Rename and Reshape the Frappe Bench Module

Files:

  • Rename/create: hosts/AZ-CLD-1/services/containers/frappe.nix

  • Remove from imports: hosts/AZ-CLD-1/services/containers/frappe-lms.nix

  • Modify: hosts/common/ports.nix

  • Step 1: Rename the Frappe module file

Run:

if [ -f hosts/AZ-CLD-1/services/containers/frappe-lms.nix ]; then
  mv hosts/AZ-CLD-1/services/containers/frappe-lms.nix hosts/AZ-CLD-1/services/containers/frappe.nix
fi

Expected: hosts/AZ-CLD-1/services/containers/frappe.nix exists.

  • Step 2: Rename the service port key

In hosts/common/ports.nix, replace:

      frappe-lms = 3052;

with:

      frappe = 3052;
  • Step 3: Replace the top-level Frappe constants

At the top of hosts/AZ-CLD-1/services/containers/frappe.nix, use this structure in the let block:

  serviceName = "frappe";
  servicePort = config.m3ta.ports.get serviceName;

  primaryDomain = "learn.az-gruppe.com";
  image = "ghcr.io/frappe/lms:stable";

  dataDir = "/var/lib/${serviceName}";
  sitesDir = "${dataDir}/sites";
  frappeEnvFile = config.age.secrets.frappe-env.path;

  dbHost = "10.89.0.1";
  dbPort = "3306";
  dbAdminUser = "frappe_admin";

  redisCacheUrl = "redis://10.89.0.1:6379/1";
  redisQueueUrl = "redis://10.89.0.1:6379/2";

  sites = {
    "learn.az-gruppe.com" = {
      apps = ["lms"];
      adminPasswordEnv = "FRAPPE_ADMIN_PASSWORD";
    };
  };

  siteNames = builtins.attrNames sites;
  • Step 4: Remove LMS-specific MariaDB and Redis definitions

Delete these pieces from hosts/AZ-CLD-1/services/containers/frappe.nix:

  redisCacheIp = "${ipBase}.71";
  redisQueueIp = "${ipBase}.72";
  redisCacheName = "${serviceName}-redis-cache";
  redisQueueName = "${serviceName}-redis-queue";

Delete the entire local services.mysql = { ... }; block.

Delete the container definitions for:

frappe-lms-redis-cache
frappe-lms-redis-queue
  • Step 5: Update shared app base dependencies

In appBase, remove container Redis dependencies so it looks like this:

  appBase = {
    inherit image;
    autoStart = true;
    volumes = ["${sitesDir}:/home/frappe/frappe-bench/sites"];
  };
  • Step 6: Change the directory preparation service

Update the directory service so it no longer checks a manual env file under /var/lib. Its script should be:

    script = ''
      set -euo pipefail

      ${pkgs.coreutils}/bin/mkdir -p ${dataDir} ${sitesDir}
      ${pkgs.coreutils}/bin/chown root:root ${dataDir}
      ${pkgs.coreutils}/bin/chmod 0750 ${dataDir}
      ${pkgs.coreutils}/bin/chown -R 1000:1000 ${sitesDir}
      ${pkgs.coreutils}/bin/chmod 0750 ${sitesDir}
    '';
  • Step 7: Update configurator ordering and Redis config

The configurator service must order after host MariaDB admin init and host Redis:

    after = [
      "mariadb-frappe-admin-init.service"
      "redis.service"
      "${serviceName}-directories.service"
    ];
    requires = [
      "mariadb-frappe-admin-init.service"
      "redis.service"
      "${serviceName}-directories.service"
    ];

Inside the bench set-config commands, use host Redis URLs:

          bench set-config -g db_host ${dbHost}
          bench set-config -gp db_port ${dbPort}
          bench set-config -g redis_cache ${redisCacheUrl}
          bench set-config -g redis_queue ${redisQueueUrl}
          bench set-config -g redis_socketio ${redisQueueUrl}
          bench set-config -gp socketio_port 9000
          bench set-config -g chromium_path /usr/bin/chromium-headless-shell
  • Step 8: Replace site init with agenix-based multisite init

In ${serviceName}-site-init, source the agenix env file:

      set -a
      source ${frappeEnvFile}
      set +a

      : "''${FRAPPE_DB_ADMIN_PASSWORD:?FRAPPE_DB_ADMIN_PASSWORD is missing in ${frappeEnvFile}}"

For each configured site, use this shell pattern generated from the sites attribute set:

      : "''${FRAPPE_ADMIN_PASSWORD:?FRAPPE_ADMIN_PASSWORD is missing in ${frappeEnvFile}}"

      if [ ! -f ${sitesDir}/learn.az-gruppe.com/site_config.json ]; then
        ${pkgs.podman}/bin/podman run --rm \
          --name ${serviceName}-site-init-learn \
          --network web \
          --ip ${transientIp} \
          --env-file ${frappeEnvFile} \
          --env SITE_NAME=learn.az-gruppe.com \
          -v ${sitesDir}:/home/frappe/frappe-bench/sites \
          ${image} \
          bash -c '
            set -euo pipefail
            bench new-site \
              --db-host ${dbHost} \
              --db-port ${dbPort} \
              --mariadb-user-host-login-scope=% \
              --db-root-username ${dbAdminUser} \
              --db-root-password "$FRAPPE_DB_ADMIN_PASSWORD" \
              --admin-password "$FRAPPE_ADMIN_PASSWORD" \
              "$SITE_NAME"
          '
      else
        echo "Frappe site learn.az-gruppe.com already exists"
      fi

      for app in lms; do
        ${pkgs.podman}/bin/podman run --rm \
          --name ${serviceName}-install-app-$app \
          --network web \
          --ip ${transientIp} \
          --env SITE_NAME=learn.az-gruppe.com \
          --env APP_NAME="$app" \
          -v ${sitesDir}:/home/frappe/frappe-bench/sites \
          ${image} \
          bash -c '
            set -euo pipefail
            if bench --site "$SITE_NAME" list-apps | grep -Fxq "$APP_NAME"; then
              echo "App $APP_NAME already installed on $SITE_NAME"
            else
              bench --site "$SITE_NAME" install-app "$APP_NAME"
            fi
          '
      done

After this works for learn.az-gruppe.com, convert the repeated site/app shell to generated Nix using lib.concatMapStringsSep and lib.escapeShellArg so future sites can be added by extending sites.

  • Step 9: Update container names and service references

Ensure the container names are:

frappe-backend
frappe-frontend
frappe-websocket
frappe-queue-short
frappe-queue-long
frappe-scheduler

Ensure no names retain the frappe-lms- prefix.

  • Step 10: Update systemd ordering for all Frappe containers

Every runtime container override must require site init and host Redis:

  systemd.services."podman-${backendName}" = {
    after = ["${serviceName}-site-init.service" "redis.service"];
    requires = ["${serviceName}-site-init.service" "redis.service"];
  };

Apply the same pattern to frontend, websocket, queue-short, queue-long, and scheduler.

  • Step 11: Update the CLI helper name and status output

The package name must be frappe via serviceName = "frappe". The status command must check:

          systemctl status \
            mysql.service \
            redis.service \
            mariadb-frappe-admin-init.service \
            ${serviceName}-configurator.service \
            ${serviceName}-site-init.service \
            podman-${backendName}.service \
            podman-${frontendName}.service \
            podman-${websocketName}.service \
            podman-${queueShortName}.service \
            podman-${queueLongName}.service \
            podman-${schedulerName}.service
  • Step 12: Format and statically verify Frappe changes

Run:

alejandra hosts/AZ-CLD-1/services/containers/frappe.nix hosts/common/ports.nix
rg -n "frappe-lms|redis-cache|redis-queue|services\.mysql|/var/lib/frappe.*\.env" hosts/AZ-CLD-1/services/containers/frappe.nix hosts/common/ports.nix || true

Expected: no matches for removed local Redis containers, local MySQL service, or manual /var/lib/frappe/*.env secret file.

  • Step 13: Commit Frappe reshape

Run:

git add hosts/AZ-CLD-1/services/containers/frappe.nix hosts/AZ-CLD-1/services/containers/frappe-lms.nix hosts/common/ports.nix
git commit -m "services: reshape frappe as shared bench"

Task 5: Enable the Frappe Bench Import and Build-Test

Files:

  • Modify: hosts/AZ-CLD-1/services/containers/default.nix

  • Step 1: Import the new Frappe module

In hosts/AZ-CLD-1/services/containers/default.nix, replace:

    # ./frappe-lms.nix

with:

    ./frappe.nix
  • Step 2: Format the container import file

Run:

alejandra hosts/AZ-CLD-1/services/containers/default.nix

Expected: formatting succeeds.

  • Step 3: Run focused Nix evaluations

Run:

nix eval --raw .#nixosConfigurations.AZ-CLD-1.config.services.traefik.dynamicConfigOptions.http.routers.frappe.rule
nix eval .#nixosConfigurations.AZ-CLD-1.config.virtualisation.oci-containers.containers.frappe-backend.autoStart
nix eval .#nixosConfigurations.AZ-CLD-1.config.services.redis.servers."".enable
nix eval .#nixosConfigurations.AZ-CLD-1.config.services.mysql.enable

Expected outputs:

Host(`learn.az-gruppe.com`)
true
true
true
  • Step 4: Run repository quality checks

Run:

nix flake check

Expected: command exits with status 0.

  • Step 5: Run the allowed build test

Run:

nixos-rebuild build --flake .#AZ-CLD-1

Expected: command exits with status 0 and creates/updates ./result.

Do not run nixos-rebuild switch.

  • Step 6: Commit the activation and build-tested state

Run:

git add hosts/AZ-CLD-1/services/containers/default.nix
git commit -m "services: enable frappe bench"

Task 6: Final Static Verification and Handoff

Files:

  • Read-only verification across the repository.

  • Step 1: Verify no old Frappe LMS structural names remain

Run:

rg -n "frappe-lms|FRAPPE_LMS|frappe_lms|redis-cache|redis-queue" hosts secrets.nix secrets || true

Expected: no matches except historical documentation under docs/superpowers/specs/ or docs/superpowers/plans/.

  • Step 2: Verify no prohibited commands were run

Run:

git log --oneline -5

Expected: recent commits are local only. Do not run git push.

  • Step 3: Verify worktree state

Run:

git status --short

Expected: no unstaged implementation changes. result may appear depending on local .gitignore; remove it with rm result if it is an untracked symlink.

  • Step 4: Write handoff note in final response

Include:

Implemented host MariaDB, host Redis, agenix Frappe secrets, and shared Frappe Bench module.
Validation run: nix flake check; nixos-rebuild build --flake .#AZ-CLD-1.
Not run: nixos-rebuild switch; git push.

Self-Review

Spec coverage:

  • Host Redis: Task 3.
  • Host MariaDB: Task 2.
  • Compose MariaDB settings: Task 2 Step 1.
  • agenix secrets: Task 1 and Task 4 Step 8.
  • Rename frappe-lms.nix to frappe.nix: Task 4 Step 1.
  • Multisite-capable Bench: Task 4 Steps 3 and 8.
  • Production-style separated containers: Task 4 Steps 9 and 10.
  • No switch, no push, build allowed: header and Task 5 Step 5.

Placeholder scan:

  • The plan contains no placeholder markers and no intentionally blank implementation steps.

Type/name consistency:

  • Secret name: frappe-env everywhere.
  • DB admin user: frappe_admin everywhere.
  • Service name: frappe everywhere after rename.
  • Redis service: default NixOS Redis instance, systemd unit redis.service.
Reference in New Issue View Git Blame Copy Permalink