Ansible/scripts/addons/webgui/ADDON-INSTALLATION.md

AIM WebGUI Add-on Installation

Applies to: AIM WebGUI 2.1.0rc9
Required AIM Core: 3.3.0rc8
Core service / wire / event API: 1.0
Canonical Ansible Core: 2.19.11, provided by the separately installed Core environment
WebGUI HTTP API / database: v2 / SQLite schema 5
WebGUI Python: Python 3.11+ with virtual-environment support
Managed deployment: Linux with systemd
Prerequisite: AIM Core 3.3.0rc8 scripts/docs/INSTALLATION.md (separately supplied Core guide)

This guide continues after the Core installation guide. It installs a new AIM WebGUI add-on on an already configured AIM controller; it does not install AIM Core again. Complete Core installation, establish approved customer data, and accept a controlled terminal run before enabling WebGUI execution. Core remains usable without this add-on. [C1][C2]

For an existing managed WebGUI installation, use section 15, not the fresh-install command. For a controller already running this exact add-on version, use the verification and acceptance sections; the deployer rejects a same-version reinstall. [W2]

Reading order: fresh installations follow sections 1-13; backup, updates and recovery are in sections 14-17. The completion checklist is at the end.

The commands below use /etc/ansible, the existing executor account svc_bf-ansible, and the default web account aim-web. These are the current release profile, not requirements to rename an established controller account. This document describes a release candidate and a source-checked procedure, not evidence that a new controller has passed live acceptance.

1. Complete the Core prerequisite first

Use Core's INSTALLATION.md for the source installation, dependency environments, canonical Ansible runtime, collections, controller configuration, inventories, Vaults and keys. Do not substitute the WebGUI virtual environment for either the AIM or Ansible environment. [C1, sections 1-12]

Before continuing, establish:

• aim and aimctl refer to the intended Core 3.3.0rc8 installation.
• The AIM Python environment contains its declared dependencies, including the documented rich>=13,<15 range.
• The separately configured Ansible runtime contains exactly Core 2.19.11 and the approved collection/connection dependencies needed by the selected playbooks.
• Core 3.3.0rc8 requires ansible.windows >=3.8.0,<4.0.0 for Windows playbooks. Upgrade that collection deliberately in the canonical Core/Ansible collection path before accepting rc8; WebGUI does not install or upgrade it.
• At least one approved test customer/host is available for the terminal and add-on acceptance run.
• The non-root execution account is authorized by Core and has the required filesystem access.
• Terminal AIM has completed an approved low-risk test independently of WebGUI.

Check the installed commands:

command -v aim
command -v aimctl
aim --version
aimctl --version

Both Core commands must report 3.3.0rc8. A generic API version of 1.0 alone is not enough: this WebGUI candidate also checks its explicitly supported Core product version and capabilities. [W2][W3]

For a fresh setup, leave this existing Core setting disabled while provisioning the add-on:

addons:
  execution_enabled: false

Merge configuration changes into /etc/ansible/scripts/aim.yml; do not replace the entire file. The final opt-in is covered in section 10. Discovery and installation of the add-on do not grant permission to execute jobs. [C1, sections 8 and 13]

2. Confirm the two local service identities

The managed topology uses two different non-root accounts: [W2][W4]

Component	Default account	Responsibility
aim-web.service	aim-web	HTTP, sessions, UI and private workflow database
aim-web-worker.service	aim-web	Queue, authorization rechecks, journal/report persistence and credential handoff
aim-web-executor.service	svc_bf-ansible	Fixed public aimctl calls and Core-owned native execution under that same UID

The WebGUI installer can create the local aim-web system account and group when absent. It does not create the Core executor account or change its persistent group memberships. The executor must already exist, must not be root, and must differ from the web account. [W2]

Inspect the approved executor:

getent passwd svc_bf-ansible
id svc_bf-ansible

The passwd/NSS home must be a reviewed, usable account home. The installer will create or normalize its .ansible and .ansible/tmp directories to executor-owned 0700, in addition to the separate /var/lib/aim-web-executor tree. Review pre-existing shared uses or symlinks before applying; do not use installation as a general home-directory repair procedure. [W2][W4]

Review Core's configured required_group and the account's active local/NSS memberships. Do not assume the example directory group in the Core installation guide or the local aim-operators group exists at every site. Provision authorization through the approved local/directory administration process, not through an add-on workaround. [C1][C2]

The account needs access to the configured Core launcher/runtime, customer inventory and encrypted Vault, required role payloads and collections, and Core's cooperating customer-lock paths. In customer-key mode, the canonical private key must be an owner-only 0600 file owned by the execution UID. A key filename matching the account name does not establish ownership. [C2][C3]

service_user is not a local UID switch. runtime.private_key_owner controls newly generated keys; it does not migrate an existing root-owned key or grant access. Existing-key migration is a separate, explicit Core/operator decision. Do not recursively change /etc/ansible, make private keys group-readable, or add the web account to root/operator groups to bypass this prerequisite. [C1, section 8][C2]

The managed executor preserves its native primary group and receives the aim-web group for this service only. A normal interactive sudo -u svc_bf-ansible session therefore need not be able to read WebGUI's root:aim-web 0640 configuration. That is not a reason to make it world-readable. [W4]

3. Verify Core access as the executor

First check metadata through the installed public launcher:

sudo -u svc_bf-ansible \
  /usr/local/bin/aimctl \
  --config /etc/ansible/scripts/aim.yml \
  capabilities

Then check a protected operation:

printf '%s\n' '{"api_version":"1.0","operation":"list_customers"}' \
  | sudo -u svc_bf-ansible \
      /usr/local/bin/aimctl \
      --config /etc/ansible/scripts/aim.yml \
      request

capabilities is unguarded metadata; success there does not prove authorization. The customer-list response must also succeed. An empty list can be a valid discovery result, but it is not a completed managed-host acceptance test. [C3]

The WebGUI integration expects the detailed progress, inventory hierarchy, target outcome, dual-home staging and operation-result contracts described by the current Core documentation. The installer probes live Core capabilities and protected customer discovery; do not edit a support snapshot or disable version checks to make an incompatible pair install. [C2][W2][W3]

Confirm runtime and collection visibility

Use the executable configured in Core's runtime.ansible_playbook. For the separate environment shown in Core's fresh-install guide, the checks are:

sudo -u svc_bf-ansible \
  /opt/ansible/venv/bin/ansible-playbook --version

sudo -u svc_bf-ansible \
  /opt/ansible/venv/bin/ansible-galaxy collection list
  /opt/ansible/venv/bin/ansible-galaxy collection list ansible.windows

If Core was accepted with /usr/bin/ansible-playbook, use its corresponding runtime instead; do not create a replacement just for WebGUI. Sibling Vault/Galaxy tools and collection discovery must match the approved native runtime. [C1, sections 4-5][C3]

For rc8, confirm the discovered ansible.windows version is at least 3.8.0 and below 4.0.0. Core advertises this floor in capabilities.collection_baselines and its readiness checks reject older versions before an affected playbook launches. Do not satisfy the check by installing a private collection visible only to root or to the WebGUI virtual environment.

The collection-install command in the Core guide uses sudo. Verify discovery under the executor as well as root. Collections available only in root's private home do not establish executor access. Use Core's documented runtime.ansible_collections_path when an approved shared/nonstandard collection location is needed; WebGUI does not install collections or expose root's home. [C1, section 5][C3]

These shell checks do not reproduce systemd restrictions. The installed executor's startup preflight and the real one-host test are still required.

4. Review HTTPS, SSH trust and the shipped site profile

HTTPS and reverse proxy are prerequisites

The WebGUI deployer does not install Nginx, Nginx Proxy Manager, certificates, CA trust or DNS. It serves HTTP on its configured backend listener; the approved reverse proxy provides the browser's HTTPS endpoint. [W1][W2]

The bundled deploy/webgui.example.toml contains this controller-specific profile:

Setting	Shipped value
Backend	127.0.0.1:8080
Browser origin	https://aim.desq-gaming.de
Immediate trusted proxy	127.0.0.1
Referenced upstream topology	NPM 192.168.20.3 -> verified HTTPS controller 192.168.20.46:8443 -> local Nginx -> loopback HTTP
WebGUI execution / credential entry	Enabled
Independent approval	Disabled
Eligible playbooks	The 14 explicit catalog keys in the shipped profile
Maximum requested hosts / execution timeout	25 / 1,800 seconds
transport_verified	true, representing this site's operator attestation

An allowlist does not establish that every playbook is executable or appropriate at the site. transport_verified=true does not provision or test TLS. Review the actual proxy path and trust before using these values. [W1][W5]

The public_url must be the exact browser origin: scheme, hostname and optional port, with no path or trailing slash. The trusted-proxy list identifies the immediate backend proxy, not the browser's remote address. Execution rejects wildcard proxy trust. [W5]

deploy/nginx.example.conf is a header reference, not an installable complete TLS server configuration. Preserve the browser Host and forwarded scheme through the approved proxy chain and verify streaming. WebGUI uses authenticated SSE, not WebSockets. No certificate setup or unverified replacement proxy configuration is supplied by this guide. [W1][W5][W9]

Other sites: an explicit configuration gate

Do not treat the shipped domain, IP addresses or transport attestation as generic defaults.

The current deployer can select the Core launcher/configuration and account names, but has no --profile, --config-file or public-origin override argument. It replaces the whole installed TOML from its release template during install/update and restores the checkpoint's configuration on rollback. Editing the extracted template invalidates its manifest. [W2]

For repeatable release-managed deployment, obtain a reviewed release profile matching the new site. An operator can edit the installed /etc/ansible/scripts/config/webgui.toml before exposing it, but that is a local override, not a durable site-profile mechanism; the next managed update replaces it. Keep Core execution disabled and public ingress closed during any such adaptation. Preserve the managed state/socket paths and review account/path choices rather than copying a complete unrelated configuration.

If credentials or execution are disabled locally, disable [credentials].enabled before or together with [execution].enabled. The configuration validator does not allow credential entry without enabled execution and verified HTTPS. Run sudo aim-web config-check after editing; it validates settings, not certificates. Configuration is loaded by the service processes: apply edits in a quiesced window and restart the appropriate services before exposing the site. Leave the worker stopped/disabled when execution is intentionally disabled. [W5]

SSH host trust is a separate prerequisite

For Linux/SSH targets, the executor needs independently verified host trust for the effective connection destination. The inventory's logical hostname may differ from the actual ansible_host address.

The existing approved controller-wide /etc/ssh/ssh_known_hosts arrangement can be retained. WebGUI does not enroll keys, overwrite that file or automatically copy root's personal trust decisions. Do not infer OpenSSH's effective user known-hosts path solely from the executor's substituted HOME. Review the effective SSH configuration and account context instead. [W4]

Windows WinRM connectivity, credentials, transport dependencies and certificate policy remain the independently accepted Core/operator configuration. Do not disable TLS or SSH verification to pass add-on acceptance. [C1][C3]

5. Obtain and verify the WebGUI release

Transfer both files through a trusted channel:

AIM-WebGUI-2.1.0rc9.zip
AIM-WebGUI-2.1.0rc9.zip.sha256

Stage outside the active add-on and Core source directories:

cd /var/tmp
sha256sum -c AIM-WebGUI-2.1.0rc9.zip.sha256
unzip AIM-WebGUI-2.1.0rc9.zip
cd aim-web-2.1.0rc9
sha256sum -c MANIFEST.sha256

The expected ZIP SHA-256 is:

bf0a6977d2672477bcb48c78b49085ff79e70195e780813b934d87ad334d7556

Use a new extraction directory, not a directory containing an older release. Never extract over /etc/ansible/scripts/addons/webgui or /opt/aim-web/current. A checksum verifies integrity, not publisher identity. The deployer additionally verifies manifest hashes, safe paths and required-file coverage. [W1][W2]

Do not edit manifest-covered files, remove integrity checks or regenerate a manifest just to bypass a deployment error. This documentation companion is not an updated release ZIP and should be kept outside the verified extraction.

6. Prepare WebGUI dependencies and inspect the deployer

The controller must provide Linux/systemd, Python 3.11+ with venv support and administrative permission to perform installation. Inspect the local Python before running the installer:

python3 --version
python3 -m venv --help
python3 deploy/deploy.py --help

The WebGUI deployer creates its own versioned virtual environment and installs the declared WebGUI/build dependencies using the supplied constraints.txt. It does not run pip in AIM's environment, install Ansible, or install the test dependency extra. There is no need to pre-install FastAPI or pytest globally. [W2]

If the deployment interpreter is not the Python intended for WebGUI, use the supported --python /absolute/path/to/python3 option. Do not point it at an interpreter with an unsupported version. Separate environments can use the same Python installation without sharing package environments.

The package is not a complete offline dependency bundle. A new installation needs approved package/asset access or prepared offline artifacts. Browser assets are the exact pinned Bootstrap 5.3.8 CSS and HTMX 2.0.10 JavaScript; the installer verifies their SHA-384 pins and serves them locally. Do not substitute fixture assets or a newer version. [W1][W8]

For an offline installation, prepare wheels compatible with the chosen Python/OS/architecture, including constrained build dependencies, and an asset directory containing:

bootstrap.min.css
htmx.min.js

Pass the existing --wheelhouse and --assets-dir options in section 7. The detailed wheelhouse preparation depends on the operator's build environment; the supplied guide does not establish a universal offline-build recipe. [W1][W2]

Core and WebGUI deployers have different commands

Operation	Core guide	WebGUI 2.1.0rc9
Preview	install --dry-run	No equivalent dry-run mode
Fresh apply	install --apply --quiesced	install performs installation
Existing add-on update	Not applicable	update
Rollback selector	--from-backup /path	--backup SNAPSHOT_ID

Do not append Core's --dry-run, --apply, --quiesced or --aim-python options to the WebGUI deployer. Review its help, profile and source/documentation before applying. Its internal checks are not a whole-system no-write preview: preparatory add-on directories, dependencies or the web account may be created before a later failure. [C1][W2]

An existing unrecognized installation, unrelated CLI link or unmanaged systemd drop-in stops deployment. Review and back up such files; do not delete overrides or disable guard checks blindly.

7. Apply a fresh add-on installation

Proceed only after the preceding Core, identity, profile and transport gates are satisfied. Do not run a Core source update or change its inventory/configuration concurrently.

From the verified fresh extraction:

sudo python3 deploy/deploy.py install \
  --aim-scripts /etc/ansible/scripts \
  --service-user aim-web \
  --executor-user svc_bf-ansible \
  --aimctl /usr/local/bin/aimctl \
  --core-config /etc/ansible/scripts/aim.yml

These options make the default binding explicit. The Core account and launcher must already exist. [W2]

For the same installation using prepared offline artifacts, add:

sudo python3 deploy/deploy.py install \
  --aim-scripts /etc/ansible/scripts \
  --service-user aim-web \
  --executor-user svc_bf-ansible \
  --aimctl /usr/local/bin/aimctl \
  --core-config /etc/ansible/scripts/aim.yml \
  --wheelhouse /secure/wheelhouse \
  --assets-dir /secure/aim-assets

The successful fresh-install sequence: [W2]

1. Verify the release, stage dependencies/assets and probe public Core metadata as the executor.
2. Create the web account if needed; create private WebGUI state and a protected deployment checkpoint.
3. Install the reviewed TOML and provision the managed executor staging directories.
4. Initialize schema 5 and the initial local administrator without resetting existing initialized accounts.
5. Activate the full source/virtualenv and the aim-web launcher; write the three managed service units.
6. Run executor startup checks, wait for its correctly permissioned socket, start HTTP and its local readiness check, and start the worker when the WebGUI execution setting is enabled.

The deployer does not change Core's addons.execution_enabled, Core source, customer keys/Vaults, persistent executor group memberships, SSH trust or proxy certificates. The shipped profile starts the queue worker even when Core's independent execution opt-in is still false; that does not bypass the Core gate.

Record the printed add-on source, configuration, checkpoint identifier and initial-credential location. They are needed for verification and recovery.

Nondefault layouts

--aim-scripts, --aimctl and --core-config can bind the add-on to reviewed locations. They do not create an independent second WebGUI instance: /opt/aim-web, state paths, service names and the executor socket are fixed in this managed profile. The executor's inventory write exception also names /etc/ansible/inventories. A different inventory root or parallel installation needs a reviewed deployment profile; changing the CLI paths alone is insufficient. [W2]

8. Verify the installed services and permissions

Refresh command discovery:

hash -r
command -v aim-web
aim-web --version
sudo aim-web config-check

Expected version:

AIM WebGUI 2.1.0rc9 (AIM compatibility: 3.3.0rc8 / service API 1.0)

The installer supplies /usr/local/bin/aim-web -> /opt/aim-web/current/bin/aim-web; do not manually copy a versioned launcher over it. config-check validates TOML only and can run as root without opening the workflow database. [W2][W6]

Check services and the actual startup preflight:

sudo systemctl status \
  aim-web-executor.service \
  aim-web.service \
  aim-web-worker.service \
  --no-pager

sudo journalctl -u aim-web-executor.service -n 80 --no-pager

systemctl show aim-web-executor.service \
  -p User -p Group -p SupplementaryGroups

The intended executor identity is:

User=svc_bf-ansible
Group=svc_bf-ansible
SupplementaryGroups=aim-web

Numeric group IDs in systemctl show are acceptable when they resolve to the intended groups. Systemd may also initialize other account memberships; an empty explicit SupplementaryGroups field is not proof that the process has no supplementary groups. [W4]

Inspect the managed paths, substituting the actual passwd/NSS home if different:

sudo stat -c '%U:%G %a %n' \
  /var/lib/aim/webgui \
  /var/lib/aim/webgui/webgui.sqlite3 \
  /var/lib/aim-web-executor \
  /var/lib/aim-web-executor/.ansible \
  /var/lib/aim-web-executor/.ansible/tmp \
  /home/svc_bf-ansible/.ansible \
  /home/svc_bf-ansible/.ansible/tmp \
  /run/aim-web-executor \
  /run/aim-web-executor/core.sock \
  /etc/ansible/scripts/config/webgui.toml

Resource	Expected owner/group	Mode
WebGUI state directory	aim-web:aim-web	0700
Workflow database	aim-web:aim-web	0600
Executor state and process-home .ansible/tmp hierarchy	Executor:native primary group	0700
Executor passwd-home .ansible and .ansible/tmp	Executor:native primary group	0700
/run/aim-web-executor	Executor:native primary group	0711
core.sock	Executor:aim-web	0660
webgui.toml	root:aim-web	0640
Managed unit files	root:root	0644

The runtime-directory traverse permission does not grant access to the private executor state or database. Socket permissions and peer checks govern IPC. These are checks of installer-managed state, not a recursive repair recipe. [W2][W4]

Both staging homes matter

The executor sets HOME=/var/lib/aim-web-executor, but delegated local Ansible tasks can expand ~svc_bf-ansible through the account database. The installer provisions both applicable staging paths and only their narrow write exceptions while retaining ProtectHome=read-only, ProtectSystem=strict, NoNewPrivileges=true and empty capabilities. [C1, section 11][W4]

ExecStartPre runs aim-web core-staging-check inside the actual executor unit. Successful journal output followed by service startup is the relevant staging evidence. A plain interactive sudo -u svc_bf-ansible aim-web core-staging-check can fail to read the web-group configuration because it does not reproduce the unit's group context. A root invocation tests the wrong key/staging identity. Do not weaken file permissions to make those commands pass. [W4][W6]

9. Open the HTTPS site and initialize administration

Verify the approved proxy endpoint serves the correct installation with trusted HTTPS. For the shipped profile, the browser origin is:

https://aim.desq-gaming.de

Do not use a raw LAN HTTP address as the browser origin or use disabled certificate validation as acceptance. No operational secret should be entered before transport has been accepted.

Read the generated bootstrap credential file locally:

sudo cat /var/lib/aim/webgui/.credentials

The fresh administrator username is admin. The JSON file contains the generated password and is protected as service-owned private state; it is not served over HTTP. Do not paste it into tickets, chat, deployment logs or this documentation. [W6][W7]

Sign in and perform the mandatory first password change. The application revokes the sessions for that account and invalidates/removes the bootstrap credential file after the password change. Sign in again with the new password as prompted. A later update or init command does not regenerate bootstrap credentials or reset an initialized administrator. [W7]

Use User administration to establish named accounts and reviewed execution grants. An eligible administrator or a matching customer/playbook grant is required in addition to the configured allowlist and Core authorization. Job/history/report visibility is owner-or-admin; supplying one-run credentials remains requester-only. [W7][W9]

Do not enable independent approval without providing a suitable separate approver. The shipped single-administrator site profile has require_approval=false; that does not bypass review of targets, mode, options or Core revisions.

10. Accept local readiness, then enable Core execution deliberately

Through the running add-on's executor:

sudo -u aim-web \
  /opt/aim-web/current/bin/aim-web \
  --config /etc/ansible/scripts/config/webgui.toml \
  core-check

Confirm Core product version, live capabilities and authorized customer discovery. An execution.enabled=false value at this point is the expected independent Core gate. [C3][W6]

For one existing approved Linux host with customer-key mode, replace the two literal identifiers before running:

sudo -u aim-web \
  /opt/aim-web/current/bin/aim-web \
  --config /etc/ansible/scripts/config/webgui.toml \
  core-check \
  --customer CUSTOMER \
  --playbook debug_test_connection \
  --host HOST \
  --key-mode customer

For a Windows/native-inventory test use --key-mode none. This is not forced password authentication. The CLI readiness request uses check mode and performs local preparation/runtime checks; it does not submit a job, unlock Vault, verify remote credentials or prove remote connectivity. [C3][W6]

Once Core terminal, transport, executor identity and local staging/readiness have been accepted, edit Core's existing configuration:

sudoedit /etc/ansible/scripts/aim.yml

Merge the approved opt-in, without creating a duplicate addons block:

addons:
  execution_enabled: true

Leave Core's accepted runtime.ansible_playbook, collection path, key-owner and other controller settings intact. Re-run aim-web core-check as aim-web and confirm that effective Core execution is now enabled. [C1, section 13][C3]

The gates remain separate:

Core addons.execution_enabled
    + WebGUI execution/credential/transport policy
    + eligible playbook and host limit
    + enabled account/grant and any required approval
    + fresh reviewed Core revision
    + worker readiness and required one-run credentials
    = an eligible execution request

No single true setting proves successful or safe managed-host execution.

11. Run the first controlled end-to-end test

Use a small approved scope. For a reporting acceptance test, Detected host roles is the Core guide's suggested starting operation. [C1, section 14][C4]

In the browser:

New run
  -> Select the test customer and Detected host roles
  -> Select one approved host
  -> Review options, Apply/Check mode and key handling
  -> Review the normalized scope, warnings and report contract
  -> Confirm and Run once
  -> Unlock this run when the worker is ready

A saved plan is optional. For an encrypted Linux customer key, choose customer-key loading; the credential modal can use the Vault-stored passphrase where Core permits it. Windows/native-inventory runs do not need a Linux customer key. Only fields requested by Core are offered. A supplied connection password is a native default, not an override of inventory credentials. [C3][W9]

The modal's acceptance message means the handoff was accepted, not that a password was verified. The empty reservation lasts five minutes, followed by the existing bounded handoff/start interval. Each new attempt needs fresh credentials; never put passwords in CLI arguments, URLs, plans, config examples or a queued record. [W9]

Check all three result dimensions:

Dimension	What to inspect
Core verdict	Status, stage, native exit and whether remote work may have started
Target outcomes	Each reviewed host's authoritative final outcome and counters
Operation report	Schema, availability, retained content and Apply/Check meaning

The role report should expose host_capabilities_v1 with the eight declared booleans. A missing report is not eight false values. A successful native exit can still produce failed / result_validation when required output is missing or invalid. Do not rerun automatically: remote changes may already be complete. [C3][C4][W9]

For a longer approved run, close/reopen the job, reload and check from another authorized device. Confirm recorded progress survives and new events continue without replaying execution or requesting the same credentials again. Validate Reports after finalization and verify the host activity link. No total-task percentage or ETA is promised.

After the first success, use Core's SANITY.md and WebGUI's docs/CONTROLLER-PILOT.md for separately approved Windows/Linux, mixed-result, cancellation, recovery and reporting cases. Updating packages, rebooting, starting services, exporting logs or deleting files needs its own test scope. One successful role query does not qualify every playbook. [C4][W10]

12. Review retention and storage capacity

Schema 5 stores accounts/workflows plus job-linked progress and operation reports. The shipped limits are: [W1][W9]

[journal]
max_events = 20000
max_bytes = 8388608

[reports]
max_bytes = 16777216
retain_configuration = false

The progress limit retains a bounded tail, not a complete raw terminal transcript. Omissions and capture gaps are disclosed. Report limits also remain subject to Core/catalog caps. Full parsed Checkmk configuration sections are not retained by default; availability/file/redaction metadata remain with a metadata-only label. Enabling full retention is a separate operator data decision and applies to future captured reports.

Deleting a job removes its journal and report rows and its contribution to the displayed host history. A compact audit deletion event remains. Existing backups have their own retention policy; deleting a job is not physical erasure of backups or storage remnants. Only WebGUI-owned retained runs contribute to Host Activity and Insights. [W9]

Allow space for the database, journals/reports, protected backups and multiple versioned virtual environments. The sources do not specify a universal minimum disk or memory size; capacity depends on retained jobs and report sizes. Monitor it rather than treating per-job limits as a total database cap.

The full webgui.toml is release-managed. Local policy edits are overwritten on a later managed update unless incorporated into the next reviewed release profile. [W1]

13. Know the managed layout and ownership boundary

/usr/local/bin/aim-web                  -> /opt/aim-web/current/bin/aim-web
/opt/aim-web/current                    -> selected versioned virtual environment
/opt/aim-web/venvs/                     versioned add-on environments
/opt/aim-web/deployment.json            active deployment metadata
/opt/aim-web/pending.json               interrupted/in-progress deployment journal
/etc/ansible/scripts/addons/webgui/     managed add-on source
/etc/ansible/scripts/config/webgui.toml managed add-on configuration
/var/lib/aim/webgui/                    private accounts, workflows and evidence DB
/var/lib/aim-web-executor/              private executor process HOME/staging
/run/aim-web-executor/core.sock         private group-accessible Core bridge
/var/backups/aim-web/                   protected add-on deployment checkpoints

The passwd-home .ansible staging directories and the three systemd units are also release-managed as described above. An existing account-home .ansible parent is normalized to the managed owner-only policy; review shared/custom uses of that directory before installation. The whole home is not made writable inside the service. [W2][W4]

Core source, aim.yml, inventories, Vaults, canonical keys, collection installation, global SSH trust and certificates remain outside the add-on installer. Core and add-on recovery data are different directories with different deployer commands. [C1][W1]

Patch-wave behavior after Core 3.3.0rc8

The public catalog adds the Windows-only Continue patching after reboot option (catalog hint false), plus reboot message/delay. Blank controls inherit native inventory/role policy; the add-on never forces continuation on. A successful Windows wave can require another reviewed run. See Patch waves before testing update/reboot behavior. Reports show pending/unknown, reboot-deferred and fixed per-update failure metadata without parsing raw output or creating follow-up jobs. Historical patch reports retain their recorded schema.

14. Back up the add-on independently

The deployer creates checkpoints during install/update. Maintain an operator backup policy in addition to those checkpoints, especially now that the database contains retained reports. Keep Core runtime/customer data backups separate. [W1]

The supported database backup command uses SQLite's backup API. Run it as the database-owning web identity, with a new, nonexisting destination it can write:

BACKUP_FILE="/var/lib/aim/webgui/manual-backup-$(date -u +%Y%m%dT%H%M%SZ).sqlite3"

sudo -u aim-web \
  /opt/aim-web/current/bin/aim-web \
  --config /etc/ansible/scripts/config/webgui.toml \
  db backup "$BACKUP_FILE"

The backup is created as 0600. Transfer it through the approved protected backup process, preserving confidentiality, and record the matching Core/WebGUI versions and configuration. Database contents include account hashes, session/workflow data and retained operational reports. A database backup alone is not the deployer's full source/configuration recovery checkpoint. [W6][W11]

Do not use an uncoordinated live copy of webgui.sqlite3 as a replacement for the SQLite backup operation. Do not install pytest into a production environment to perform an installation check.

15. Updating an existing add-on

For an existing compatible managed installation, use the new release's fresh extraction:

cd /var/tmp/aim-web-2.1.0rc9
sudo python3 deploy/deploy.py update

Use this exact command only when moving from an earlier version to 2.1.0rc9; it is not a same-version refresh. Preserve the previously selected --aim-scripts and account options when they differ from defaults. [W2]

When moving to Core 3.3.0rc8 from the older supported Core, finish/cancel work deliberately, stop submissions and quiesce terminal writers. Stop the add-on services before independently updating Core:

sudo systemctl stop \
  aim-web-worker.service \
  aim-web.service \
  aim-web-executor.service

Follow Core's own preview/apply procedure, then deploy the compatible WebGUI release. Do not expect the older adapter to accept a newly installed Core automatically. [C1][W1]

From WebGUI 2.1.0rc3, this update keeps SQLite schema 5: no new database migration or historical report rewrite. From a schema-4 baseline, the existing schema-5 migration remains (pending/queued work blocked, running work interrupted, stale reviews removed). Source/schema changes still require fresh review; do not replay an old prepared request. [W1]

Ordinary 2.x updates do not need --migrate-core. That flag is reserved for the documented legacy 1.x integration/state migration and recognized key-export/capability retirement. Read docs/MIGRATION-3.1.md before using it. Never treat it as a general permission or version-check bypass.

16. Recover without changing Core accidentally

Failed or interrupted installation/update

Record the complete deployment error and printed checkpoint identifier. The deployer attempts recovery where a checkpoint exists; an incomplete recovery leaves pending.json to prevent ambiguous repeated installation. Keep services stopped until the failure and recovery state are understood. [W1][W2]

For a recorded interrupted deployment, use the same release's extracted deployer:

cd /var/tmp/aim-web-2.1.0rc9
sudo python3 deploy/deploy.py recover

recover is not a generic repair command: it requires the pending deployment journal. Do not delete that journal or switch the current symlink manually to bypass recovery.

A failed first install preserves private state for deliberate recovery/retry; it does not automatically wipe accounts or generate another admin. A completed first-install checkpoint is not a prior installed version, and the rollback command rejects it. Inspect and preserve state instead of removing the database to force bootstrap. [W2][W7]

Roll back to a previous managed add-on

After independently backing up current state and quiescing work, use the exact checkpoint identifier, not Core's --from-backup syntax:

sudo python3 deploy/deploy.py rollback \
  --backup CHECKPOINT_ID

If the selected previous release requires schema 4, crossing back from schema 5 requires an explicit historical database restore:

sudo python3 deploy/deploy.py rollback \
  --backup CHECKPOINT_ID \
  --restore-auth-db

That can restore old password hashes and discard newer accounts, plans, jobs, journals and reports. Restored sessions are revoked. The backup ID must be one actually printed by this installation. [W1][W2]

The restored adapter is checked against the installed Core. If incompatible, restored add-on services remain stopped and disabled. Core rollback is separate; after validating a matched Core/WebGUI/database combination, explicitly enable/start the intended services. Neither rollback reverses completed remote changes, dependency installation or independent account/trust decisions.

Disable the add-on

After active work is finished or deliberately canceled:

sudo systemctl disable --now \
  aim-web-worker.service \
  aim-web.service \
  aim-web-executor.service

This disables the three add-on units, not AIM terminal. It is not a data purge. There is no uninstall command in this deployer; retain private state and backups until a separately approved removal decision. [W2]

17. Troubleshooting at the correct boundary

Symptom	First check / action
No managed add-on installation exists	Use install for a genuinely new add-on, not update.
Missing manifest, coverage or hash error	Re-verify the published ZIP and use a fresh extraction. Do not disable the verifier.
Unmanaged unit drop-in blocks deployment	Inspect the exact file and preserve it for review; do not blindly delete or reinstate older capability overrides.
Root command reports unsafe database/staging ownership	Use the correct service identity or real executor startup context. Do not transfer service state to root.
Manual executor command gives PermissionError on webgui.toml, but startup passed	The plain shell may lack the unit-scoped aim-web group. Inspect service journal and active identity.
Web config says execution enabled, Core says disabled	Review Core's independent addons.execution_enabled opt-in.
Missing collections or wrong native version	Fix the separately approved Core runtime/discovery as executor; do not pip-install Ansible into WebGUI.
Customer key rejected	Inspect the selected canonical key's owner, 0600 mode and traversal as executor. private_key_owner does not migrate existing keys.
Delegated controller task becomes UNREACHABLE	Inspect actual task/delegation and both staging paths inside the sandbox before blaming WinRM.
Linux trust failure despite root terminal success	Review effective executor SSH trust and actual destination; retain the approved global trust file.
Browser origin/security-token rejection	Check the exact HTTPS origin, immediate proxy/header chain, current page/session and clock. Do not disable CSRF.
Queued job never asks for credentials	Check worker activity, Core execution policy, approval/schedule and current job state. Only its requester gets the modal.
Native exit 0 but failed at result_validation	Inspect report availability/schema. Remote work may have completed; no automatic replay.
Empty old timeline/report	Pre-capture history is not reconstructed. Check the job's version and capture/retention markers.
pending.json blocks another deployment	Investigate and use the same deployer's recorded recovery path.

Start with version/status/configuration checks, the specific job's structured facts and the relevant service journal. Do not share passwords, full inventories/Vault dumps or unrestricted configuration/report bodies in diagnostics. Local Ansible/module logging may contain operational details even though the WebGUI journal excludes raw output. [C4][W1][W9]

Fresh add-on completion checklist

Before broader operational use confirm:

• Core's prerequisite installation and controlled terminal acceptance are complete.
• The WebGUI ZIP checksum and release manifest pass, and the current profile fits the site.
• The existing executor is independently authorized and can discover the intended Core runtime/collections.
• HTTPS/proxy trust is accepted; no placeholder origin or unverified transport attestation was reused.
• Fresh installation used install, or a real managed upgrade used update.
• All three expected services start, and executor startup checks pass inside their actual sandbox.
• Account/group, socket, configuration, state and dual-home staging match the managed policy.
• The bootstrap admin password has been changed and appropriate named accounts/grants reviewed.
• Core add-on execution was enabled only through the separate operator opt-in.
• A small approved end-to-end run has accepted credentials, native outcomes and its declared report.
• Reload/second-device progress and finished-job evidence behave correctly without replaying work.
• Report retention, protected backups, storage growth and recovery checkpoints have been reviewed.
• Candidate acceptance is recorded by actual UID/sandbox/runtime/platform; unsupported or untested cases are not claimed as passed.
• Terminal AIM remains independently usable.

Related documentation and source basis

This companion preserves the prerequisite boundary and terminology of the supplied Core guide. Add-on-specific commands and paths were checked against the published WebGUI archive. Core's installation guide has not been edited. This companion is updated with the add-on release; it is not a claim of a performed controller installation.

Reference	Source	What it governs
C1	Core scripts/docs/INSTALLATION.md, AIM 3.3.0rc8	Prerequisite Core installation, independent environments, opt-in, terminal acceptance and Core recovery
C2	Core ADDON_AGENTS.md	Same-UID public integration, ownership and hardening boundaries
C3	Core scripts/docs/ADDON_API.md	Authorization, discovery, readiness, request/credential/result semantics
C4	Core scripts/docs/SANITY.md, VALIDATION.md, OPERATION_RESULTS.md	Controller acceptance, candidate evidence and report rules
W1	WebGUI docs/DEPLOYMENT.md	Coordinated migration, retention, backups and rollback
W2	WebGUI deploy/deploy.py, pyproject.toml, constraints.txt	Actual supported installer flags, initialization, generated units, dependency handling and fixed paths
W3	WebGUI adapters/core_v1.py, core/executor.py under src/aim_webgui/	Qualified public Core compatibility and executor boundary
W4	WebGUI docs/PERMISSIONS-ROLLOUT.md, AGENTS.md	Resource ownership, split-home staging and known-hosts cautions
W5	WebGUI deploy/webgui.example.toml, deploy/nginx.example.conf, src/aim_webgui/config.py	Actual shipped site profile, reference headers and configuration validation
W6	WebGUI src/aim_webgui/cli.py	Version/configuration/readiness, backup and administration commands
W7	WebGUI src/aim_webgui/auth/service.py, workflows.py	Bootstrap, first password change, account/grant and job authorization
W8	WebGUI deploy/fetch_assets.py	Exact browser-asset pins and offline input names
W9	WebGUI docs/CREDENTIALS.md, RUN-COMFORT.md, EXECUTION.md, JOURNAL.md, REPORTS.md	Credential modal, execution lifecycle and retained-evidence behavior
W10	WebGUI docs/CONTROLLER-PILOT.md, docs/VERIFICATION.md	Operator tests and recorded qualification limits
W11	WebGUI src/aim_webgui/db/store.py	Database ownership and consistent backup behavior

Core topic files live in the independently installed Core tree. WebGUI topic/source paths above are relative to the verified aim-web-2.1.0rc9 extraction unless stated otherwise. The current Core archive contains scripts/docs/INSTALLATION.md. Read that separately installed prerequisite; no Core source or runtime is copied into this add-on.

Evidence limit: this document was produced from the supplied guides and inspected release code. It does not add new installation, dependency-security, proxy, systemd or managed-host test evidence. Use the two products' current validation records and complete the controller acceptance steps for the actual deployment.