From 343fe9b22f8aebe77de7d7e3f8df0e051fce9a57 Mon Sep 17 00:00:00 2001 From: admin_rb Date: Tue, 15 Sep 2026 18:53:28 +0200 Subject: [PATCH] added docs --- docs/CHANGELOG.md | 90 ++++++++++++++ docs/DEVELOPMENT.md | 67 +++++++++++ docs/INVENTORY.md | 122 +++++++++++++++++++ docs/OPERATIONS.md | 104 ++++++++++++++++ docs/RECOVERY.md | 69 +++++++++++ docs/SECURITY.md | 58 +++++++++ docs/WINDOWS.md | 133 ++++++++++++++++++++ docs/install.md | 286 ++++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 929 insertions(+) create mode 100644 docs/CHANGELOG.md create mode 100644 docs/DEVELOPMENT.md create mode 100644 docs/INVENTORY.md create mode 100644 docs/OPERATIONS.md create mode 100644 docs/RECOVERY.md create mode 100644 docs/SECURITY.md create mode 100644 docs/WINDOWS.md create mode 100644 docs/install.md diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md new file mode 100644 index 0000000..89f576d --- /dev/null +++ b/docs/CHANGELOG.md @@ -0,0 +1,90 @@ +# AIM Changelog + +## 2.1.x + +### 2.1.2 + +- Removed the experimental ASCII shield. +- Retained bitformer orange `#ff7a00` accent. +- Added compact `bitformer · AIM · Ansible Inventory Manager` header. +- Preserved `p / n` pagination convention. +- No intended Ansible behavior changes. + +### 2.1.1 + +- Changed the primary UI accent to bitformer orange. +- Introduced temporary ASCII branding experiment. +- Standardized pagination on `p / n`. + +### 2.1.0 + +- Major operator-console UI/UX refactor. +- Split presentation into focused UI modules. +- Added customer overview/dashboard. +- Unified selectors, review screens, result presentation, pagination + and filtering. +- Kept interactive commands live where password/editor interaction is + required. + +## 2.0.x + +- Added domain WinRM GPO rollout. +- Added malformed-YAML handling and multiple GPO/AD/GPP reliability + hotfixes. +- Corrected Windows local-account credential handling to be + host-specific. +- Corrected new-customer domain UPN defaults. +- Final 2.0.7 GPP Scheduled Task XML fix removed the invalid inner + `LogonType` element and added immediate registration + execution/retries. + +## 1.9.x + +- Added semantic Git-style template/Vault validation. +- Added non-destructive template consolidation. +- Consolidated playbook categories and generic multi-select behavior. + +## 1.8.x + +- Added structured Vault template creation. +- Added Windows credential models. +- Added SSH-agent/private-key-passphrase integration. +- Added direct multi-select workflows. + +## 1.7.x + +- Added customer domain/network defaults. +- Added batch Windows member-server domain access. + +## 1.6.x + +- Added/expanded automated Sophos configuration. +- Improved lock cleanup and Sophos menu organization. + +## 1.5.x + +- Added Sophos defaults, host variables and customer-specific playbook + support. + +## 1.4.x + +- Split Windows domain-account and member-server access workflows. + +## 1.3.x + +- Added local/domain Windows account flows and safer credential + transport. + +## 1.2.x + +- Added Windows WinRM service-account bootstrap and localized + Administrators handling. + +## 1.1.x + +- Added curated playbooks, routine local-only inventory validation, + empty default host-vars files and Enter=`0` navigation behavior. + +## 1.0.0 + +- Initial consolidated Python AIM base. diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md new file mode 100644 index 0000000..35f441c --- /dev/null +++ b/docs/DEVELOPMENT.md @@ -0,0 +1,67 @@ +# AIM Development Guide + +## Project root + +``` text +/etc/ansible/scripts/ +├── pyproject.toml +└── src/ + └── aim/ +``` + +The environment is installed editable: + +``` bash +/etc/ansible/.venv/bin/python -m pip install -e /etc/ansible/scripts +``` + +## Architecture + +Backend areas include inventory loading/writing/validation, customer +management, backup/session recovery, locking, permissions, playbooks, +SSH, Vault, Sophos and WinRM. + +The 2.1 UI is split into focused modules under `src/aim/ui/`, including +shared components, selection, execution, customers, hosts, access, +playbooks, administration and target selection. + +## UI conventions + +- Orange `#ff7a00` is the AIM/bitformer accent. +- Green = success. +- Yellow = warning. +- Red = failure. +- No ASCII logo. +- Header identity: `bitformer · AIM · Ansible Inventory Manager`. +- Pagination: `p / n`. +- Numbered navigation: Enter/`0` = Back or Cancel. +- Multi-select: number toggles; Enter reviews; `0` cancels. + +Opening menus should not unexpectedly run Ansible, contact hosts, +decrypt Vaults or prompt for passwords. + +## Inventory invariants + +`hosts.yml` is authoritative. Preserve arbitrary valid YAML/custom +keys/comments where possible. + +Writes should be validated and atomic, with stale-write/concurrency +protection and a session recovery backup. + +Do not change unrelated Ansible connection/authentication parameters as +part of feature work. + +## Packaging + +Production packages should contain runtime source and metadata, without +caches, `.pyc`, test artifacts, legacy Bash implementations or developer +notes unless explicitly requested. + +A release archive should expose `pyproject.toml` and `src/` at its root +rather than adding an extra wrapper directory. + +## Versioning + +Use a patch release for contained fixes and a feature/minor release for +larger functional changes. Update package metadata and the changelog +together. diff --git a/docs/INVENTORY.md b/docs/INVENTORY.md new file mode 100644 index 0000000..d1c2607 --- /dev/null +++ b/docs/INVENTORY.md @@ -0,0 +1,122 @@ +# AIM Inventory Model + +## Source of truth + +The authoritative inventory is: + +``` text +/etc/ansible/inventories//hosts.yml +``` + +AIM does not use `.hosts.tsv` as a secondary database and does not +reverse-sync TSV data into YAML. + +AIM uses round-trip YAML handling so valid manually maintained +structures/comments can be preserved where possible. + +## Platform groups + +Default top-level platform groups: + +``` text +linux +windows +sophosxgs +pfsense +``` + +Platform remains top-level because it determines connection semantics +such as SSH, WinRM or HTTPAPI. + +Hosts may have multiple memberships and optional one-level functional +subgroups. + +## Linux + +Linux group variables normally include the SSH connection and service +account. The customer SSH key directory is: + +``` text +group_vars/linux/.ssh/ +``` + +not: + +``` text +group_vars/linux/files/.ssh/ +``` + +`ansible_ssh_pass` may remain configured as a legacy +remote-login-password fallback. A private-key passphrase is a separate +secret. + +## Windows + +Domain-joined Windows hosts normally inherit the group-level service +identity/password. + +A local-account host can override credentials in: + +``` text +host_vars//main.yml +``` + +Shared local example: + +``` yaml +ansible_user: svc_bf-ansible +ansible_password: "{{ vault_windows_local_ansible_password }}" +``` + +Host-specific example: + +``` yaml +ansible_user: svc_bf-ansible +ansible_password: "{{ vault_ansible_password_server01_example_lan }}" +``` + +## Host vars + +Every newly managed host has: + +``` text +host_vars//main.yml +``` + +Existing host-vars content is not blindly overwritten. + +## Customer defaults + +AIM customer defaults live in: + +``` text +/etc/ansible/inventories//.aim.yml +``` + +Example: + +``` yaml +domain_suffix: bfmiglabor.lan +network_address: 10.20.30.0 +netmask: 255.255.255.0 +ad_dns_domain: intra.company.de +ad_netbios_domain: COMPANY +``` + +The AD DNS domain is used for service-account UPNs. NetBIOS remains +metadata/legacy naming information. + +## Safe writes + +Inventory mutations use the conceptual sequence: + +``` text +candidate temp file +→ local YAML validation +→ compare +→ session backup +→ atomic replace +``` + +AIM also protects against stale/concurrent writes and uses an inventory +lock. diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md new file mode 100644 index 0000000..2a92d9c --- /dev/null +++ b/docs/OPERATIONS.md @@ -0,0 +1,104 @@ +# AIM Operations Guide + +## Navigation + +AIM is organized around a customer context: + +``` text +Customer +├── Hosts Management +├── Access Management +├── Vault Management +├── Group Variables +├── Host Variables +├── Playbooks +└── Administration +``` + +For numbered navigation menus, Enter or `0` means Back/Cancel; at the +main menu it means Exit. Single selectors use Enter/`0` to cancel. +Multi-select uses numbers to toggle, Enter to review, and `0` to cancel. +Paginated views use `p / n` for Previous / Next. + +## Customer overview + +Opening a customer should provide local information without unexpectedly +contacting hosts, running Ansible or decrypting Vaults. The dashboard +includes inventory YAML state, Vault presence, host/platform counts and +available customer defaults. + +## Hosts + +`hosts.yml` is the source of truth. + +Creating a host also ensures: + +``` text +host_vars//main.yml +``` + +For ordinary non-Sophos hosts this file may be empty. Existing host +variable files are not overwritten. Removing a host also removes its +corresponding host-vars directory. + +Routine add/update/remove operations perform local YAML validation and +do not request the Vault password. + +## Access Management + +Linux access manages SSH keys/service-user access. + +Windows access includes temporary WinRM testing, local account creation, +domain account creation/repair, member-server domain access, domain +WinRM GPO rollout and configured-service-user testing. + +See `WINDOWS.md` for the Windows model. + +## Vault Management + +Vault operations include information, create, edit and delete. + +A new Vault is populated as plaintext with mode `0600`, YAML-validated, +then encrypted using: + +``` bash +ansible-vault encrypt --vault-id @prompt vault.yml +``` + +A failed encryption must not leave populated plaintext secrets behind. + +## Variables + +Group and host variable views are generally operator-readable without +AIM rewriting arbitrary custom configuration. AIM only changes values in +workflows explicitly designed to do so. + +Template consolidation is explicit and non-destructive: missing AIM +defaults/comments can be added, while existing non-empty values and +custom keys are retained. + +## Playbooks + +Curated categories include CheckMK, Debug, Maintenance and Sophos XGS. +Compatible host/group selection is used to build the Ansible `--limit`. + +Interactive playbooks and operations that require password/Vault prompts +retain live terminal access. + +## Administration + +Administration includes inventory validation, template consolidation, +recovery and customer defaults. + +Explicit inventory validation performs YAML parsing and +`ansible-inventory`. When a customer Vault exists, validation uses the +customer's Vault identity and may prompt for its password. + +AIM maintains one pre-change inventory recovery backup per inventory per +AIM process/session: + +``` text +hosts.aim-session.bak.yml +``` + +Restores are explicit and YAML-validated. diff --git a/docs/RECOVERY.md b/docs/RECOVERY.md new file mode 100644 index 0000000..1c75820 --- /dev/null +++ b/docs/RECOVERY.md @@ -0,0 +1,69 @@ +# AIM Recovery Guide + +## Principle + +Git protects source code, not AIM's ignored runtime secrets. + +A `git reset --hard` or fresh checkout cannot restore ignored data such +as inventory Vaults, SSH keys or a Python virtual environment. + +## Critical persistent data + +Back up separately: + +``` text +/etc/ansible/inventories//group_vars/all/vault.yml +/etc/ansible/inventories//group_vars/linux/.ssh/ +``` + +Other customer inventory YAML should also be covered by the system +backup policy. + +## Rebuildable data + +Do not recover `.venv` from Git. Rebuild it: + +``` bash +sudo rm -rf /etc/ansible/.venv +sudo python3 -m venv /etc/ansible/.venv +sudo /etc/ansible/.venv/bin/python -m pip install --upgrade pip setuptools wheel +sudo /etc/ansible/.venv/bin/python -m pip install -e /etc/ansible/scripts +sudo ln -sfn /etc/ansible/.venv/bin/aim /usr/local/bin/aim +``` + +## Recover Vaults and SSH keys from a filesystem backup + +When restoring from a backup tree, copy only files that do not already +exist in the active inventory. Never overwrite surviving current secrets +as part of a bulk recovery. + +See `../install.md` for the current dry-run and restore commands. + +## AIM session inventory backup + +AIM can create: + +``` text +hosts.aim-session.bak.yml +``` + +This is a pre-change recovery aid, not the primary backup strategy for +customer secrets. + +Restore from it only through an explicit recovery decision after +validating the relevant inventory state. + +## Post-recovery validation + +Before deleting the backup source: + +1. Confirm Vault files exist for expected customers. +2. Confirm SSH private/public key files and permissions. +3. Test Vault decryption/access for representative customers. +4. Test Linux SSH authentication. +5. Test Windows WinRM for representative domain/local credential + models. +6. Test any other customer-specific access that depends on recovered + secrets. + +Keep the filesystem backup until these checks pass. diff --git a/docs/SECURITY.md b/docs/SECURITY.md new file mode 100644 index 0000000..6cae2b4 --- /dev/null +++ b/docs/SECURITY.md @@ -0,0 +1,58 @@ +# AIM Sensitive Data and Security Notes + +## Sensitive files + +AIM deliberately keeps secrets outside Git. + +Important locations include: + +``` text +/etc/ansible/inventories//group_vars/all/vault.yml +/etc/ansible/inventories//group_vars/linux/.ssh/ +``` + +Vault files may contain Windows and Linux authentication material. +`.ssh` directories may contain private keys that cannot be regenerated +without coordinating key rotation on managed systems. + +## Backup requirement + +Git is not a backup for ignored secrets. + +System backup procedures must include customer Vaults and SSH key +material. Recovery should preserve existing current files and restore +only missing data unless an operator explicitly chooses otherwise. + +## Vault handling + +AIM encrypts newly created Vaults with `ansible-vault`. Plaintext +populated Vault data should not remain on disk after a failed encryption +attempt. + +Semantic Vault comparison must not print secret values. It should +compare key existence/state rather than exposing plaintext. + +## SSH key handling + +Private-key passphrases and remote SSH passwords are separate concepts. + +The Linux private key location is: + +``` text +group_vars/linux/.ssh/svc_bf-ansible +``` + +Private keys should have restrictive filesystem permissions. + +## Authorization + +AIM authorization is based on membership in a configured group resolved +through NSS/SSSD/winbind/local group services. The configured group name +is authoritative; numeric GIDs may differ across hosts. + +Operators should verify effective membership with: + +``` bash +getent group '' +id +``` diff --git a/docs/WINDOWS.md b/docs/WINDOWS.md new file mode 100644 index 0000000..5763148 --- /dev/null +++ b/docs/WINDOWS.md @@ -0,0 +1,133 @@ +# Windows, WinRM and Active Directory + +## Service account + +The standard service account is: + +``` text +svc_bf-ansible +``` + +For an AD domain, the configured identity is normally the UPN: + +``` text +svc_bf-ansible@ +``` + +The normal domain-account Vault variable is: + +``` yaml +vault_windows_ansible_password: "..." +``` + +The shared-local-account Vault variable is: + +``` yaml +vault_windows_local_ansible_password: "..." +``` + +## Windows credential models + +New Windows hosts can use: + +1. Domain service account +2. Shared local service account +3. Host-specific local service account + +Local credential choices are host-specific overrides and must not +rewrite `group_vars/windows/main.yml` for every Windows machine. + +## Domain account creation + +AIM can create/repair the domain service identity and add it to the +appropriate built-in Administrators context used by the current design. +It does not make the account a Domain Admin. + +Administrative bootstrap credentials should only be requested where +genuinely required. + +## Member-server access + +AIM can grant the existing domain service identity local Administrators +membership on selected member servers. Domain Controllers are +rejected/skipped for this workflow. + +## Domain WinRM GPO rollout + +The rollout uses one prepared, already-manageable Domain Controller as +its administration point. + +The managed objects are: + +``` text +AD group: GG_bitformer_Ansible_Admins +GPO: bitformer - Ansible WinRM +Task: bitformer - Configure Ansible WinRM +``` + +The GPO configures the local Administrators membership, deploys the +WinRM setup payload/scheduled task, configures HTTPS WinRM and firewall +access, and verifies the resulting state. + +### Multiple target OUs + +A single GPO should be linked to multiple selected OUs rather than +creating a separate GPO for Servers, Clients, etc. + +Example: + +``` text +bitformer - Ansible WinRM +├── OU=Servers,DC=intra,DC=company,DC=de +└── OU=Clients,DC=intra,DC=company,DC=de +``` + +The OU selector should therefore support multi-selection. + +GPO link management is **additive and idempotent**: + +- Selected OU already linked: keep/repair as appropriate. +- Selected OU not linked: create the link. +- Unselected OU: do nothing. + +Selecting only `Servers` on a later run must **not** imply that an +existing `Clients` link should be removed. + +Link removal should be an explicit operation if/when AIM implements it. + +### Child OUs + +AIM links the GPO to the selected OU. It should not create redundant +links on every descendant OU merely to emulate inheritance. Normal Group +Policy inheritance handles descendants unless AD policy configuration +changes that behavior. + +### Domain Controllers + +The Domain Controllers OU must remain unavailable/rejected for the +normal member-machine WinRM rollout. + +## WinRM payload + +The payload ensures WinRM is running, configures/reuses a suitable +certificate or creates a self-signed Server Authentication certificate, +creates the HTTPS listener, allows TCP/5986 and verifies the final +state. + +The scheduled task runs immediately after registration and can retry +periodically. After successful verification it disables itself. + +## Troubleshooting + +Useful client-side checks include: + +``` powershell +gpupdate /force +gpresult /h C:\Temp\gpresult.html +Get-Service WinRM +winrm enumerate winrm/config/listener +Get-ScheduledTask -TaskName "bitformer - Configure Ansible WinRM" +``` + +Also inspect Group Policy operational logs and Task Scheduler events +when Group Policy Preferences reports a task import failure. diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 0000000..713ad68 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,286 @@ +# AIM Installation Guide + +This guide installs AIM with `/etc/ansible/scripts` as the project root. + +## Layout + +``` text +/etc/ansible/scripts/ +├── pyproject.toml +└── src/ + └── aim/ + +/etc/ansible/.venv/ +/usr/local/bin/aim -> /etc/ansible/.venv/bin/aim +``` + +The virtual environment is intentionally not stored in Git. + +## 1. Prerequisites + +On Debian/Ubuntu: + +``` bash +sudo apt update +sudo apt install -y python3 python3-venv python3-pip +``` + +## 2. Rebuild the virtual environment + +``` bash +sudo rm -rf /etc/ansible/.venv +sudo python3 -m venv /etc/ansible/.venv +sudo /etc/ansible/.venv/bin/python -m pip install --upgrade pip setuptools wheel +``` + +## 3. Install AIM + +``` bash +sudo /etc/ansible/.venv/bin/python -m pip install -e /etc/ansible/scripts +``` + +Verify: + +``` bash +ls -l /etc/ansible/.venv/bin/aim +/etc/ansible/.venv/bin/python -m pip check +``` + +## 4. Restore the system-wide command + +``` bash +sudo ln -sfn /etc/ansible/.venv/bin/aim /usr/local/bin/aim +``` + +Verify: + +``` bash +ls -l /usr/local/bin/aim +aim --version +``` + +## 5. Authorization group / using another GID + +AIM authorization is group-based rather than root-based. The default +required group is: + +``` text +srv_debsansible01_admins@bitformer.lan +``` + +AIM resolves the configured **group name** through the operating +system's NSS layer (for example local groups, SSSD or winbind). The +operator must have that group active as a primary or supplementary +group. + +Do not hard-code a numeric GID into AIM merely because a particular +server uses a different GID. Numeric GIDs can differ between systems; +configure the appropriate group name and let NSS resolve its GID. + +Inspect the default group and its resolved GID: + +``` bash +getent group 'srv_debsansible01_admins@bitformer.lan' +``` + +Check the current user's active groups/GIDs: + +``` bash +id +id -G +id -Gn +``` + +If another authorization group should be used, change AIM's **Required +Group** through: + +``` text +Global Config +└── Required Group +``` + +For example, if the local/AD-backed group is: + +``` text +ansible_operators +``` + +verify it first: + +``` bash +getent group ansible_operators +``` + +Then configure `ansible_operators` as AIM's Required Group. AIM will use +the GID returned by NSS for that group. + +After adding a user to a group, the login session may need to be renewed +before the supplementary group becomes active. Verify with `id` before +troubleshooting AIM authorization. + +### Same group name, different GID + +This is supported. For example, one server may resolve: + +``` text +ansible_operators:x:1200:... +``` + +and another may resolve: + +``` text +ansible_operators:x:48001:... +``` + +AIM should be configured with `ansible_operators`, not `1200` or +`48001`. + +### Different group name + +Configure the alternative group name in AIM Global Config and confirm: + +``` bash +getent group '' +id +``` + +If `getent` cannot resolve the group, fix NSS/SSSD/winbind/local group +resolution first. + +## 6. Git-ignored runtime data + +Important ignored data includes: + +``` gitignore +.vscode/* +.venv/* +.ansible/* +secure/* +secure/keys/* +vault.yml +.ssh +*.msi +*.deb +*.rpm +**/.hosts.tsv +**/hosts.yml.aim-session.bak +**/hosts.aim-session.bak.yml +*.bak +``` + +For AIM inventory recovery, the critical persistent data is `vault.yml` +and inventory `.ssh/` content. Do not restore `.venv`; rebuild it. + +## 7. Recover missing Vaults and SSH keys + +Example restored backup: + +``` text +/etc/ansible/inventories_RESTORED_20260915_152127 +``` + +Active inventories: + +``` text +/etc/ansible/inventories +``` + +### Dry run + +``` bash +BACKUP="/etc/ansible/inventories_RESTORED_20260915_152127" +TARGET="/etc/ansible/inventories" + +echo "=== vault.yml ===" +sudo find "$BACKUP" -type f -name 'vault.yml' -print0 | +while IFS= read -r -d '' src; do + rel="${src#"$BACKUP"/}" + dst="$TARGET/$rel" + if [ -e "$dst" ]; then + echo "KEEP $dst" + else + echo "RESTORE $src -> $dst" + fi +done + +echo +echo "=== .ssh files ===" +sudo find "$BACKUP" -path '*/.ssh/*' -type f -print0 | +while IFS= read -r -d '' src; do + rel="${src#"$BACKUP"/}" + dst="$TARGET/$rel" + if [ -e "$dst" ]; then + echo "KEEP $dst" + else + echo "RESTORE $src -> $dst" + fi +done +``` + +### Restore + +``` bash +BACKUP="/etc/ansible/inventories_RESTORED_20260915_152127" +TARGET="/etc/ansible/inventories" + +sudo find "$BACKUP" -type f -name 'vault.yml' -print0 | +while IFS= read -r -d '' src; do + rel="${src#"$BACKUP"/}" + dst="$TARGET/$rel" + if [ -e "$dst" ]; then + echo "KEEP $dst" + continue + fi + sudo mkdir -p "$(dirname "$dst")" + sudo cp -a "$src" "$dst" + echo "RESTORED $dst" +done + +sudo find "$BACKUP" -path '*/.ssh/*' -type f -print0 | +while IFS= read -r -d '' src; do + rel="${src#"$BACKUP"/}" + dst="$TARGET/$rel" + if [ -e "$dst" ]; then + echo "KEEP $dst" + continue + fi + sudo mkdir -p "$(dirname "$dst")" + sudo cp -a "$src" "$dst" + echo "RESTORED $dst" +done +``` + +`cp -a` preserves ownership, permissions and timestamps. Existing active +files are never overwritten. + +Verify: + +``` bash +sudo find /etc/ansible/inventories -type f -name 'vault.yml' -print | sort +sudo find /etc/ansible/inventories -path '*/.ssh/*' -type f -print | sort +sudo find /etc/ansible/inventories -path '*/.ssh/*' -type f -exec ls -l {} \; +``` + +Keep the restored backup until Vault and SSH access have been tested. + +## 8. Final verification + +``` bash +aim --version +/etc/ansible/.venv/bin/python -m pip check +aim +``` + +## Updating AIM + +AIM is installed editable from `/etc/ansible/scripts`. + +If its entry point needs recreation: + +``` bash +sudo /etc/ansible/.venv/bin/python -m pip install -e /etc/ansible/scripts +sudo ln -sfn /etc/ansible/.venv/bin/aim /usr/local/bin/aim +``` + +Customer data under `/etc/ansible/inventories` is separate from the AIM +source installation.