DeSq-Gaming/Ansible
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

aim-web2.1.0rc9

Browse Source
This commit is contained in:
admin_rb
2026-09-22 19:23:17 +02:00
parent d095887d2e
commit 3dfc80b782
438 changed files with 31613 additions and 1510 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deploy/README.md
+205
View File
@@ -0,0 +1,205 @@
# AIM core ZIP deployment - 3.3.0rc8
This standard-library operational helper installs a complete source release and
its two source-bound command launchers. No Git, patch files, release manifest,
package installation, add-on inspection or account/group migration is used.
Python 3.11+ on Linux is required. Development validators are not distributed.
## Verify and preview
Obtain the ZIP and its SHA-256 sidecar through a trusted channel. The sidecar is an
integrity check, not a publisher signature. Stage outside the installation tree:
```bash
cd /var/tmp
sha256sum -c AIM-Ansible-3.3.0rc8.zip.sha256
unzip AIM-Ansible-3.3.0rc8.zip
cd aim-core-3.3.0rc8
sudo python3 deploy/deploy.py update --dry-run
```
The target defaults to `/etc/ansible`. `update` requires existing core source;
`install` is for a fresh tree. Preview is the default without `--apply`.
Do not extract over the live installation. Source and target must not overlap;
symlink paths and unexpected source files are rejected.
3.3.0rc8 checks the existing **AIM** Python interpreter before making changes. It can
infer it only from an unambiguous installed `aim` Python shebang. It does not assume
that `sudo python3`, the Ansible interpreter or an add-on environment contains AIM's
dependencies. It preserves a virtual environment's Python path without resolving
its symlink to the system Python.
When discovery is unavailable (for example sudo has a different PATH), provide the
already identified AIM interpreter explicitly. In the operator test session,
`AIM_PYTHON` is the interpreter that successfully ran `scripts/aimctl.py`:
```bash
test -x "$AIM_PYTHON" || { echo 'Set AIM_PYTHON to the existing AIM interpreter first.'; exit 1; }
sudo python3 deploy/deploy.py update --aim-python "$AIM_PYTHON" --dry-run
```
Shell wrappers and `#!/usr/bin/env ...` shebangs are not guessed. The interpreter
must be an absolute executable Python 3.11+ path with no spaces (up to 120 characters),
and must already contain the dependencies from `scripts/pyproject.toml`.
## Command directory and multiple installations
The preview prints the chosen interpreter, command directory, source operations
and `@launchers/aim` / `@launchers/aimctl` operations. `@launchers` is a journal
identifier, not a directory shipped in the source archive.
By default the command directory is the existing `aim` command's parent directory,
or `/usr/local/bin` when no command exists and an interpreter was supplied. Override
with `--bin-dir /absolute/command/directory`. A nondefault `--target` **requires** its
own explicit `--bin-dir` so development deployment cannot silently replace production
commands. Use the same chosen interpreter/directory on preview and apply.
Only recognized AIM Python entry scripts or AIM-managed launchers for this target
may be replaced. Unrelated programs, unsafe symlink command destinations and launchers for
another installation are refused. There is no automatic force-overwrite escape hatch.
Choose a reviewed unused command directory when the existing layout is nonstandard.
Ensure the selected directory is on the intended operator's PATH; aliases and another
installation earlier on PATH remain the operator's responsibility.
## Apply while quiesced
Stop new jobs and exit active AIM/Ansible sessions. `--quiesced` acknowledges that
source writers/runners have been stopped; it does not discover, kill or pause jobs.
```bash
sudo python3 deploy/deploy.py update --apply --quiesced
# Include the same --aim-python and --bin-dir options used in the preview, if any.
hash -r
command -v aim
command -v aimctl
aim --version
aimctl --version
aimctl capabilities
```
The helper verifies the installed `aim --version` and `aimctl capabilities` against
the source release before reporting success. It uses an explicit installed config
path for its capability check. It does not invoke Ansible or contact managed hosts.
Both launchers import the deployed `scripts/src/aim` source with the selected AIM
Python. Old launchers and core source are included in protected recovery data.
A stable deployment lock prevents another cooperating deployment. Each source file
is replaced atomically, preserving existing UID/GID/mode/extended attributes.
New source files use 0644; new launchers use 0755, and recognized existing launchers
retain their metadata with executable bits enabled. Directories use normal caller
ownership/inheritance. This is not a recursive ownership-policy migration.
Recovery defaults to `/var/backups/aim-core`; the helper prints the exact private
0700 recovery directory. `--backup-dir /private/path` selects another root outside
source and installation. Keep sufficient space and apply your retention policy.
Obsolete files inside `scripts/src/aim/` and the explicitly retired Core documents below are pruned. Unknown customer files in
other locations are retained. Add-on code must use its own namespace, not the core
Python namespace.
**Preserved when present:** operator `scripts/aim.yml`, customer `.aim.yml`, inventories,
Vaults, SSH keys, add-ons and their state/configuration, environments, external assets,
unknown playbooks/roles and staged agent files. No dependencies, services, accounts,
LDAP/local group memberships, remote credentials or add-on version gates are modified.
Missing new settings are not automatically inserted into an operator's existing YAML.
## Python package and runtime scope
These launchers are source-bound entry points, not a pip reinstall. The helper does
not change installed wheel/distribution metadata or upgrade packages. Machine clients
calling the installed `aimctl` reach the deployed source. In-process Python clients
must also resolve `aim` to this source (for example an existing editable installation
or an explicitly configured source import path), not an old separately installed
wheel. Verify `aim.__file__` and `aim.__version__` in that client's own environment.
No add-on's Python environment is changed by core deployment.
For a fresh installation, provision an AIM Python 3.11+ environment and its declared
`ruamel.yaml`/`rich` dependencies first, then pass that interpreter to `install`.
Provide the separate canonical Ansible Core **2.19.11** runtime and approved collections
from `requirements-controller.txt` / `requirements.yml` explicitly. The helper does
not install from the network or mutate a system-managed Ansible installation.
For a nondefault controller root, maintain that installation's operator configuration
and use `aimctl --config /absolute/root/scripts/aim.yml ...` when necessary. Existing
terminal configuration discovery is unchanged; creating another launcher does not
silently redirect a terminal's global configuration.
## Opt-in API execution
Follow `scripts/docs/SANITY.md` (historical evidence is in
`scripts/docs/VALIDATION.md`). External execution remains disabled by
default and is not enabled by deployment, readiness or the launcher smoke test.
Preserve the existing YAML and merge only deliberately approved settings:
```yaml
addons:
execution_enabled: true
runtime:
ansible_playbook: /usr/bin/ansible-playbook
```
The executable path is the operator's approved native runtime, not an assumption
for every installation. Worker authorization, filesystem/key access, collections,
connection dependencies and same-UID execution still apply. Do not make private keys
group-readable to bypass an unsupported cross-user deployment.
## Recovery and interruption
Ordinary application/launcher-check failures attempt to restore touched source and
launchers. The update is not a whole-tree atomic transaction: power loss or SIGKILL
can leave partial source. Keep jobs stopped until recovery/version checks complete.
Recovery journals contain intent, hashes and original metadata; they are local
recovery records, not a distributed release manifest or inventory backup.
Use this 3.3.0rc8 deployer and the printed recovery directory; include the same target
for a nondefault installation. Launcher paths are recorded in the journal.
```bash
sudo python3 deploy/deploy.py rollback --from-backup /var/backups/aim-core/RECOVERY-DIRECTORY --dry-run
sudo python3 deploy/deploy.py rollback --from-backup /var/backups/aim-core/RECOVERY-DIRECTORY --apply --quiesced
hash -r
aim --version
```
Rollback validates installed/recovery hashes and refuses to overwrite subsequently
modified source or launchers. Existing aim.yml is never rolled back or removed,
even after a fresh install. Empty directories may remain. A newly created aimctl
launcher is removed when restoring a previous release that had no such launcher.
No remote Ansible action, dependency installation, account/group change or add-on
state is reversed. Keep independent backups of runtime/customer data.
## Required executor staging (independently provisioned units)
Read `scripts/docs/EXECUTOR_STAGING.md` in the installed tree. New add-on executor
installers should provision owner-only staging and a narrow directory write
exception by default, then run `aimctl staging-check` inside the actual unit at
startup. Ordinary `sudo -u` success does not reproduce mount/syscall restrictions.
The source deployer does not inspect/edit/restart services. It preserves the
working exception already applied by the operator. The automatic Core preflight
is on by default, but cannot make a read-only mount writable. Do not remove the
exception, broaden home write access or recursively chown anything during this
update. The new startup command is available after the normal launcher refresh.
## Documentation consolidation in 3.3.0rc8
Current docs have stable topic names with one validation record and one acceptance
checklist. Upgrade removes only these explicitly retired Core filenames (with
protected rollback copies), including locally modified versions of those exact files:
- scripts/docs/RC19_HANDOFF.md
- scripts/docs/SANITY_3.2.0.md
- scripts/docs/SANITY_3.2.1rc2.md
- scripts/docs/VERIFICATION.md
- scripts/docs/LOCAL_VALIDATION.md
- scripts/docs/CONTROLLER_ACCEPTANCE.md
Review REMOVE entries before applying. Move any operator notes out of these retired
Core-owned names before deployment. Unknown Markdown files and other operator
documents are preserved. Rollback restores retired document bytes/metadata through
the existing recovery journal. No recursive docs purge or runtime deletion occurs.
The new playbooks/filter_plugins/*.py files and playbooks/schemas/*.yml files are
Core-owned reporting source. They are deployed with the playbooks; no add-on Python
environment or collection is modified.
deploy/deploy.py
+593
View File
@@ -0,0 +1,593 @@
#!/usr/bin/env python3
"""Operational full-source install/update, not a Git patcher or release validator.
Python 3.11+, standard library only. Does not install dependencies, change groups,
start services, touch inventory/add-on data, or replace operator configuration.
"""
from __future__ import annotations
import argparse
from dataclasses import dataclass
import base64
from contextlib import contextmanager
from datetime import datetime, timezone
import fcntl
import hashlib
import json
import os
from pathlib import Path
import shutil
import stat
import subprocess
import tomllib
import sys
import tempfile
class DeploymentError(Exception):
pass
def no_symlink(path: Path):
for part in (path, *path.parents):
if part.is_symlink():
raise DeploymentError('Refusing a symlink in a deployment path: ' + str(part))
def no_symlink_parents(path: Path):
for part in path.parents:
if part.is_symlink():
raise DeploymentError('Refusing a symlink in a deployment parent path: ' + str(part))
def canonical(path: Path) -> Path:
# Check before normalization so an intermediate symlink cannot disappear.
no_symlink(path.absolute())
return Path(os.path.abspath(path))
@contextmanager
def deployment_lock(target: Path):
lock = target / '.aim-core-deploy.lock'
fd = os.open(lock, os.O_WRONLY | os.O_NONBLOCK | os.O_CREAT | os.O_CLOEXEC | getattr(os, 'O_NOFOLLOW', 0), 0o600)
try:
if not stat.S_ISREG(os.fstat(fd).st_mode):
raise DeploymentError('Deployment lock is not a regular file.')
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
yield
finally:
os.close(fd)
def allowed(relative: Path) -> bool:
parts = relative.parts
if relative.as_posix() in {'requirements.yml', 'requirements-controller.txt', 'deploy/deploy.py', 'deploy/README.md'}:
return True
if len(parts) >= 3 and parts[:3] == ('scripts', 'src', 'aim'):
return relative.suffix == '.py' or relative.as_posix() == 'scripts/src/aim/integrations/ansible.cfg'
if len(parts) >= 3 and parts[:2] == ('scripts', 'docs'):
return relative.suffix in ('.md', '.json')
if len(parts) == 2 and parts[0] == 'scripts':
return parts[1] in {'pyproject.toml', 'aim.yml', 'AIM-WinRM-OneTime.ps1', 'aimctl.py'}
if len(parts) == 3 and parts[:2] == ('playbooks', 'filter_plugins'):
return relative.suffix == '.py'
if len(parts) >= 2 and parts[0] == 'playbooks':
return relative.suffix in ('.yml', '.yaml', '.md')
if len(parts) >= 3 and parts[0] == 'roles':
if relative.name == 'README.md':
return True
return len(parts) >= 4 and parts[2] in ('tasks', 'defaults', 'handlers', 'meta', 'vars', 'templates') and relative.suffix in ('.yml', '.yaml', '.j2')
return False
def source_files(source: Path) -> dict[str, Path]:
result = {}
for parent, dirs, files in os.walk(source, followlinks=False):
dirs[:] = sorted(d for d in dirs if d != '__pycache__')
for d in dirs:
no_symlink(Path(parent) / d)
for name in sorted(files):
path = Path(parent) / name
relative = path.relative_to(source)
if path.suffix == '.pyc':
continue
if path.is_symlink() or not path.is_file() or not allowed(relative):
raise DeploymentError('Unexpected source entry; refusing deployment: ' + str(relative))
result[relative.as_posix()] = path
required = {'scripts/src/aim/__init__.py', 'scripts/pyproject.toml', 'scripts/docs/AGENTS.md', 'scripts/docs/ADDON_AGENTS.md', 'playbooks/aim_catalog.yml'}
if not required.issubset(result):
raise DeploymentError('Not a complete AIM replacement source tree.')
return result
def digest(path: Path | bytes) -> str:
if isinstance(path, bytes):
return hashlib.sha256(path).hexdigest()
with path.open('rb') as stream:
return hashlib.file_digest(stream, 'sha256').hexdigest()
LAUNCHER_MARKER = '# AIM core managed launcher v1'
def _launcher_digest(path: Path) -> str | None:
if path.is_symlink():
return digest(('symlink:' + os.readlink(path)).encode('utf-8'))
if path.exists():
return digest(path)
return None
def _read_launcher_text(path: Path) -> str:
candidate = path.resolve(strict=True) if path.is_symlink() else path
if not candidate.is_file() or candidate.stat().st_size > 65536:
raise DeploymentError('A non-launcher occupies ' + str(path))
try:
return candidate.read_text(encoding='utf-8')
except UnicodeDecodeError:
raise DeploymentError('Refusing to replace an unrecognized launcher: ' + str(path)) from None
def _restore_symlink(destination: Path, target: str):
no_symlink_parents(destination)
temporary = destination.parent / ('.aim-link-' + next(tempfile._get_candidate_names()))
try:
os.symlink(target, temporary)
os.replace(temporary, destination)
dfd = os.open(destination.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
temporary.unlink(missing_ok=True)
def _launcher_path(item, target: Path, launcher_dir: Path | None = None):
relative = Path(item['path'])
if item.get('kind') == 'launcher':
if launcher_dir is None or relative.parts not in (('@launchers', 'aim'), ('@launchers', 'aimctl')):
raise DeploymentError('Invalid launcher recovery path.')
directory = canonical(launcher_dir)
if not directory.is_absolute() or directory == Path('/'):
raise DeploymentError('Invalid launcher directory.')
destination = directory / relative.name
else:
if relative.is_absolute() or '..' in relative.parts or relative.parts[:1] == ('@launchers',):
raise DeploymentError('Invalid core source path.')
destination = target / relative
if item.get('kind') == 'launcher':
no_symlink_parents(destination)
else:
no_symlink(destination)
return destination
def _command(args, *, timeout=20):
env = os.environ.copy()
for name in ('PYTHONPATH', 'PYTHONHOME', 'PYTHONSTARTUP', 'PYTHONINSPECT'):
env.pop(name, None)
env['PYTHONDONTWRITEBYTECODE'] = '1'
try:
result = subprocess.run(args, stdin=subprocess.DEVNULL, capture_output=True,
text=True, timeout=timeout, env=env, cwd='/')
except (OSError, subprocess.TimeoutExpired):
raise DeploymentError('AIM interpreter/launcher check could not complete; no dependency installation is attempted.') from None
if result.returncode:
raise DeploymentError('AIM interpreter/launcher check failed. Supply the existing AIM environment with --aim-python; ensure its Python 3.11+, ruamel.yaml and rich dependencies and operator configuration are usable.')
return result.stdout
@dataclass(frozen=True)
class EntryPoints:
python: Path
directory: Path
target: Path
def contents(self):
result = {}
for name, module in (('aim', 'aim.__main__'), ('aimctl', 'aim.ctl')):
metadata = json.dumps({'target': str(self.target), 'python': str(self.python), 'command': name}, sort_keys=True)
content = (f'#!{self.python}\n{LAUNCHER_MARKER}\n# {metadata}\n'
'import sys\n'
'sys.dont_write_bytecode = True\n'
f'sys.path.insert(0, {str(self.target / "scripts/src")!r})\n'
f'from {module} import main\n'
'if __name__ == "__main__":\n raise SystemExit(main())\n')
result['@launchers/' + name] = content.encode('utf-8')
return result
def verify_python(self, source):
# An explicit interpreter is an administrator-selected executable, not
# user-controlled input to an elevated web wrapper. Preserve venv symlinks.
expected = tomllib.loads((source / 'scripts/pyproject.toml').read_text())['project']['version']
code = ('import sys,json; assert sys.version_info >= (3,11); '
f'sys.path.insert(0, {str(source / "scripts/src")!r}); '
'import ruamel.yaml,rich,aim; from aim.ui.app import App; '
'from aim.services.v1 import AimService; from aim.ctl import main; '
'print(json.dumps({"version":aim.__version__}))')
value = json.loads(_command([str(self.python), '-I', '-B', '-c', code]))
if value.get('version') != expected:
raise DeploymentError('Extracted source/package versions disagree.')
return expected
def verify_installed(self, version):
got = _command([str(self.directory / 'aim'), '--version']).strip()
value = json.loads(_command([str(self.directory / 'aimctl'), '--config',
str(self.target / 'scripts/aim.yml'), 'capabilities']))
if got != 'AIM ' + version or not value.get('ok') or value.get('result', {}).get('core_version') != version:
raise DeploymentError('Installed AIM/aimctl launchers do not report the deployed core version.')
print('PASS installed aim --version and aimctl capabilities (' + version + ').')
def entry_points(target: Path, python: Path | None, directory: Path | None) -> EntryPoints:
existing = shutil.which('aim')
if target != Path('/etc/ansible') and directory is None:
raise DeploymentError('A nondefault --target requires an explicit --bin-dir to avoid replacing another installation\'s commands.')
if python is None:
if not existing:
raise DeploymentError('Cannot discover the AIM interpreter. Supply --aim-python /absolute/path/to/the/existing/AIM/bin/python.')
with Path(existing).open(encoding='utf-8') as stream:
first = stream.readline().strip()
if not first.startswith('#!/') or len(first[2:].split()) != 1 or Path(first[2:]).name not in ('python', 'python3', 'python3.11', 'python3.12', 'python3.13', 'python3.14'):
raise DeploymentError('The existing aim launcher has no unambiguous Python shebang. Supply --aim-python explicitly; shell/env wrappers are not guessed.')
python = Path(first[2:])
if not python.is_absolute() or not python.is_file() or not os.access(python, os.X_OK) or any(c.isspace() for c in str(python)) or len(str(python)) > 120:
raise DeploymentError('--aim-python must be an executable absolute, space-free Python path (maximum 120 characters). Venv symlinks are supported.')
python = Path(os.path.abspath(python)) # do NOT resolve a venv symlink to the system Python
directory = directory if directory is not None else (Path(existing).parent if existing else Path('/usr/local/bin'))
if not directory.is_absolute():
raise DeploymentError('--bin-dir must be absolute.')
directory = canonical(directory)
if directory == Path('/') or directory == target or directory.is_relative_to(target / 'scripts/src'):
raise DeploymentError('Use a dedicated command directory, not the root or core source namespace.')
if directory.exists() and not directory.is_dir():
raise DeploymentError('The command directory is not a directory.')
for name, module in (('aim', 'aim.__main__'), ('aimctl', 'aim.ctl')):
dest = directory / name
no_symlink_parents(dest)
if dest.exists() or dest.is_symlink():
if dest.is_symlink():
try:
resolved = dest.resolve(strict=True)
except (OSError, RuntimeError):
raise DeploymentError('Refusing a broken launcher symlink: ' + str(dest)) from None
if not resolved.is_file():
raise DeploymentError('Launcher symlink does not resolve to a regular file: ' + str(dest))
text = _read_launcher_text(dest)
if LAUNCHER_MARKER in text:
try:
info = json.loads(text.splitlines()[2][2:])
except (ValueError, IndexError):
raise DeploymentError('Invalid AIM launcher metadata: ' + str(dest)) from None
if info.get('target') != str(target):
raise DeploymentError('AIM launcher belongs to another installation; choose its own --bin-dir.')
elif f'from {module} import main' not in text:
raise DeploymentError('Refusing to replace an unrecognized launcher. Choose a reviewed --bin-dir: ' + str(dest))
return EntryPoints(python, directory, target)
def plan(source, target, mode, *, entrypoints=None):
source, target = canonical(source), canonical(target)
no_symlink(source)
no_symlink(target)
if target == Path('/') or source == target or source.is_relative_to(target) or target.is_relative_to(source):
raise DeploymentError('Use separate extracted-source and installation directories; never / as target.')
if mode == 'update' and not (target / 'scripts/src/aim/__init__.py').is_file():
raise DeploymentError('Existing AIM source was not found; use install for a fresh tree.')
files = source_files(source)
operations = []
for relative, src in sorted(files.items()):
dest = target / relative
no_symlink(dest)
if dest.exists() and not dest.is_file():
raise DeploymentError('A non-file occupies a core destination: ' + relative)
if relative == 'scripts/aim.yml' and dest.exists():
continue # operator-owned, always preserved, even on first install
if dest.exists() and digest(src) == digest(dest):
continue
operations.append({'path': relative, 'action': 'replace' if dest.exists() else 'create',
'old_sha256': digest(dest) if dest.exists() else None,
'new_sha256': digest(src)})
# Only the Python core namespace is an authoritative replaceable directory.
# Other unlisted playbooks/roles/files remain operator-owned additions.
core = target / 'scripts/src/aim'
if core.exists():
for parent, dirs, names in os.walk(core, followlinks=False):
for d in dirs:
no_symlink(Path(parent) / d)
for name in names:
existing = Path(parent) / name
no_symlink(existing)
relative = existing.relative_to(target).as_posix()
if relative not in files:
if not existing.is_file():
raise DeploymentError('Unsupported core namespace entry: ' + relative)
operations.append({'path': relative, 'action': 'remove', 'old_sha256': digest(existing), 'new_sha256': None})
# Explicitly retired Core document names only; unknown operator documents stay.
# Removal is previewed and recorded in the same protected rollback journal.
retired_docs = (
# Root/scripts-root AIM-owned docs moved into scripts/docs.
# Component-local docs (deploy/README.md, role READMEs, playbook docs) stay beside their code.
'AGENTS.md', 'ADDON_AGENTS.md', 'scripts/CHANGELOG.md',
'scripts/docs/RC19_HANDOFF.md', 'scripts/docs/SANITY_3.2.0.md',
'scripts/docs/SANITY_3.2.1rc2.md', 'scripts/docs/VERIFICATION.md',
'scripts/docs/LOCAL_VALIDATION.md', 'scripts/docs/CONTROLLER_ACCEPTANCE.md',
)
for relative in retired_docs:
existing = target / relative
no_symlink(existing)
if existing.exists() and relative not in files:
if not existing.is_file():
raise DeploymentError('Non-file occupies a retired document: ' + relative)
operations.append({'path': relative, 'action': 'remove',
'old_sha256': digest(existing), 'new_sha256': None})
if entrypoints is not None:
if entrypoints.target != target or entrypoints.directory == source or entrypoints.directory.is_relative_to(source):
raise DeploymentError('Launcher target/directory conflicts with the extracted source.')
for relative, content in entrypoints.contents().items():
dest = _launcher_path({'path': relative, 'kind': 'launcher'}, target, entrypoints.directory)
files[relative] = content
current = _launcher_digest(dest)
if not dest.is_symlink() and dest.exists() and current == digest(content) and os.access(dest, os.X_OK):
continue
operations.append({'path': relative, 'kind': 'launcher', 'action': 'replace' if (dest.exists() or dest.is_symlink()) else 'create',
'old_sha256': current, 'new_sha256': digest(content)})
return files, operations
def atomic_file(source: Path | bytes, destination: Path, *, metadata=None, executable=False, allow_replace_symlink=False):
destination.parent.mkdir(parents=True, exist_ok=True)
if allow_replace_symlink:
no_symlink_parents(destination)
else:
no_symlink(destination)
fd, filename = tempfile.mkstemp(prefix='.aim-install-', dir=destination.parent)
staged = Path(filename)
try:
with os.fdopen(fd, 'wb') as stream:
if isinstance(source, bytes):
stream.write(source)
else:
with source.open('rb') as incoming:
shutil.copyfileobj(incoming, stream)
stream.flush()
os.fsync(stream.fileno())
if metadata is None and destination.exists() and not destination.is_symlink():
old = destination.stat()
os.chown(staged, old.st_uid, old.st_gid)
# Preserve ACLs/attributes, not the old file timestamp.
for key in os.listxattr(destination):
os.setxattr(staged, key, os.getxattr(destination, key))
staged.chmod(stat.S_IMODE(old.st_mode) | (0o111 if executable else 0))
elif metadata is not None:
os.chown(staged, metadata['uid'], metadata['gid'])
for key, value in metadata.get('xattrs', {}).items():
os.setxattr(staged, key, base64.b64decode(value, validate=True))
staged.chmod(metadata['mode'])
else:
staged.chmod(0o755 if executable else 0o644)
os.replace(staged, destination)
dfd = os.open(destination.parent, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
staged.unlink(missing_ok=True)
def apply(source, target, mode, backup_root, *, entrypoints=None):
source, target, backup_root = canonical(source), canonical(target), canonical(backup_root)
files, operations = plan(source, target, mode, entrypoints=entrypoints)
version = entrypoints.verify_python(source) if entrypoints else None
launcher_dir = entrypoints.directory if entrypoints else None
if not operations:
if entrypoints:
entrypoints.verify_installed(version)
print('AIM source and selected entry points are already current; operator configuration was preserved.')
return None
no_symlink(backup_root)
if backup_root == target or backup_root.is_relative_to(target) or backup_root == source or backup_root.is_relative_to(source):
raise DeploymentError('Backups must be outside the installation and extracted source trees.')
backup_root.mkdir(parents=True, exist_ok=True)
backup = Path(tempfile.mkdtemp(prefix=datetime.now(timezone.utc).strftime('%Y%m%dT%H%M%SZ-'), dir=backup_root))
backup.chmod(0o700)
record = {'target': str(target), 'state': 'preparing', 'entries': [], 'created_directories': [],
'format': 2, 'launcher_dir': str(launcher_dir) if launcher_dir else None}
for operation in operations:
dest = _launcher_path(operation, target, launcher_dir)
item = dict(operation)
if dest.is_symlink():
item['old_kind'] = 'symlink'
item['link_target'] = os.readlink(dest)
elif dest.exists():
st = dest.stat()
item['old_kind'] = 'file'
item['metadata'] = dict(uid=st.st_uid, gid=st.st_gid, mode=stat.S_IMODE(st.st_mode),
xattrs={key: base64.b64encode(os.getxattr(dest, key)).decode('ascii') for key in os.listxattr(dest)})
recovery = backup / 'files' / operation['path']
recovery.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(dest, recovery)
recovery.chmod(0o600)
record['entries'].append(item)
def save_record():
fd, temporary = tempfile.mkstemp(prefix='.recovery-', dir=backup)
try:
with os.fdopen(fd, 'w', encoding='utf-8') as stream:
json.dump(record, stream, indent=2)
stream.flush()
os.fsync(stream.fileno())
os.replace(temporary, backup / 'recovery.json')
dfd = os.open(backup, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(dfd)
finally:
os.close(dfd)
finally:
Path(temporary).unlink(missing_ok=True)
save_record()
try:
record['state'] = 'applying'
save_record()
for item in record['entries']:
dest = _launcher_path(item, target, launcher_dir)
current = _launcher_digest(dest) if item.get('kind') == 'launcher' else (digest(dest) if dest.exists() else None)
if current != item['old_sha256']:
raise DeploymentError('Source changed during installation; stop writers and retry.')
item['started'] = True
save_record() # persist intent before changing any installed file
if item['action'] == 'remove':
dest.unlink()
else:
if digest(files[item['path']]) != item['new_sha256']:
raise DeploymentError('Extracted release source changed during installation.')
missing_dirs = []
parent = dest.parent
while not parent.exists():
missing_dirs.append(str(parent))
parent = parent.parent
record['created_directories'].extend(missing_dirs)
atomic_file(files[item['path']], dest, executable=item.get('kind') == 'launcher',
allow_replace_symlink=item.get('kind') == 'launcher')
item['applied'] = True
save_record()
if entrypoints:
entrypoints.verify_installed(version)
record['state'] = 'completed'
save_record()
print('AIM core source installed. Recovery directory: ' + str(backup))
print('Preserved existing scripts/aim.yml, inventories, Vaults, keys, add-ons, virtual environments and unlisted customer files.')
print('No dependencies, OS identities, permissions policy or services were provisioned.')
if entrypoints:
print('AIM and aimctl launchers: ' + str(entrypoints.directory))
print('Ensure this directory is in the operator PATH; use hash -r in existing shells.')
return backup
except BaseException:
# Ordinary failures can restore the source files already touched. A power
# loss/kill -9 is not an atomic whole-tree transaction: retain recovery data.
for item in reversed(record['entries']):
if not item.get('started') or item['path'] == 'scripts/aim.yml':
continue
dest = _launcher_path(item, target, launcher_dir)
now = _launcher_digest(dest) if item.get('kind') == 'launcher' else (digest(dest) if dest.exists() else None)
if now == item['old_sha256'] and not (item.get('kind') == 'launcher' and dest.exists() and
stat.S_IMODE(dest.stat().st_mode) != item.get('metadata', {}).get('mode')):
continue
if now != item['new_sha256']:
raise DeploymentError('A concurrently modified file prevented rollback; use the protected recovery directory.')
if item['old_sha256'] is None:
dest.unlink(missing_ok=True)
elif item.get('old_kind') == 'symlink':
_restore_symlink(dest, item['link_target'])
else:
atomic_file(backup / 'files' / item['path'], dest, metadata=item['metadata'])
record['state'] = 'rolled_back_after_error'
save_record()
raise
def rollback(backup: Path, target: Path, *, execute: bool):
backup, target = canonical(backup), canonical(target)
path = backup / 'recovery.json'
if path.is_symlink() or not path.is_file():
raise DeploymentError('Recovery metadata is missing or unsafe.')
record = json.loads(path.read_text(encoding='utf-8'))
if record.get('target') != str(target) or record.get('state') not in ('completed', 'applying'):
raise DeploymentError('Recovery target/state does not match this installation.')
actions = []
launcher_dir = Path(record['launcher_dir']) if record.get('launcher_dir') else None
for item in record['entries']:
if not (item.get('started') or item.get('applied')):
continue
relative = Path(item['path'])
if relative.is_absolute() or '..' in relative.parts or relative.as_posix() == 'scripts/aim.yml':
# Initial install may have created aim.yml: never delete an operator's
# subsequent configuration through rollback. Always preserve this file.
if relative.as_posix() == 'scripts/aim.yml':
continue
raise DeploymentError('Unsafe recovery path.')
if item.get('kind') != 'launcher' and not allowed(relative) and relative.parts[:3] != ('scripts', 'src', 'aim'):
raise DeploymentError('Recovery may only address the core source namespace.')
dest = _launcher_path(item, target, launcher_dir)
now = _launcher_digest(dest) if item.get('kind') == 'launcher' else (digest(dest) if dest.exists() else None)
if now == item['old_sha256'] and not (item.get('kind') == 'launcher' and dest.exists() and
stat.S_IMODE(dest.stat().st_mode) != item.get('metadata', {}).get('mode')):
continue # interruption before publication, or an already restored entry
if now != item['new_sha256']:
raise DeploymentError('Installed core source changed since deployment; refusing to overwrite it during rollback: ' + str(relative))
if item['old_sha256'] is not None and item.get('old_kind') != 'symlink':
recovered = backup / 'files' / relative
no_symlink(recovered)
if digest(recovered) != item['old_sha256']:
raise DeploymentError('Recovery file checksum mismatch.')
actions.append(item)
print('Rollback source files: ' + str(len(actions)))
if not execute:
print('Dry run only. Use --apply --quiesced to restore these source files.')
return
for item in reversed(actions):
dest = _launcher_path(item, target, launcher_dir)
if item['old_sha256'] is None:
dest.unlink(missing_ok=True)
elif item.get('old_kind') == 'symlink':
_restore_symlink(dest, item['link_target'])
else:
atomic_file(backup / 'files' / item['path'], dest, metadata=item['metadata'])
print('Core source and recorded launchers restored; no remote Ansible work was reversed. Use hash -r and verify aim/aimctl for the restored release.')
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('operation', choices=('install', 'update', 'rollback'))
parser.add_argument('--target', type=Path, default=Path('/etc/ansible'))
parser.add_argument('--backup-dir', type=Path, default=Path('/var/backups/aim-core'))
parser.add_argument('--from-backup', type=Path)
parser.add_argument('--aim-python', type=Path, help='Existing AIM interpreter; inferred only from an unambiguous installed aim Python shebang')
parser.add_argument('--bin-dir', type=Path, help='Install aim and aimctl here; defaults to the existing aim command directory or /usr/local/bin')
group = parser.add_mutually_exclusive_group()
group.add_argument('--apply', action='store_true', help='Apply the planned core-source replacement')
group.add_argument('--dry-run', action='store_true', help='Preview only (the default)')
parser.add_argument('--quiesced', action='store_true', help='Confirm CLI/add-on jobs and other source writers have been stopped')
args = parser.parse_args(argv)
try:
if not args.target.is_absolute():
raise DeploymentError('An absolute non-root installation directory is required.')
args.target = canonical(args.target)
if args.target == Path('/'):
raise DeploymentError('An absolute non-root installation directory is required.')
if args.apply and not args.quiesced:
raise DeploymentError('Stop active CLI/add-on jobs and retry with --apply --quiesced.')
if args.operation == 'rollback':
if not args.from_backup:
raise DeploymentError('rollback requires --from-backup.')
if args.apply:
with deployment_lock(args.target):
rollback(args.from_backup, args.target, execute=True)
else:
rollback(args.from_backup, args.target, execute=False)
return 0
source = Path(__file__).absolute().parents[1]
entrypoints = entry_points(args.target, args.aim_python, args.bin_dir)
entrypoints.verify_python(source)
_, operations = plan(source, args.target, args.operation, entrypoints=entrypoints)
print('Target: ' + str(args.target))
print('AIM interpreter: ' + str(entrypoints.python))
print('Command directory: ' + str(entrypoints.directory))
for operation in operations:
print(operation['action'].upper() + ' ' + operation['path'])
print('Planned file operations: ' + str(len(operations)))
print('KEEP existing scripts/aim.yml and all unlisted runtime/add-on/customer files.')
if not args.apply:
print('Dry run only. Apply from this extracted archive with --apply --quiesced.')
return 0
args.target.mkdir(parents=True, exist_ok=True)
with deployment_lock(args.target):
apply(source, args.target, args.operation, args.backup_dir.absolute(), entrypoints=entrypoints)
return 0
except (DeploymentError, OSError, ValueError, KeyError, TypeError) as exc:
print('AIM deployment stopped: ' + str(exc), file=sys.stderr)
return 1
if __name__ == '__main__':
raise SystemExit(main())