docserver: self-healing Task Scheduler config + docs

Companion to the worker MinIO-retry fix. Makes the worker auto-recover from
process death (crash, manual kill, missed boot trigger), not just MinIO outages.

- start_worker.bat: propagate Python's exit code (exit /b %rc%) so Task
  Scheduler can actually detect a failed run (it previously always exited 0).
- reconfigure_task.ps1 (new): re-registers PW-DocserverWorker with
  RestartCount=99 / 1-min interval, StartWhenAvailable, and two triggers —
  AtStartup plus a 5-min repeating trigger with MultipleInstances=IgnoreNew, so
  a dead worker relaunches within ~5 min and never double-runs. Idempotent.
- install.ps1: same self-healing settings for fresh installs.
- Verified on the box: killed the worker -> task relaunched it; firing again
  while running stayed at one instance.

Docs updated to match reality:
- docserver/README.md: new 'Reliability / self-healing' section.
- document-generation.md: corrected the stale 'Flask DocServer :5050 / HTTP'
  description to the actual MinIO outbound-only transport.
- e2e-test-plan.md: removed the outdated 'Word COM fails under SYSTEM / requires
  RDP after every reboot' limitation; now self-healing under SYSTEM session 0.
- infrastructure.md: fixed VM spec (Win Server 2019, Word 16.0, Python 3.13,
  SSH port 22422) + self-healing note.
- architecture.md / formation-system.md: trigger + self-healing details.
This commit is contained in:
justin 2026-06-15 22:49:21 -05:00
parent 7929413eeb
commit b48d0cb799
9 changed files with 150 additions and 24 deletions

View file

@ -166,11 +166,18 @@ $action = New-ScheduledTaskAction `
-Argument "/c `"$AppDir\start_worker.bat`"" `
-WorkingDirectory $AppDir
$trigger = New-ScheduledTaskTrigger -AtStartup
# Two triggers for self-healing: at boot, plus a repeating 5-minute safety net
# that relaunches the worker if its process ever dies (crash, manual kill, or a
# missed boot trigger). MultipleInstances=IgnoreNew keeps it to one instance.
$atStartup = New-ScheduledTaskTrigger -AtStartup
$repeat = New-ScheduledTaskTrigger -Once -At (Get-Date) `
-RepetitionInterval (New-TimeSpan -Minutes 5)
try { $repeat.Repetition.Duration = 'P3650D' } catch {} # some builds need an explicit long duration
$trigger = @($atStartup, $repeat)
$settings = New-ScheduledTaskSettingsSet `
-ExecutionTimeLimit (New-TimeSpan -Hours 0) `
-RestartCount 10 `
-RestartCount 99 `
-RestartInterval (New-TimeSpan -Minutes 1) `
-StartWhenAvailable `
-MultipleInstances IgnoreNew `