Skip to content

H8: Composition regression-guard tests for the H1-H7 chain

Placeholder ppxd requested to merge feat/upgrade-hardening-h8-e2e-matrix into main

Summary

Final phase of the 1.8.0 Upgrade Hardening initiative (H1-H7 already merged).

Closes the initiative with a composition-layer test class that asserts the cumulative H1-H7 behavior the operator experiences. Each individual phase has its own pinned unit suite; H8 adds the composition layer — tests that exercise multiple H-phases together and would catch a regression that breaks one link in the chain while another link masks it.

What H8 specifically prevents

If a future refactor:

  • Drops H1's cold-cache short-circuit (but H5's UpgradeAsync guard stays)
  • Reverts H2's persistence (but H1 still short-circuits)
  • Renames an H3 error code (FE consumers parsing the string break)
  • Re-introduces the Linux historical default (but H1's check still fires)

→ the per-phase tests might still pass, because each only validates its own surface. H8's composition tests fail because they exercise multiple phases together and assert the cumulative property.

Test breakdown (11 total)

# Test Pins
1 H1_ColdCacheTentacleStyle_ProducesNoOsDetected_NotLinuxDockerHubError Cold cache → 0 slots projected (operator-visible: no Docker Hub message)
2 H1_LongFormWindowsOs_ProjectsToWindowsSlot_NotUnknown Full chain: WindowsOsStringHelper → IsWindows → MachineCapabilitySet.From
3 H7_IISDeployToMachineWithoutIIS_BlockedAtPlanTime_NotAtRuntime Real CapabilityValidator + real projection: role:iis missing → violation
4 H7_IISDeployToMachineWithIIS_NoViolation_DispatchProceeds Inverse: role:iis present → no violation (positive case pin)
5 H7_PreH7Agent_DoesNotAdvertiseRoles_OptimisticAllowKeepsExistingFleetsWorking Backward-compat: empty InstalledRoles → no role:* slots
6 H6_LinuxScriptExitCode4_MapsToRolledBack_NotFailed LinuxExitRolledBack (= 4) + MachineUpgradeStatus.RolledBack (= 5) both pinned from same test
7-10 H3_AllErrorCodes_LowerSnakeCase_StableForFEConsumption (Theory × 4) All four error codes follow lower_snake_case convention
11 Operator17xFailureChain_EveryLinkHasAHardeningPin Documentation-as-test: lists every link in the chain with the H1-H7 pin guarding it

Why NOT real E2E (Halibut + Postgres + Tentacle)?

The Tentacle E2E projects already have extensive upgrade coverage:

  • tests/Squid.LinuxTentacleE2ETests/: 5 upgrade-specific test files
    • TentacleLinuxUpgradeBinaryIntegrationE2ETests.cs
    • TentacleLinuxUpgradeLifecycleE2ETests.cs
    • TentacleLinuxUpgradePollingCompositeE2ETests.cs
    • TentacleLinuxServerRestartReconnectE2ETests.cs
    • UpgradeLinuxScriptE2ETests.cs
  • tests/Squid.WindowsTentacleE2ETests/TentacleUpgradeE2ETests.cs
  • tests/Squid.IntegrationTests/Services/Machines/Upgrade/UpgradeDispatchLockReconcilerIntegrationTests.cs (real Redis)

Adding 44 new E2E scenarios (per the original H8 plan) would duplicate substantial existing coverage. The composition tests added here run in milliseconds without infra prerequisites — part of every dev's local test cycle, not the slow E2E lane.

Test plan

  • 11 new composition tests exercising H1-H7 surfaces together
  • 5596/5596 unit tests green (+11 net new vs 5585 baseline post-H7)
  • Existing E2E + integration coverage referenced inline in test docstrings
  • Manual operator validation: replay original 1.7.x failure path (cold-cache Windows machine + dispatch Upgrade) — expect NoOsDetected with health-check hint, NOT Docker Hub message

After H8 merges — what to do

  1. Merge this PR
  2. Close milestone 1.8.0
  3. Tag + release 1.8.0 with all 8 PRs in the changelog
  4. Open milestone 1.8.1

The 1.8.0 changelog summary

Phase PR What it does for operators
H1 #354 Structured UpgradeEligibilityReason codes + cold-cache NoOsDetected + OS-aware messages (no more misleading "Docker Hub" hints to Windows operators)
H2 #355 Capability cache persisted to DB — server pod restart no longer wipes operators into the cold-cache trap
H3 #356 Active health-check returns structured ManualHealthCheckResult so FE can show fresh OS/version + diagnose unreachable agents
H4 #357 Lock contention message includes dispatchedAt + targetVersion + expected-remaining time (no more confusing "wait under 2 minutes" fib)
H5 #358 Linux strategy strict-OS-only; direct POST /upgrade with cold cache rejected (closes the bypass that the FE's H1 gate couldn't catch)
H6 #359 New MachineUpgradeStatus.RolledBack (vs. Failed) so operators see "safely restored to baseline" not "broken"
H7 #360 role:* capability slots → IIS deploy to non-IIS machine fails at plan-time with actionable hint, not runtime with Import-Module WebAdministration
H8 #361 (this) Composition regression-guard tests pinning the cumulative behaviour

Merge request reports