# v2.2.16: what changed for each point of the v2.2.15 review

Package `marianos-scraper-2.2.16.zip`
SHA-256 `04f5c4bca887d9f04f408162fc97b378476b9a922bf8f21d94180df6c26b94b8`, code fingerprint `4d768340f2d4ce48`.
96 offline tests pass on Windows (Python 3.13.17, fresh venv, pinned `requirements-dev.txt`) and on Linux (Python 3.12.3).
`infra/run_probe.ps1` SHA-256 `b7ae0da2...5e7613e` (only its comment changed). Full diff: `marianos-scraper-2.2.15-to-2.2.16.diff`.
No VM was started and nothing was sent to Oxylabs or marianos.com.

## 1. Durable shared reservation in Blob (new `marianos_scraper/allowance.py`)

* The probe now works on the ledger blob itself (`--ledger blob:<url>`, VM managed identity). The session script no
  longer downloads a copy or uploads one afterwards, so a stale copy can never overwrite a newer ledger.
* Every change is one read-modify-write under an exclusive 30 s blob lease. It is written with `If-Match` on the ETag
  that was read and the lease id, then read back and compared byte for byte.
* **Reserve before dispatch:** an entry with `status reserved`, `reserved_jobs` (this run's cap, at most what is left)
  and a `reservation_id`. It is written and verified before the first request. If the lease, the ETag condition or the
  read-back fails, the probe stops with exit 2 and sends nothing.
* **Final update:** done on a fresh read, matched by `reservation_id`, and verified the same way. If it cannot be
  verified, the run ends with exit 2 and `allowance.final_update = "UNRESOLVED: ..."`. The reservation stays counted in
  full, and no unverified ledger copy goes into the evidence.
* **Reconciliation on restart:** any entry still `reserved` becomes `unfinished`. That covers a run that was
  interrupted, killed, or whose final update was not verified. Such an entry stays counted at its full reservation and
  is never lowered automatically.
* A missing ledger blob, or an entry with neither dispatched jobs nor a reservation, stops the probe (fail closed).
  The two existing entries (1 + 3 jobs) are read as they are.
* Tests, all in `tests/test_oxylabs_probe.py` (a fake blob with Azure's lease and If-Match rules):
  * the reservation is written before the first POST and the final count after the last one;
  * **an interruption after the reservation and dispatch, before the final update and the evidence upload, followed by
    a fresh session:** the fresh session counts the full reservation (cap 15 − 4 − 5 = 6) and marks the entry
    unfinished; when nothing is left, a further session sends nothing;
  * a lease held by another execution stops the probe before anything is sent;
  * a stale ETag cannot overwrite a newer ledger;
  * a write without the lease is refused;
  * a read-back mismatch at reservation stops before dispatch;
  * a read-back mismatch at the final update ends with exit 2 and the ledger UNRESOLVED;
  * a missing or malformed ledger fails closed;
  * the local-file ledger uses an exclusive lock.

## 2. Bounded orchestrator (`operator-tools/probe_orchestrator.py`)

* `--session-deadline` (default 3,600 s, monotonic) covers everything before the deallocation. The VM start (at most
  900 s) and every polling loop stop at that deadline; when it passes, the orchestrator goes straight to the
  deallocation. A probe still running on the VM is ended by the deallocation, and its reservation stays counted.
* Deallocation runs in `finally`: 3 attempts, each with at most 600 s of operation polling and 300 s of power-state
  confirmation. An unreadable power state never counts as deallocated.
* Exit codes: 0 = done and confirmed deallocated; 2 = failed or deadline reached, confirmed deallocated; 3 = **not
  confirmed**, printed loudly with the manual `az vm deallocate` command.
* `operator-tools/test_orchestrator.py` (virtual clock, fake Azure) has 6 tests: normal run; a session that never ends
  is cut at the deadline and deallocated; a failed start still deallocates; a deallocation that never completes is
  retried a bounded number of times and exits 3; an unreadable state exits 3; no credential route exists.

## 3. Key Vault / managed identity only (`operator-tools/vm_probe_session.tpl.ps1`)

* The encrypted hand-over is gone: `cred_send.ps1`, the RSA key exchange and the OXU/OXP environment route. The test
  above asserts they no longer exist.
* If the Key Vault read fails or returns an empty secret, the session stops with exit 5 and nothing is sent. It reports
  the Key Vault error (HTTP status and message) in the session log and as Application event 1202.
* Dry run of the filled script with local stand-ins (`test-logs/session-dry-run-*.txt`):
  * normal path: hashes, venv, 96 tests on the "VM", and the launcher receives `-Ledger blob:https://.../probe/cumulative-usage.json`;
  * Key Vault failure path: exit 5 with the error.

## 4. Full-run retention regression (`tests/test_scraper.py`)

`test_unavailable_and_later_absent_products_survive_repeat_runs_with_their_real_last_observation` runs
`SiteRunner.run()` three times against one offline store (every category, LocalStorage, all validation checks on):

* **Oct 01:** everything is listed.
* **Oct 02:** product GONE is no longer listed, and product OOS is listed but not sellable.
* **Oct 03:** both are absent.

Every published output keeps every product. A carried record keeps the `crawl_date` of its last observation: GONE
shows Oct 01 in both later runs, and OOS shows Oct 02 in the third run. Run 2 does not reset GONE's date. The record
itself is unchanged from that observation. `availability.jsonl` marks each product `fresh` or `carried_forward` with
`last_observed_run`; the fresh records carry the current date, and `availability_changes.json` lists both carried
products. Two deliberate code mutations make the test fail: re-dating carried records, and dropping them.

## 5. Read-only exports (`azure-exports-2026-11-01/`, see its README)

* The Blob ledger byte for byte: 4 of 15 jobs counted.
* The complete Log Analytics query result, as JSON and CSV, untruncated: 18 ShifullahScraper events.
* A per-table summary of the workspace.
* The VM's Activity Log, 230 events, with sign-in claims and client IPs removed, plus a start / runCommand /
  deallocate summary.

Nothing in Azure was started or changed.