Browse documentation

🧭 Start Here

Blackcap Overview ✨ Blackcap Feature Catalog 🤖 AI Features 🧱 Technology, Administration, and Reliability Installation First Run

📘 User Guides

Administration and Operations Platform Configuration Organization Configuration Email and Consent Management

🚀 Deploy Blackcap

Platform Stacks and Raspberry Pi Hardware Raspberry Pi Deployment Raspberry Pi Client Services GCP Deployment Packaged Blackcap deployment Application Updates After Your First Blackcap Login: Self-Hosted Setup Guide Blackcap Release Notes Environment Variables and Secrets Reverse Proxy and TLS Background Jobs and Schedules

🛠️ Administer Blackcap

Organizations Users, Permissions, and Authentication Configuration Workspace Backups and Restore Database Administration Regression Testing Performance and Job Status Audit, Access Activity, and Logging GeoIP and Access Location Data Retention and Purge Support Chat Administration Support Requests API Tester and Postman Instance Reporting Third-Party Packages and Attribution AI Provider Usage and Billing Portals Microwave Pie licensing for Blackcap Microwave Pie licensing for Blackcap

🍽️ Use Recipes

Recipes and the Recipe Library Recipe Import and Discovery Recipe Editing and Cache Artifacts Recipe Sharing Social Recipe Import AI Recipe Image Generation 🪄 Adapt Recipe and 🎲 Variations

📅 Plan Meals

Meal Planner

🛒 Use Shopping Lists

Shopping Lists and Shop a List External and Household Shopping 🔗 Chrome Extension Shop With

🧺 Manage Kitchen Inventory

🧺 Kitchen Inventory

🖥️ Use Displays

Displays and Connections Assigning and Scheduling Display Content Remote Pi Client E-Ink Rendering Menu Refresh and Rendering Noun Project Footer Images

🧑‍🍳 Cook with Let’s Cook

🧑‍🍳 Let’s Cook 🧑‍🍳 Let’s Cook Controls and Timers

🤖 Use and Administer AI

🤖 AI in Blackcap 🤖 AI Providers and Connections 🤖 AI Seeds and Usage Support Chat

🔗 Use the Chrome Extension

🔗 Blackcap Chrome Extension 🔗 Chrome Extension Recipe Capture 🔗 Chrome Extension Shop With 🔗 Chrome Extension Release and Privacy

🎮 Play Games

Games and trivia Bramble Harvest — Player Guide Flavor Lab — Player Guide

🔌 Integrations

Email Integration Cloud Storage Integrations Voice Assistants Shop With Integrations Authentication Providers

⚙️ Develop Blackcap

Application Architecture Database Service and Data Access SQLite and PostgreSQL Database Migrations Background Job Architecture Testing Games platform — current developer reference Bramble Harvest developer tools API Architecture Security and Organization Scoping UI, Icons, and Documentation Assets Blackcap-Safe Emoji Support Chat Architecture Extending Support Chat Support Chat Model Benchmark Documentation Standards Terminology

⚠️ Troubleshoot Blackcap

⚠️ Troubleshooting Licensing Troubleshooting Deployment Troubleshooting Display Troubleshooting Recipe Import Troubleshooting 🤖 AI Troubleshooting Backup Troubleshooting Database Troubleshooting Diagnostic Organization Clones Support Chat

Regression Testing

Audience: System Admin, Developer, Support Related: Performance And Job Status · Testing · Application Updates

Blackcap includes an administrative regression harness for immediate, scheduled, resumable, and diagnostic-clone runs. Test activity is organization-attributed, results are stored in the database, and the UI supports filtering, details, exports, performance comparison, cleanup, and completion email behavior.

Regression documentation is intentionally detailed because the harness is both a product feature and the primary end-to-end verification mechanism. Keep test names and expected behavior synchronized with the code.

AI spend summaries: Run Details shows charged AI Seeds and estimated provider USD cost with a use-case breakdown on hover. CSV exports also include a best-effort Seed-spend attribution per test (not shown as a default table column). The detailed source is AI Usage within the regression organization, while it still exists. Before automatic regression-organization cleanup, Blackcap snapshots the run total for historical reporting. See Regression AI Seed spending.

Blackcap includes a System Admin-only regression harness for creating isolated test organizations, accepting generated invites without real email delivery, logging in as generated users, exercising key Blackcap routes/API-token behavior, collecting structured step results, and optionally cleaning up through the normal account-delete / delete-last-user-in-organization path.

Synthetic versus real AI provider regression mode

Use real AI providers is an optional checkbox in the Regression Tests run configuration, off by default. The setting is saved with each run (including scheduled and diagnostic runs) and displayed in Run Details. Both modes use the same underlying fixture inputs (recipes, meal slots, dietary requests, and Support Chat prompts); only the AI response generation is switched:

  • Unchecked — synthetic: uses run-scoped deterministic adapters for Plan Meal, Plan Meals, image generation, and single/bulk Adapt Recipe. Support Chat uses its existing regression fixture response. No external AI requests are required for those AI responses; normal workflow and AI Seed reservation/settlement rules are still exercised.
  • Checked — real: submits fixtures through Blackcap's ordinary durable AI jobs to eligible configured OpenAI, Gemini, or Anthropic providers. The tests use real provider results and check required response structure, source links, Seed reservation/settlement, AI Usage and saving without comparing variable AI prose byte-for-byte. A real-provider run never falls back to synthetic results.

Missing real-provider prerequisites are Skipped, not Failed. When a use case lacks an enabled real Quality Profile, a compatible enabled provider, configured credentials, or a valid model, that AI test records a clear Skipped step before a paid request. The Support Chat endpoint uses its configured default profile; if the default is unusable it is also skipped. When a real request is submitted, provider errors, invalid results, timeouts, or incorrect accounting are Failed tests. A missing test fixture that the regression itself should have created is likewise a test failure, not missing provider configuration.

Same-scenario coverage in both modes (October 20): Real-provider testing now continues through the established Plan Meal and Plan Meals lifecycle assertions instead of stopping after a single successful provider job. For Plan Meal that includes preserved saved plans, AI Usage/What Happened, whole-meal image where an eligible image profile exists, Meal review/PDF, moving the meal, planned Let’s Cook, invalidation/replanning, deletion and sequential fallback. For Plan Meals it includes classification/naming guards, additive Meal Slot behavior, planning guidance, retained evidence and a second duplicate-safety request. Recipe image generation checks image preview, organization-scoped AI Usage privacy, candidate selection and actual recipe saving. Support Chat uses the same conversation, escalation, Extension, privacy and transcript scenarios in both modes, with the real default Quality Profile rather than a synthetic fixture profile. Assertions about variable AI output are schema/behavior-based; tests of intentionally injected synthetic provider errors remain identified as synthetic-only fault-injection tests (Skipped in real mode). Missing real credentials/profile skip their dependent scenarios; no run ever silently substitutes synthetic AI responses.

Provider cost and run time: Matching the synthetic lifecycle means a real-provider run may make multiple paid requests for Plan Meal (initial plan, image, replan), Plan Meals (first plan and additive retry), recipe images, and Support Chat. This is deliberate end-to-end coverage and is more expensive and slower than the earlier single-request smoke tests. Synthetic remains the default and recommended regular schedule for Raspberry Pi. A missing optional real image profile skips only the image-dependent Plan Meal step, not its unrelated text-planning checks.

Real Plan Meal verification: The completed provider job must correlate to a saved current Meal Plan whose accepted recipe steps are found in the nested plan.plan.steps field of GET /admin/meal-planner/meal-plan (the outer plan holds persistence metadata). Regression verifies a positive total duration, at least one accepted step, the saved AI job ID, a settled Seed reservation, and successful AI Usage for the actual provider. Failed checks now identify each unsuccessful condition rather than reporting only a terminal job status.

Real-provider mode can consume real provider credits/cost. The suite grants required test organization AI Seeds where appropriate; this does not waive any provider fees. Synthetic mode remains the fast, offline default for Pi and low-resource deployments. Run Details/AI Usage separate provider-mode evidence and usage costs. The checkbox does not change fixture creation, the actual provider configuration, or unrelated production AI requests.

Scheduled runs: The Schedule Runs panel has its own Use real AI providers checkbox, independent of the Start New Run checkbox and the shared manual-run defaults. It is unchecked by default, including on existing schedules upgraded to this release. Each scheduled run snapshots that choice when queued; the run details identify which provider mode was used. The scheduled selection determines whether AI steps use real providers or synthetic adapters while preserving the same fixtures and skip/failure rules above. Enabling it may incur recurring provider charges. Changing the manual-run default will not silently switch a scheduled run to real AI.

Create Recipe, Combine Recipes, and Plan Meals original/image regression (0.7.3-alpha)

Current suite: 2026.10.24.create-combine-plan-ai.8. The Plan Meals image regression now scopes child recipe creation, image generation, image attachment, and result verification to the original regression organization. This fixes saved recipes from non-default organizations being invisible to the image-generation lookup ("Recipe not found") without weakening organization isolation. The related local test simulates lookup from an unrelated default organization and verifies the context is restored afterward. Prior coverage remains: the October 24 follow-up adds lease-aware waiting for optional Plan Meals image jobs, clearer failure diagnostics, and saved AI recipe/image counts in the completed Plan Meals result. The earlier original GCP synthetic regression follow-up fixed the Plan Meals polling URL encoder being shadowed by the quote HTTP response; it also replaced all seven wildcard reservation queries with explicit columns, bringing the Direct SQL Audit back to zero findings. Synthetic Create Recipe and Plan Meals child tests now read the shared regression HTTP client’s json_body field (including accepted HTTP 202 jobs). The synthetic Recipe Image profile uses the same narrow capability validation as other regression image fixtures, and provider-level authorization still requires the registered test run and organization. The run summary is intentionally a short two-sentence description rather than a catalog of every individual check. Normal runs continue past a failing optional Create/Combine fixture by recording a failed regression step; independent checks and cleanup must still run. A failed Direct SQL Audit records its first actionable file/line in exported results. A full run typically contains hundreds of steps, depending on options and environment; an unhandled harness exception is not a normal way to finish a run.

The normal System Admin regression and its diagnostic-clone equivalent exercise Create Recipe and Combine Recipe workflows in their own disposable regression organizations; the normal Plan Meals regression additionally runs the two original-recipe scenarios. These are not page-layout or button-absence tests. In addition to the non-charging options, unconfirmed-request and Seed-quote guards, coverage includes:

  • Create Recipe: choose the lowest configured eligible Create Recipe Quality Profile for the selected provider mode; submit an original recipe through the normal AI endpoint, wait for the durable job, validate ingredient and cooking-step output, save the new Library recipe, and reconcile the settled reservation and AI Usage to that profile's AI Seed price.
  • Combine via Adapt Recipe: generate a comparison between the new original and an independently created recipe, validate the compatible choices, confirm a selected attribute and submit the second generation, then save the combined recipe. Check the two linked provider transactions and their separate settled Seed charges, the original-recipe relationship, and selected-input persistence. The original recipe stays unchanged.
  • Plan Meals + Create Recipe, without image: quote the planned slots and their worst-case recipe charges separately. Run a two-day Plan Meals fixture with one empty slot and one existing main, require at least one generated and saved original, verify that the Meal Slot receives that recipe rather than being left with an unresolved idea, and verify that the initial parent hold matches the full approved maximum, the final parent charge contains only planning Seeds, generated children settle against transferred allocations, and unused Seeds are released.
  • Plan Meals + Create Recipe + Create Image: repeat with a separate Meal Slot fixture. The estimate now includes per-recipe image Seeds, the actual image job must settle its own reservation, and the accepted image must be attached to the persisted Recipe rather than merely left as a draft candidate.

Synthetic mode is the default and uses private, run-scoped Create/Adapt/Plan Meals/Image fixture profiles. Synthetic Create and Adapt profile selection is verified against the exact persisted provider connection owned by the regression run; if a synthetic fixture is unavailable, the affected scenario Fails instead of being Skipped. Fixtures use the same endpoints, validators, durable jobs and normal AI Seed ledger as production but do not send external requests. Test-only provider credentials/profiles are removed after each scenario; the Plan Meals image fixture is accepted only for its registered, active regression organization and run. Real mode selects the lowest-priced enabled, credentialed real profile independently for Create Recipe, Adapt Recipe, Plan Meals and Create Image. If one is missing, the dependent scenario is Skipped without falling back to a fixture. An actual provider failure remains Failed. Real mode can issue additional paid requests (one original, two combination stages, and Plan Meals plus recipe/image child requests). Only run real-provider regressions when that cost is acceptable.

The detailed results, selected Quality Profiles, requested and charged Seeds, provider job identifiers, and failures are recorded in Regression History. Completion emails include the standard email-preferences footer: a recipient-specific Tracker link when available, otherwise the public Consent preferences page. The fixtures use unique job idempotency keys; a resumed scenario reuses stored jobs where available instead of intentionally double-submitting the provider request. Diagnostic clones never charge the source organization. Unknown-outcome provider requests must still be examined before retrying them manually.

The standard regression suite includes a Consent Management Integration group for registered online Blackcap installations. It intentionally validates the real production-style service boundaries instead of creating fake recipient addresses or changing preference state:

  1. authenticate to the Microwave Pie management service at tracking.microwavepie.com/api/v1/instances/consent/status with the same permanent registered-instance bearer token used by Health Monitoring;
  2. validate the returned Consent capability contract, including all four email categories plus the email-policy/embed-session paths and public Consent base URL; and
  3. independently request https://consent.microwavepie.com/consent/health over HTTPS to verify the public alias, TLS, proxy routing, and host-specific Consent service contract.

No recipient email address, preference row, iframe session, or unsubscribe token is created by this regression group. An unregistered/offline installation skips the registered-instance checks because it does not participate in centrally managed optional email. A registered instance treats connectivity, authentication, contract, or public-host failures as real regression failures. The group runs in both the normal full regression suite and Diagnostic Clone regression passes. The same non-mutating diagnostic is available manually in API Tester as System → Consent Management Integration Status.

License-aware preflight — no license mutation

Regression testing must never replace, rotate, revoke, intentionally conflict, or otherwise mutate the installed License Key. Before a run record is created, the harness performs a read-only entitlement capacity preflight against the currently installed signed license.

A normal regression run requires capacity for 2 temporary organizations (Org A plus the paired Org B). A Diagnostic Clone and a System Admin context tests only run each require 1 temporary organization. Full Use is treated as unlimited. On Limited Use, if the required temporary slots are not available, the run is not started and the page explains the required and available capacity.

Scheduled runs use the same rule. If a scheduled run is due but lacks temporary organization capacity, Blackcap does not partially start the run; it advances the schedule and writes a safe audit event describing the capacity skip. This protects the environment from licensing-related halfway failures while leaving the installed credential untouched.

Design Summary

The harness is intentionally modular and keeps the main Flask app small:

  • inky_admin/blueprints/regression_tests.py owns the System Admin page and actions.
  • inky_admin/services/regression_harness_service.py creates run records and page context.
  • inky_admin/services/regression_runner_service.py runs the background test lifecycle.
  • inky_admin/services/regression_http_client_service.py wraps a real Flask HTTP session/client and records timings.
  • inky_admin/services/regression_cleanup_service.py performs cleanup through normal user account deletion behavior.
  • inky_admin/services/regression_schedule_service.py stores quiet-hour recurring schedules and queues due runs.
  • inky_admin/services/regression_report_service.py stores sanitized HTML/JSON/JSONL-style report payloads in the database.
  • inky_admin/services/regression_test_org_service.py marks/list test organizations separately from normal organizations.

The harness uses the existing app DB transaction/service patterns, the normal organization service, invite service, API-token service, audit service, background job service, and account lifecycle service.

Test Lifecycle

A manual or scheduled run follows these phases:

  1. System Admin starts or schedules a run from /admin/regression-tests.
  2. A background job is created with created_via=regression_harness.
  3. A unique organization name is generated, such as Test-Org-20260623-083015-a7f3.
  4. The organization is created through the normal organization creation service.
  5. The generated organization is marked in regression_test_organizations so it is hidden from normal organization lists.
  6. An Org Admin invite is created through the normal invite service.
  7. The invite token returned by the service is accepted without requiring a real mailbox.
  8. A generated password is set for the Org Admin test user.
  9. When System Admin organization-context tests are enabled, the harness creates an isolated HTTP session authenticated as the real System Admin identity that initiated the run, switches that session into Regression Org A through the production global organization-switch route, and verifies context persistence plus effective Org Admin isolation before explicitly returning it to Default. The operator's browser session is never changed by the background worker.
  10. Unless System Admin context tests only is selected, the harness logs in as the generated Org Admin through a separate real HTTP session and records request durations/results.
  11. A test API token is created through the normal API-token service.
  12. Extension API-token checks run with Authorization: Bearer <test token>.
  13. If configured, one secondary user is invited, accepted, logged in, and permission-denial checks are recorded.
  14. Unsupported first-pass areas are not attempted by the first-pass harness.
  15. Full cleanup mode deletes the secondary user first, then the Org Admin user, through the normal account lifecycle service.
  16. Leave-open mode keeps the org/users active and shows generated credentials on the regression page.
  17. Sanitized HTML/JSON/JSONL-style/performance report payloads are stored on the regression run record in the database.

Regression environment deletion and purge lifecycle

Normal paired Regression runs create two temporary organizations. A is the primary environment used by the main test session and B is the paired environment used by cross-organization/import-export coverage; the suffix does not imply that either organization is permanent. Diagnostic Clone is the separate one-organization workflow.

Automatic cleanup at the end of a run and the historical Delete Environment(s) action call the same cleanup service. Cleanup discovers all disposable active users that remain in each verified Regression organization instead of assuming a fixed list of generated accounts. It then moves every associated Regression organization through the normal organization-delete service. Successful cleanup therefore leaves both A and B in the normal Deleted tombstone state with their existing retention/purge deadline. Repeating cleanup is safe and does not reactivate an environment or restart its retention clock.

Physical purge happens later through the centralized organization purge service after the normal retention deadline. Active organizations are never made purgeable merely because they belong to a completed Regression run. Regression run history is retained independently according to its own retention rules so historical results can remain after their temporary organizations have been purged.

Regression environment lifecycle is deliberately not tested by a Regression suite item that creates and cleans its own Regression environment. Cleanup/purge behavior is covered by conventional service/database tests so the harness does not validate its own lifecycle circularly.

New Admin Page

Route: /admin/regression-tests

Access rules:

  • System Admins with system.regression_tests.view can view the page.
  • System Admins with system.regression_tests.manage can start runs, save schedules, switch into test org context, and trigger cleanup.
  • Org Admins, normal users, and unauthenticated users cannot access the page.

The page includes:

  • Start New Run
  • Schedule Runs
  • Active Runs
  • Test Performance
  • Test Organizations
  • Historical Runs
  • Run Details, including generated credentials while applicable
  • Collapsible panels so long regression history, performance, organization, and schedule sections can be opened only when needed
  • A generated-user reference table with copy buttons and hidden passwords by default
  • Step-level pass/fail results with expanded failure request/response detail
  • User-friendly labels for modes/statuses and Blackcap-formatted timestamps with seconds
  • Sticky table headers, pinned action columns, and contained horizontal scrollbars for long run/history/performance tables
  • Cleanup actions from the Test Organizations list only

Run result counts: Active Runs, Historical Runs, Run Details and CSV exports count saved regression_test_steps by result status, not by the highest Step number. This includes formal steps and manual/diagnostic activity substeps displayed as 426.01, 426.02, etc. Such substeps have a persisted sequence=0 so they never affect regression resume ordering but do contribute to Passed, Failed, Warnings and Skipped totals. The run header already stores these four counters; they are updated atomically as steps arrive and reconciled at finalization. The dashboard also derives authoritative counts in a single batched indexed query to display older, previously miscounted runs correctly without a migration or rewriting history. Test Organizations shows only results attributed to that environment, and therefore may legitimately differ from whole-run totals; the row tooltip identifies legacy runs that lack organization attribution.

Scheduled Runs

Scheduled runs are stored in regression_test_schedules. Saved test-data defaults are stored in regression_test_defaults and are merged into scheduled runs so recurring quiet-hour runs use the latest saved fake recipe/API input values.

Supported fields:

  • enabled / disabled
  • daily or weekly
  • weekly day of week
  • preferred time of day
  • time zone selected from the common Blackcap time-zone list
  • schedule-specific run mode
  • schedule-specific secondary user creation/tests/role
  • schedule-specific keep-org-on-failure behavior
  • schedule-specific optional backup restore test
  • schedule-specific Code & Schema Verification option, enabled by default, which runs and waits for the System page Direct SQL Audit and read-only Database Parity/Schema Validation jobs
  • schedule-specific System Admin organization-context tests option, enabled by default, plus System Admin context tests only when the scheduled run should create only Regression Org A and skip the generated-Org-Admin/paired-org suite
  • report retention count
  • next and last run timestamps

Production scheduling should call run_regression_tasks.py from a cron/systemd timer using the same pattern as run_daily_purge.py; the Admin page does not need a manual due-schedule trigger when that background path is active. Use the Start New Run panel to save shared recipe/API test-data defaults; schedules intentionally keep their own run mode and run-option checkboxes while referencing the shared fake recipe/API input values.

Diagnostic Clone Runs

Diagnostic Clone extends the regression harness for organization-specific troubleshooting. A System Admin starts it from the normal Organizations list or organization details page, and the resulting run is tracked on the Regression Tests page with the rest of the harness history and generated credentials.

Diagnostic Clone creates exactly one regression/test organization and always leaves it open for inspection. The source organization is associated to the run for attribution, but the run history remains platform/Default-owned. Regression/test organizations are not valid diagnostic clone sources; the Default organization can be used as a source if needed. Only one active diagnostic clone is allowed per source organization; while one is active, the organization actions open that clone/run instead of creating another.

The first implementation is configuration-focused:

  • Copies safe organization-scoped configuration, including timezone/locale fields, scoped settings, meal planner settings and slots, Noun Project footer/rule configuration, grocery departments/category mappings, kitchen inventory storage locations, organization AI Seed entitlement/billing configuration, and other non-secret org preferences available through the service layer. A finite diagnostic organization starts with the full current monthly allotment unless current billing-period AI usage is explicitly copied.

  • Clones display rows for diagnostic use, but does not copy display clients, receiver tokens, kiosk tokens, physical hardware write targets, or other display secrets.

  • Creates a safe mock display when no source displays are available.

  • Creates generated regression users through the same invite/acceptance flow as normal regression runs.

  • Runs a single-organization diagnostic regression profile when selected. This profile skips A/B import-export, cross-organization sharing, and backup-restore tests, but includes the deterministic mocked AI recipe-image generation/save workflow, Org Admin AI Usage scope and provider-diagnostic privacy checks, the dedicated AI Usage page, and the read-only organization and global personal AI Seed summary endpoints. The mocked image test spends from the generated user's personal bucket so it does not alter the copied organization balance. If the regression option is not selected, the run only creates the diagnostic org, accepts the generated Org Admin invite, copies configuration/displays, records the setup steps, and leaves the org open.

  • Adapt Recipe regression (real or synthetic): Both full System Admin and diagnostic-clone regression runs exercise the same Adapt Recipe source fixtures and normal durable-job/Seed/AI Usage workflow, in the run's selected AI-provider mode. Synthetic mode uses the run-scoped deterministic adapter without contacting a paid provider. Real mode selects the lowest-priced eligible configured provider and submits the fixture through it; unavailable real-provider configuration produces Skipped, while a submitted provider failure produces Failed. Both modes verify saving, cache preservation, Held behavior and purge protection. The test uses its isolated organization, never spends the source organization's Seeds, and reuses an existing job on resume. Configuration-only clones do not submit AI requests.

  • Restores temporary configuration changes made by diagnostic/config checks, such as the recipe unit preference, before the diagnostic run is finalized.

  • Adds a collapsed Current Operational Data copy panel on every diagnostic clone admin page directly under the regression context banner. This panel is for source org current-state snapshots that may change over time.

  • Adds page-specific targeted copy panels for diagnostic clones: Copy Recipes from Source Organization on Recipe Library, Copy Shopping Lists from Source Organization on Shopping Lists, and Copy Meal Plans from Source Organization on Meal Planner. Recipe copy supports source recipe search by title or recipe ID. Shopping-list copy brings the selected list, the current recipe cart, and any related recipes. Meal-plan copy shows only weeks within 8 weeks before/after the source org's current planner week that actually contain data. Already-copied source recipes, shopping lists, and meal-plan weeks show a green check indicator instead of a selectable checkbox. Copied recipes include metadata, editable/converted content, ingredients, tags, source-facing paths, ready recipe cache artifacts, thumbnails/images, related recipe cache metadata, and recipe-scoped AI image status. The latest/success/failure AI jobs are remapped to the generated diagnostic Org Admin, and candidate image files are copied when present, so a source image failure is visible and can be retried under normal retry rules. This recipe AI status copy is independent of the optional AI Usage copy. Cache profile metadata is remapped to the diagnostic clone's regenerated display IDs so the Recipe Cache Status panel mirrors the source state instead of falsely reporting that the default display render profile changed.

  • Adds an optional Current operational data copy mode at clone creation and as a later manual action from the diagnostic clone. This syncs a minimal issue-reproduction snapshot: current display assignments, display content settings, selected/displayed recipe references, current display preview assets, the source organization's current planner week and associated recipes, meal-plan entries and recipes referenced by displays showing day/week/month meal plans, active Let’s Cook sessions and their recipe/progress/timer data, only the single current active standard shopping list with related recipe data, the local household list, the current recipe cart, and kitchen inventory items including both on-hand and not-on-hand records. External provider sessions/mappings remain excluded. Current-state recipe/display artifacts are copied into diagnostic storage when they are part of the active operational slice, so there is no user-facing cache rebuild option for diagnostic data copy. Operational data can be copied multiple times as source state changes; later copies sync the diagnostic slice instead of duplicating meal-plan slots, shopping lists, household list data, Let’s Cook sessions, or inventory items. Copied diagnostic data keeps the source organization’s user-facing names/titles, including shopping list names, recipe titles, display names, meal-plan labels, Let’s Cook labels, and inventory item names, while only internal IDs/UUIDs are regenerated for isolation. The regression context banner shows an Operational Data Copied - N pill after the first copy.

  • Adds an independent optional Current billing-period AI usage and organization balance selection at clone creation and in the later Current Operational Data panel. It copies only usage events whose timestamps fall within the source organization's active AI billing period, their diagnostic-safe job context, the current monthly bucket's initial/spent/reserved amounts, and an aggregate of active additional available/reserved organization AI Seeds. Older periods, reservations, ledger history, and unrelated AI jobs are not copied. Re-running the copy replaces the prior imported current-period snapshot. If this selection is not used, the clone retains the full monthly allotment initialized during configuration copy.

  • Diagnostic regression tests can be run again, or for the first time, after the clone exists from the Run Details panel. These tests append new steps to the same diagnostic clone run instead of creating a second run, so step numbering continues from the existing run history. Diagnostic clone total time is shown as active step duration so idle time between later diagnostic test passes is excluded. The action is hidden after the diagnostic clone has been cleaned up.

Diagnostic Clone does not copy users, source invites, API tokens, Chrome Extension tokens, OAuth tokens, Dropbox/Google Drive credentials, SMTP secrets, external shopping provider sessions, browser profiles, backups, restore staging, support requests, audit history, unrelated job history, older AI usage periods, or performance history. Configuration-only clone creation does not copy operational cache/display artifacts; selected recipe copy and current operational-data sync copy only the artifacts and recipe-linked AI job state needed for the copied diagnostic slice into diagnostic-org storage. Provider/display shell configuration may be copied only in a disconnected diagnostic-safe form.

Diagnostic clone organizations appear in the Regression Tests Test Organizations section with source organization metadata. They remain hidden from the normal Organizations list like other regression/test organizations and are cleaned up with the existing regression cleanup path. When any associated test organization still exists, the Run Details header shows a View Organization button for a single-org run or distinct View Organization A/B/C buttons for paired-org runs.

The Regression Tests page is platform-only. It is visible to System Admins in the Default organization context, but not to Org Admins and not while a System Admin is viewing another organization as an org-admin support context. The diagnostic clone banner's View Clone Run action first returns to Default context, then opens the run detail.

Suite 2026.06.25.10 fixes the page-specific Shopping Lists and Meal Planner copy panels so they resolve the active organization through the same organization-context service used by the rest of Admin. It also refreshes stale browser-session recipe-cart state from the DB when diagnostic copy actions update the cart in app_state, so copied recipe cart data appears immediately in the diagnostic clone.

Suite 2026.06.26.1 adds Household List pin/unpin regression coverage. The shopping-list section now creates a Household List item, pins it, verifies session-only completion leaves the persistent Household item unchecked, unpins it, and confirms anchor/session metadata is cleared.

Suite 2026.08.29.1 adds regression coverage for the usability/consistency work around recipe time metadata and Recipe Library shopping. The Recipes group now creates a deterministic web-backed recipe with Prep 20 + Cook 40 and no explicit Total Time, reloads the canonical recipe record to verify total_minutes=60 was actually persisted, then performs a name-only edit and verifies the stored component times and Total Time remain unchanged. The Shopping Lists group now sets a Recipe Cart multiplier before creation, verifies successful Recipe Cart → Shopping List conversion persists that multiplier and clears the persisted cart, checks that the response exposes the Shop This List and live multiplier URLs used by the Recipe Library confirmation, then changes the saved Shopping List recipe multiplier and verifies both the persisted multiplier and regenerated ingredient payload update immediately without a page refresh.

The same suite keeps its broader API Tester and Component Recipe checks aligned with those production rules. Package-managed update endpoints are required in /api-tester/spec only when BLACKCAP_PACKAGED_RUNTIME identifies a package-managed installation; source/unmanaged runtimes are expected to hide those endpoints just as API Tester does. Component Recipe Shopping List scenarios also repopulate the Recipe Cart between independent Shopping List creations because every successful Recipe Cart → Shopping List conversion now intentionally empties the cart.

Suite 2026.08.26.2 adds an Organizations-page diagnostic-clone visibility regression. While a diagnostic clone is active, the normal source organization row is rendered with a subtle lavender full-row highlight and left accent without adding a badge, cell, or extra line, so row height is unchanged. The diagnostic-clone profile opens the real System Admin Organizations page and verifies that the source row carries the active-clone highlight. This suite revision also aligns older regression contract tests with the current executable suite version so suite-version assertions do not fail after the August 26 regression additions.

Suite 2026.08.26.1 adds manual/OCR content-fingerprint duplicate coverage to both the full regression run and the diagnostic-clone profile. The diagnostic check verifies that the real manual recipe created in the clone is rediscovered by its original ingredients/instructions, that deliberately unrelated content remains distinct, and that production matcher call sites remain limited to manual creation and recipe-photo/OCR finalization. The shared Shopping Lists regression also verifies that the edit page uses the contextual Shop This List action and no longer renders Shop a List.

Suite 2026.08.09.19 uses bounded active-state-aware recipe-cache waiting. The normal 120-second regression timeout remains the base limit, but a durable cache job that is still running may continue up to a 360-second hard cap when either its heartbeat is fresh or its recorded started_at is still inside that bounded active window. The second signal is important on Raspberry Pi + SQLite because background-job heartbeat writes are intentionally best-effort and may defer during CPU/I/O-heavy editable rendering. The production editable renderer already has a 300-second subprocess limit, so the regression waiter remains long enough to observe either successful completion or the renderer's real terminal timeout. Queued jobs, terminal failures, and running rows whose heartbeat and start time are both stale are not granted the extension. Regression results record both whether the extension was used and which liveness signal caused it.

Suite 2026.07.30.9 fixes diagnostic-clone AI reporting and balance fidelity. When a System Admin is actively viewing a live diagnostic clone, the AI Usage page includes that clone as the current authorized organization and defaults to it instead of falling back to All Organizations. Current-period usage copying remains explicitly scoped to the source organization. A copied current-period monthly bucket is also treated as authoritative when its boundaries reflect a transitional source billing period, preventing normal allotment maintenance from creating a second full bucket. Refreshing the current-period snapshot removes only empty overlapping diagnostic buckets created by older behavior and refuses to remove any overlapping bucket that has spending or reservations. Suite 2026.07.30.10 aligns the diagnostic-clone simulated Chrome Extension recipe with the normal regression harness. Diagnostic runs now select Blackcap’s deterministic hosted quick-add fixture through the shared _extension_recipe_test_url() path and no longer contain a diagnostic-only example.test fallback. This keeps the cache build fully offline and prevents DNS-dependent failures while preserving the normal web-backed recipe extraction path.

Suite 2026.07.30.7 keeps the current-period AI usage/balance snapshot, recipe-scoped AI image failure/status copy, and diagnostic-safe AI regression coverage introduced in 2026.07.30.6. On diagnostic clone creation, the current-period AI usage option is now grouped under Current operational data so all optional data-copy selections share one hierarchy. Recipe ingredient copying also normalizes legacy nullable notes/preparation_notes values before insert, preventing older Pi data from failing clone creation against current non-null recipe ingredient columns. The PostgreSQL dependency-safe AI backup restore order introduced in 2026.07.30.5 remains in place; full regression backup restore remains outside the diagnostic single-org profile.

Voice Assistant regression coverage

Suite 2026.07.01.1 adds basic voice-provider regression coverage without creating real Alexa or Google Home OAuth tokens. The harness intentionally does not call live Smart Home discovery/fulfillment routes because those require provider-issued bearer tokens. Instead, it verifies the shared provider-neutral command layer used by ask Blackcap, Alexa Smart Home, and Google Home after provider validation.

Implemented provider-neutral smoke tests

The run now includes a Voice Assistant group that:

  • loads /api/voice/alexa/status;
  • loads /api/voice/google-home/status;
  • verifies /api/voice/test-command rejects direct calls without the API Tester header;
  • starts a dedicated Let’s Cook session on the regression display;
  • runs NEXT_STEP and PREVIOUS_STEP through the API Tester-only provider-neutral command path;
  • starts a direct Let’s Cook timer and then runs PAUSE_TIMER, RESUME_TIMER, ADD_TIMER_MINUTES, SUBTRACT_TIMER_MINUTES, and CANCEL_TIMER;
  • verifies unsupported timer amounts such as minutes=2 are rejected;
  • ends the dedicated voice-test Let’s Cook session with END_LETS_COOK.

These checks use provider labels such as alexa and google_home only as internal adapter markers. They do not mint provider tokens, do not depend on Alexa Lambda, and do not depend on Google Home Cloud-to-cloud account linking.

Remaining manual / future provider-adapter coverage

Alexa Custom Skill

Use signed provider tests manually, but regression should at least verify the internal mapping artifacts:

  • account-linking OAuth status endpoint is configured;
  • previous-step and subtract-timer commands are present in the provider-neutral command set;
  • the selected/default display mapping is preserved for the ask Blackcap connection;
  • rich custom-skill speech remains available from shared command results.

Alexa Smart Home

When a fixture connection can be created without real Amazon tokens, regression should validate the Blackcap adapter directly:

  • discovery returns one microwave-style endpoint per enabled display, not one endpoint per action;
  • endpoints include cooking, time, hold/pause-resume, power, fallback mode, and endpoint-health capabilities;
  • ReportState reflects the current Let’s Cook on/off state for the resolved display;
  • native directives map correctly: turn on/start cooking, turn off/end cooking, hold/pause timer, resume/start-or-resume timer, adjust cook time, and fallback mode actions;
  • old SceneController endpoints are still tolerated only as legacy compatibility.

Google Home

Regression should validate the Cloud-to-cloud fulfillment adapter with synthetic OAuth-linked connection rows:

  • SYNC returns one microwave device per enabled display;
  • QUERY reports current cooking/start-stop/timer state even when the session was changed outside Google Home;
  • EXECUTE native traits map to the shared service: OnOff/StartStop/Cook, TransportControl, Timer, and Modes fallback;
  • timer play/resume resumes a paused timer and starts the current-step timer when nothing is paused;
  • DISCONNECT disables only the Google Home connection.

Regression should exercise the voice-link email service with SMTP disabled and with a mocked SMTP sender:

  • linking/relinking queues the correct template for Alexa Custom Skill, Alexa Smart Home, or Google Home;
  • recipients include the linked user plus active org admins;
  • SMTP disabled skips quietly;
  • the Connections page resend action reports success, skipped, and failure cases.

Regression Maintenance and Scheduling

Regression runs are still tracked in the database as durable rows. Manual runs may execute in the Admin app process for immediate convenience, but the durable maintenance path is now centralized in inky_admin/services/regression_worker_service.py and can also be run from run_regression_tasks.py.

The regression maintenance pass performs two jobs:

  1. Recover queued or interrupted regression runs.
  2. Start due scheduled regression runs.

It does not purge regression history. Historical run/report retention is handled by the normal data purge flow.

The Admin app schedules a one-shot startup recovery pass so a queued or interrupted regression run can resume after the app restarts. A file lock and short marker window prevent multiple Gunicorn workers from duplicating the recovery pass. Scheduled quiet-hour runs should be started by the timed run_regression_tasks.py process.

For recurring scheduled runs, use the same systemd/cron style already used by Blackcap scheduled purge jobs. A typical systemd timer or cron entry can invoke:

cd /home/pi/Blackcap-Pi && /home/pi/inky_env/bin/python3 run_regression_tasks.py

Running it every 15 minutes is usually enough for quiet-hour schedules. The script executes scheduled and recovered runs synchronously so the process does not exit before a regression run finishes.

Historical regression run/report retention is controlled by platform purge settings:

  • purge.regression_test_runs.enabled, default true
  • purge.regression_test_runs.after_days, default 30

This purge removes historical run rows, step rows, and DB-backed report/request/performance payloads only when the run is old enough and no associated test environment is still available. If any regression_test_organizations marker still joins to an organizations row, the run details are protected so the Test Organizations panel can continue to link back to the run that created or cloned that environment. Test organizations continue to be cleaned up through the regression cleanup action and the normal organization lifecycle purge.

Test Organization Model

Test organizations are not marked by changing the core organizations table. Instead, they are tracked in regression_test_organizations:

  • organization id/name
  • run id/public id
  • org admin user/email
  • secondary user/email
  • cleanup status
  • metadata JSON

Normal organization listing functions exclude rows that appear in this marker table by default. The regression page lists live generated organizations separately.

Regression run details are retained while at least one associated environment remains available. In practice, the regression history purge skips any old run that still has a regression_test_organizations row joined to a current organizations row, including environments that are cleaned up but not yet fully lifecycle-purged. Once the normal organization lifecycle purge removes the generated organization row, the historical run/details become eligible for the configured regression history retention window. The Test Organizations dashboard panel joins back to the live organizations row, so a fully purged organization disappears from that operational list.

Invite Acceptance Strategy

Setup uses the normal invite service and the token returned by create_invite() / organization creation. No real email inbox is required. The harness does not expose invite tokens in the normal UI.

Credential Handling

Generated passwords are stored only in transient run fields so they can be displayed for active or leave-open runs. The regression page exposes a generated-user reference table with copy buttons for generated Org Admin, secondary user, and paired-org admin emails/passwords when values are still available. Passwords are hidden by default in the UI. Reports redact credentials. Cleanup clears transient password fields after successful cleanup.

Never log or report raw values for:

  • generated passwords
  • API tokens
  • session cookies
  • CSRF tokens
  • SMTP secrets
  • OAuth secrets
  • display tokens

Secondary User / Role Testing

A run can create at most one secondary generated user. The Org Admin is created first, then the secondary user invite is created and accepted. Lower-role permission denials are recorded as passing assertions when Blackcap correctly blocks the action.

The first-pass secondary suite includes login and a known admin-only route denial. It also records an ownership-sensitive recipe edit/delete denial scaffold for future deeper route coverage.

Run Modes

Full Run with Cleanup

Runs setup/tests, then deletes generated users through the normal account lifecycle behavior. If the Org Admin is the last user in the generated org, the existing delete-last-user-in-organization path schedules the organization for purge.

Run and Leave Open

Runs setup/tests and leaves the generated organization active. Credentials remain visible on the regression page for manual inspection. Cleanup can be triggered later from the page.

Organization time-format coverage

The regression suite saves preferred_time_format=24h through the Org Admin Organization route, verifies the setting persists on the generated test organization, and then runs page-load smoke tests with that organization context. This gives practical coverage for org-scoped pages such as Backups, Configuration, Meal Planner, and Mobile views under a 24-hour organization preference. Meal Planner also updates a slot to canonical 15:00/16:30 values and restores the original window so the API contract remains protected while the UI uses native time controls.

Rapid Shop a List checkoff coverage

The regression suite includes a dedicated Shopping Lists check for the rapid checkoff API contract. It creates a manual shopping list with twelve rows, sends a multi-item checked-state update through /shopping-list/{list_id}/toggle, reloads the list from storage, and verifies all twelve rows stayed checked.

The same block also verifies mixed check/uncheck final intent and duplicate-label safety. Duplicate-label rows must be keyed by stable item id first, with section/index retained only as a legacy fallback. Checking one regression duplicate eggs row should not check the other row unless both item ids are included.

These tests primarily protect the backend/API side of the mobile single-flight queue. Browser-level behavior should still be spot-checked manually when changing inky_admin/static/mobile_shopping.js: rapidly check 12 items, wait for the saved status, refresh, and confirm all 12 remain checked.

Account page smoke coverage

The page-load smoke suite includes My Account, My API Tokens, and Password pages. These checks are intentionally simple GET requests that fail the run if shared account templates, role labels, access-location decoration, token rows, or password-page rendering introduce a 500.

Interactive Let’s Cook display and multiplier coverage

The Let’s Cook suite creates a kiosk display and a hosted web receiver, starts a Let’s Cook session on each display, reads display-specific state, checks off an ingredient and instruction, starts a 10-minute timer, pauses the timer, stops the timer, and ends the session with default-content restore requested. /api/kiosk/{display_id}/lets-cook/... is the canonical display-scoped transport. The API Reconciliation group exercises every canonical Kiosk action and separately verifies that all nine retired /api/displays/{display_id}/lets-cook/... URLs return HTTP 410 Gone with the corresponding Kiosk successor.

Suite 2026.08.21.1 also adds a persisted Let’s Cook multiplier quantity verification. After the normal 1× session establishes the recipe's displayed ingredient rows, the harness starts a second real session for the same regression recipe with a permitted non-1× multiplier, reloads that session from the API/database path, and compares every actionable ingredient row with ingredient_quantity_display_service.scale_ingredient_line(), the same shared quantity-scaling function used by normal recipe rendering. The step requires the stored multiplier to remain non-1×, requires at least one numeric quantity to actually change, and fails on any persisted quantity mismatch. This protects both direct Let’s Cook starts and the shared service used by meal-plan-driven session creation without relying only on source-string assertions.

Harness contract safeguards

The regression runner includes focused unit/contract coverage for the harness itself so simple harness mistakes fail during the normal application test suite instead of after a long GCP regression run. In particular:

  • every keyword used by a _http_step(...) call is checked against the helper signature, preventing a late-run TypeError from an unsupported diagnostic keyword;
  • _http_step(...) supports an optional custom actual-summary string while retaining its normal HTTP summary when no override is supplied;
  • the expired passive-session assertion validates the reauthentication URL semantically as /login?session_expired=1 rather than assuming the URL must literally end in /login; and
  • the exact 401 JSON shape returned by the session-expiration path has focused test coverage so a correct reauthentication response is not recorded as a regression failure.

These checks apply identically to source deployments and packaged deployments because they validate the shared Blackcap regression harness before either environment runs the full suite.

Test Coverage Plan

Coverage Matrix

The harness is intended to be deterministic and safe to run repeatedly against generated organizations. Manual and scheduled runs use the same suite runner; the scheduled configuration keeps its own run mode and run-option checkboxes while sharing the saved fake recipe/API input defaults.

Legend: ✅ covered by default, ⚙️ covered when the related option is enabled, ◐ partial/smoke coverage, — intentionally not attempted.

Area Manual Run Scheduled Run Backup Restore Option Secondary User Option Notes
Organization setup and invite acceptance ✅ ✅ ✅ ✅ Uses generated orgs and generated invites without real email delivery.
Initiating System Admin org-context persistence/isolation ✅ ✅ — — Enabled by default; can run alone. Uses the run-starting System Admin identity in an isolated regression HTTP session, switches through the production route, traverses tenant pages, verifies Org Admin-effective scope, denies platform-only surfaces, and returns to Default.
Admin, account, and mobile smoke routes ✅ ✅ — ◐ Lower-role routes may pass by being denied correctly. Account smoke coverage includes account, token, and password pages.
API token creation, use, and revocation ✅ ✅ — — Includes negative add-recipe assertion after revocation.
Simulated Chrome Extension add recipe ✅ ✅ — — Uses the API-token extension endpoint, not browser DOM extraction.
Manual recipe creation and cache readiness ✅ ✅ ✅ ✅ Dependent flows use a recipe with verified editable content.
Recipe Library select/edit/metadata/cart flows ✅ ✅ ✅ ◐ Secondary-user coverage focuses on safe role/permission behavior.
Paprika import/export and paired-org import ✅ ✅ — — Uses local deterministic fixtures.
Managed Blackcap recipe sharing ✅ ✅ — — Runs between generated paired organizations.
Shopping lists, recipe cart, and grouping views ✅ ✅ ✅ ◐ Shopping-list data is DB-backed; no filesystem shopping-list state is expected.
Household-list include/review behavior ✅ ✅ — — External provider sync is not attempted.
Meal planner create/move/push/list generation ✅ ✅ — ◐ Uses current-day/current-week anchors for stability.
Safe display assignment and mock display pushes ✅ ✅ — — Uses safe/mock-style targets rather than real e-ink writes.
Hosted receiver/kiosk-safe display route coverage ✅ ✅ — — Smoke/assignment coverage plus interactive Let’s Cook companion-route coverage.
Let’s Cook session controls ✅ ✅ — — Starts, updates, and ends normal, kiosk-display, and hosted-receiver sessions; verifies persisted non-1× ingredient scaling matches recipe rendering; includes timer start/pause/stop and display-session end/restore.
Kitchen inventory create/search/location flows ✅ ✅ — ◐ Secondary-user checks focus on allowed/blocked behavior.
Org-level settings/unit/time-format changes ✅ ✅ — — Settings are changed through service-backed routes. Generated orgs run smoke coverage under the 24-hour time-format setting.
Backup/restore mutation and usability checks ⚙️ ⚙️ ✅ — Creates sentinel data, mutates it after backup, restores, and validates usability/cache readiness.
Cleanup and lifecycle purge handoff ✅ ✅ — — Uses normal account-delete/delete-last-user behavior.
Regression reporting, timing, and redaction ✅ ✅ ✅ ✅ Run history is Default/platform-owned; generated org ids are attribution only.
Google Keep/Amazon external shopping-list sync — — — — Intentionally excluded until isolated fixtures/stubs exist.
Shop With provider website automation — — — — Intentionally excluded from the deterministic suite.
OAuth provider callbacks and real email delivery — — — — Not attempted by the harness.
Real e-ink display writes — — — — Avoided so scheduled runs are safe and non-disruptive.

Implemented Coverage Summary

Current implemented coverage includes:

  • create unique generated test organizations and hide them from normal organization lists
  • create, accept, and log in generated Org Admin users through normal invite/session flows
  • optionally create one generated secondary user and record allowed/blocked lower-role behavior
  • open key Admin and Mobile UI routes
  • create, use, and revoke an API token through normal services
  • simulate Chrome Extension add-recipe payloads through the API-token endpoint
  • wait for recipe cache readiness through the normal cache-status endpoint
  • verify editable recipe content readiness when cache succeeds
  • create manual editable recipes with deterministic cache behavior
  • exercise Recipe Library selection, edit, metadata, cart, import/export, share, and display-related flows
  • exercise shopping list creation, completion/reopen behavior, manual items, cart-derived lists, grouping views, household-list behavior, and inventory hydration
  • exercise organization time-format updates plus meal planner settings/events, canonical slot time updates, recipe/non-recipe entries, day/week/month display pushes, and meal-plan shopping-list generation
  • exercise Kitchen Inventory, Let’s Cook, safe display assignment, hosted/kiosk-safe display routes, local backup jobs, and selected purge behavior
  • optionally run backup/restore sentinel mutation and post-restore usability/cache-readiness validation
  • run normal cleanup through the account lifecycle path and preserve Default-owned regression history
  • store sanitized run reports, request logs, timing data, and step details in the database

Coverage Boundaries

The harness intentionally excludes tests that depend on real third-party sites, real mail delivery, OAuth callback ownership, or physical e-ink display writes. Those areas should remain outside the deterministic default suite unless they get isolated fixtures, stubs, or an explicitly separate external-integration test mode.

Recipe Cache and Editable Content Handling

The extension add-recipe test remains an API-token/Chrome Extension endpoint test. Because it creates a web-backed recipe whose cache build depends on a source URL, the default harness path now uses Blackcap's own temporary fixture route at /regression-fixtures/quick-add-recipe/<run_id> when [auth] external_base_url is configured. That route serves a tiny deterministic Recipe schema page with ingredients and instructions, so default regression runs do not depend on Delish, Food Network, or another third-party site that may be slow, blocked, redesigned, or learned into the slow-site list. The fixture route intentionally bypasses normal browser-login auth so the normal recipe importer can fetch it like an external recipe page, but the route-level fixture guard only returns content while the referenced regression run is actively running. Once the run finishes, its status closes the fixture. If a run is interrupted and never reaches a final status, the route stops serving the fixture two hours after the run start time. Unauthenticated access outside that active window returns 404.

The manual run form still allows a custom Extension recipe URL for deliberate external-site testing. A real configured URL is now honored exactly. Leaving that field blank is an explicit request to use the Blackcap-hosted fixture. Placeholder/documentation hosts such as example.com, example.net, example.org, *.test, and *.invalid are never used as recipe-fetch sources; the harness replaces them with the deterministic fixture instead. A custom URL known to slow-site policy is likewise replaced by the fixture and that choice is recorded in the step expectation. The API Reconciliation group reuses this same selected source policy for Extension structured capture and Recipe Discovery Save. When the fixture is selected, those adapters keep the exact authorized regression-run fixture path and add a harmless blackcap_regression_variant query marker (structured-capture or discovery-save) so their source URLs remain distinguishable without inventing fake public URLs or changing the fixture run id. When a real custom URL is selected, the regression harness preserves that configured URL. No fixture-specific database rows are created, so there is nothing separate to purge; historical regression run rows remain governed by the normal regression-history purge policy. If no external base URL is configured, the harness can use the local regression fixture base (BLACKCAP_REGRESSION_FIXTURE_BASE_URL, BLACKCAP_INTERNAL_BASE_URL, or the normal local Gunicorn fallback) rather than substituting a bogus recipe URL.

The harness waits on /api/recipes/<recipe_id>/cache-status and records that wait as a timed regression step. When the extension recipe cache reaches ready status, the harness verifies editable recipe content through the normal prepare-edit endpoint.

The harness also creates a manual editable recipe and waits for its cache build. Manual recipes are deterministic because editable content is written before render/cache work starts. Shopping-list and meal-plan tests use the extension recipe when its editable content is verified; otherwise they use the manual editable recipe and record a warning explaining the fallback. This keeps dependent shopping/meal-plan coverage meaningful while still surfacing extension recipe cache failures separately.

External Integration Limitations

The first pass does not attempt these areas and does not create step records for them:

  • Google Keep / Amazon external shopping-list sync
  • Shop With provider website automation
  • Chrome Extension DOM extraction
  • OAuth provider callbacks
  • real email delivery
  • support request creation/email delivery
  • real e-ink display writes

Cleanup Strategy

Cleanup is intentionally not a special purge path. It calls the same account lifecycle service used by normal user self-delete behavior:

  1. Delete secondary generated user first, if present.
  2. Delete generated Org Admin last.
  3. Let the existing delete-last-user-in-organization behavior schedule the org for purge.
  4. Verify organization status and purge status.
  5. Clear transient generated passwords after cleanup.

Cleanup refuses to run unless the organization is marked as a regression test organization.

Performance and Reporting

Each HTTP/API step records:

  • method and path
  • acting user type
  • auth mode
  • duration
  • response size
  • status
  • sanitized request/response summaries

Reports are stored in regression_test_runs columns:

  • report_html
  • report_json
  • request_log_jsonl
  • performance_json

The stored report payloads redact credentials and tokens. The regression page renders details from regression_test_runs and regression_test_steps, so normal review no longer depends on files under regression_runs/.

Test Performance View

The Regression Tests page includes a Test Performance panel for comparing automated regression step timing across historical runs. It is a read-only analysis view derived from existing regression_test_runs and regression_test_steps rows; it does not create a separate performance-summary table.

The view is intended to answer which repeatable tests are getting slower, which tests are most variable, and whether total run time changed because of one slow step or broad suite drift. It excludes manual diagnostic/test activity and diagnostic copy activity by default because those records are useful run documentation but are not repeatable automated test cases.

Performance rows are matched by stable test identity rather than visible step number. The identity is based on the repeatable test metadata, such as group/phase, request path, and test name/description. This keeps comparisons meaningful when new tests are inserted, diagnostic activity rows are recorded without formal step numbers, or the visible numbering changes between suite versions.

The panel includes:

  • Summary cards for latest total run time, average latest step time, slowest latest test, and biggest increase.
  • Filters for search, group/phase, status, run type, and the maximum number of tracked tests returned.
  • A Max tests selector so installations with more than the default query limit can review 150, 200, 300, or 500 tracked automated tests without changing platform-wide query settings.
  • A table showing latest duration, previous duration, change, average, worst duration, run count, status, status basis, and a small sparkline.
  • A sticky trend chart and sticky column headers while scrolling the performance table.
  • A selected-test pill state; the first visible test is selected automatically when the panel loads and its trend is shown immediately.
  • A no-data state that clears the graph when filters, such as a group with no matching automated tests, produce no rows.

Status is based on the performance comparison basis shown in the table, not only the raw seconds shown in the change column. The basis text explains whether a row was marked stable, slower, faster, or variable because of percent change, absolute duration change, or variability.

Job Status / Performance Isolation

Normal Performance and Job Status views exclude organizations marked in regression_test_organizations by default. Regression request metrics are labeled with source=regression_harness metadata and the generated test organization id. The regression details page remains the place to inspect harness results.

Audit Events

The harness writes audit activity with source regression_harness, including:

  • run started
  • test organization created
  • invite accepted
  • API token created
  • cleanup started/completed/failed
  • run completed/failed

Secrets are never included in audit metadata.

Database Page Integration

The platform Database page includes these system-only resources:

  • Regression Test Runs
  • Regression Test Steps
  • Regression Test Schedules
  • Regression Test Defaults
  • Regression Test Organizations

Generated password fields are masked in the Database page.

regression_test_organizations is a lifecycle marker, not permanent audit storage. While the generated organization still exists, its marker is retained even when run history is old so cleanup/inspection remains traceable. After the real test organization is gone, the marker follows the Regression Test Run retention window: run-linked markers are removed when that historical run is purged, and older already-detached orphan markers are pruned on the same scheduled pass.

Dashboard Summary

The Regression Tests page shows a tabular summary strip between the page description and the run controls. It uses proper-cased labels and compact values so the page can be scanned quickly before starting or resuming work. The summary includes:

  • Active Runs: currently queued or running regression runs.
  • Open Envs: generated test environments still present, with a Fully Open / Cleaned Up breakout.
  • Historical Runs: retained completed runs, with Passed / With Failures breakout.
  • Last Run: the most recent non-diagnostic regression run, including duration, local date/time, compact source label such as Manual or Sched, and run id.
  • Active Diagnostic Clones: diagnostic clone organizations still available for investigation.
  • Enabled Schedule: a check mark when the single scheduled regression run is enabled and an X when it is disabled. This is intentionally boolean rather than a count because Blackcap supports only one scheduled regression configuration.

Summary cells are navigation links. Active Runs jumps to the Active Runs panel, Open Envs and Active Diagnostic Clones jump to Test Organizations, Historical Runs jumps to and expands Historical Runs, Last Run opens the selected Run Details panel, and Enabled Schedule jumps to Scheduled Runs.

Floating Panel Navigation

When a Run Details panel is selected, a compact arrow-only top button appears while the panel is visible but its header has scrolled off-screen. Pressing it returns to the top of the selected Run Details panel. The button uses the tooltip text: Return to the top of the Run Details panel.

The Test Performance panel uses the same compact floating arrow behavior. Its button appears only while the Test Performance panel is visible, the panel heading has scrolled off-screen, and the bottom of the panel has not yet moved above the bottom of the viewport. Pressing it returns to the top of the Test Performance panel. This keeps the helper visible while the user is deep in the panel but removes it once the panel has effectively been scrolled past.

Migration Steps

The regression harness schema migrations are:

database/migrations/161_regression_testing_harness.sql
database/migrations/162_regression_test_defaults.sql
database/migrations/163_regression_reports_db_storage.sql
database/migrations/164_regression_history_purge_and_worker.sql
database/migrations/165_regression_suite_version.sql
database/migrations/166_regression_step_org_attribution_and_emoji_save_validation.sql
database/migrations/167_regression_schedule_backup_restore_option.sql
...
database/migrations/202_regression_code_schema_verification_option.py

Migration 202 adds the schedule-level Code & Schema Verification flag idempotently for SQLite and PostgreSQL. On restart, Blackcap’s normal DB initialization/migration flow creates or updates the regression tables.

Restart Commands

On the Pi:

sudo systemctl restart inky_admin.service
sudo journalctl -u inky_admin.service -n 80 --no-pager

For manual local testing:

python -m py_compile \
  inky_admin/blueprints/regression_tests.py \
  inky_admin/services/regression_harness_service.py \
  inky_admin/services/regression_runner_service.py \
  inky_admin/services/regression_http_client_service.py \
  inky_admin/services/regression_cleanup_service.py \
  inky_admin/services/regression_report_service.py \
  inky_admin/services/regression_schedule_service.py \
  inky_admin/services/regression_test_org_service.py

Testing Checklist

Page Access

  • System Admin can access /admin/regression-tests.
  • Org Admin cannot access it.
  • Unauthenticated user redirects to login.
  • Page shows active runs, history, schedules, and test organizations.

Scheduling

  • Schedule can be enabled/disabled.
  • Next run time is shown.
  • Due schedules can be queued.
  • Scheduled runs appear in Active Runs.

Run Setup

  • Manual run queues a background job.
  • Generated test org name is unique.
  • Test org is hidden from normal Organizations list.
  • Test org appears in the Regression Test Organizations section.
  • Org Admin invite is created and accepted.
  • Org Admin generated credentials are visible only on the regression page while active/leave-open.
  • Optional secondary user invite is created and accepted.

Cleanup and leave-open run modes

  • Full cleanup mode runs tests then triggers normal cleanup.
  • Leave-open mode leaves test org active.
  • Manual cleanup can be triggered later.

Role Tests

  • Org Admin login succeeds.
  • Secondary lower-role login succeeds when enabled.
  • Unauthorized lower-role access is recorded as an expected permission-denial pass.

Functional Tests

  • Admin home opens.
  • My Account, My API Tokens, and Password pages open without a 500.
  • Recipe Library opens.
  • Meal Planner opens.
  • Shopping Lists opens.
  • Job Status opens.
  • API-token ping works.
  • Simulated Chrome Extension recipe payload posts.
  • Extension recipe cache wait records cache build duration and readiness.
  • Manual editable recipe creation succeeds.
  • Manual recipe cache wait records cache build duration and readiness.
  • Shopping-list and meal-plan dependent tests use a recipe whose editable content has been verified.

Unsupported Tests Not Attempted

  • External shopping list providers are not attempted in the first pass.
  • Shop With provider websites are not attempted.
  • OAuth callbacks are not attempted.
  • Real e-ink writes are not attempted.
  • Support requests are not attempted.
  • Real email delivery is not attempted.

Reporting

  • Step results are visible.
  • Counts update.
  • HTML-style report content is stored in the database.
  • JSON report content is stored in the database.
  • Request JSONL-style content is stored in the database.
  • Performance summary is written.
  • Test Performance shows historical automated-step timing trends.
  • Test Performance filters, row limit, selected-test chart, and no-data state work.
  • Secrets are redacted.

Isolation

  • Regression org data is excluded from normal Organizations list.
  • Regression org jobs are excluded from normal Job Status by default.
  • Regression org metrics are excluded from normal Performance by default.

Cleanup

  • Secondary user is deleted first.
  • Org Admin is deleted last.
  • Existing delete-last-user behavior schedules org purge.
  • Cleanup is retryable/idempotent enough for already-inactive generated users.
  • Cleanup refuses non-test organizations.

Regression

  • Admin UI still loads.
  • Mobile UI still loads.
  • API Tester still loads.
  • Chrome Extension endpoints still work.
  • Existing purge behavior is unchanged.

Regression task process versus purge

run_regression_tasks.py should not purge regression history. It only looks for due scheduled runs and queued/interrupted runs, then executes them in that process. Regression historical run/report retention is controlled by the existing Data Purge page/settings and defaults to 30 days.

The Admin UI can still queue/manual-start runs for immediate feedback, but the task script is the durable scheduled/recovery worker. If the app stops during a run, the next startup recovery or timed task pass can mark the run queued and continue it.

CSRF-sensitive session JSON requests use the same X-CSRF-Token header used by Blackcap browser JavaScript.

Error Export

Run details provide a Copy errors action whenever failed steps are present. The action copies failed steps as tab-separated text so the output can be pasted directly into a spreadsheet, document, or support/debugging conversation. The export includes run ID, step number, timestamp, group, test name, acting user, status, duration, request method/path, expected result, actual/error summary, failure details, sanitized request, and sanitized response.

Human-facing byte counts in regression summaries are formatted with thousands separators for readability.

2026-06-24 Regression Suite Expansion

The regression harness now records a suite_version and suite_label on each run. This keeps run duration comparisons meaningful as coverage grows; a short early harness run should not be compared directly with a later suite that exercises more screens and flows.

The run history shows total run time, and Run Details displays a larger total-time pill at the top of the detail panel.

Additional coverage added in this pass:

  • Shopping list item completion, unchecking, and re-completion.
  • Manual item completion on a recipe-derived shopping list.
  • Recipe-derived shopping list view validation.
  • Let’s Cook context retrieval.
  • Start a local Let’s Cook session using the verified dependent recipe.
  • Retrieve Let’s Cook session state.
  • Complete one Let’s Cook step.
  • Mark one Let’s Cook ingredient used when an ingredient is available.
  • Create and exercise a kiosk-display Let’s Cook session with step/ingredient checkoff, timer start/pause/stop, and end/restore.
  • Create and exercise a hosted-web-receiver Let’s Cook session with step/ingredient checkoff, timer start/pause/stop, and end/restore.
  • List Let’s Cook history.
  • End the Let’s Cook session.
  • Open Let’s Cook history detail for the completed session.

Manual cleanup of a leave-open regression organization now changes the run out of Left Open status. Once cleanup completes, the run is marked as Passed when there were no failures, otherwise Completed with Failures, matching the status it would have had if cleanup happened as part of the original run.

2026-06-24.3 regression harness expansion

Suite 2026.06.24.3 expands the default harness beyond the initial 49-step suite. The historical run table now keeps Total Time and Counts near the left side of the table, immediately after Status, so run duration and pass/fail count can be compared without horizontal scrolling.

The Started timestamp is the execution start time, not the moment the System Admin presses Start. The run row is created in a queued state first; started_at is set when the runner actually begins executing. Queue timing can still be inspected from the created/updated timestamps in the platform Database page if needed.

The page intentionally does not auto-refresh. Each major section has a lightweight Refresh link that reloads the page back to the same section anchor so the System Admin can update the data without losing their place or adding background polling load.

Additional default tests include:

  • broader Admin/Mobile page-load smoke tests that treat permission denial/redirects as acceptable as long as the route does not 500
  • recipe selection after cache build
  • editable recipe content and metadata save through the Recipe Library editable-content API
  • recipe display assignment from Home to the mock/default display
  • meal-plan display assignment from the Displays API
  • hosted web receiver creation and recipe assignment
  • Kitchen Inventory add/search/on-hand/not-on-hand flow
  • inventory hydration check from a shopping list
  • safe org-level unit setting changes through As Written, US, and Metric values
  • local backup job trigger
  • selected completed background-job purge through the centralized data purge service
  • API token revocation followed by a negative add-recipe assertion
  • emoji picker payload, unsupported emoji validation, and picker emoji validation
  • expanded secondary-user smoke, recipe, shopping-cart, delete/permission, and inventory checks when a secondary user is configured

Recipe Discovery web search is intentionally not part of the default suite because Google-backed discovery can be affected by bot checks and external network/provider behavior. The default harness may test local discovery support such as inventory ingredient suggestions, but external recipe discovery search should remain an optional slow/external test mode with dedicated fixtures.

2026-06-24.4 Paprika import/export and paired-org coverage

Suite 2026.06.24.4 adds deterministic Paprika fixture coverage using local files under regression_fixtures/paprika/.

New coverage includes:

  • Preview a multi-recipe Paprika fixture.
  • Import the fixture into the primary regression org.
  • Re-import the same fixture as a different recipe type to verify URL-based duplicate handling updates the existing recipes instead of creating duplicates.
  • Export an imported recipe back to Paprika format.
  • Create a paired regression test organization for the same run using the -A / -B naming pattern.
  • Preview the exported Paprika file in the paired org.
  • Import the exported recipe into the paired org as a different recipe type.
  • Wait for paired-org imported recipe cache build timing.

Historical run cleanup is available for multi-org runs when active test organizations still exist. That cleanup action cleans up every regression test organization associated with the run. Each org still appears independently in the Test Organizations list and can also be cleaned up from that list.

The section Refresh links now force a real page reload with a cache-busting query parameter before returning to the requested anchor. View Details links also jump directly to #run-detail after loading the selected run.

The background-job purge probe now records a compact summary instead of the full purge result payload.

Emoji unsupported validation now chooses an unsupported probe dynamically from a list of uncommon emojis. This avoids false failures on installations where a previously unsupported emoji, such as 🎠, was added to the custom emoji registry.

Suite 2026.06.24.5 adds organization-attributed regression step records. This lets the Test Organizations table summarize only the steps executed against each generated organization in multi-org runs, instead of showing the full run summary for both the -A and -B organizations.

Additional coverage includes:

  • Rebuild a recipe cache after changing each org-level recipe units setting.
  • Create a deterministic Spaghetti & Meatballs recipe and an org-level Noun Project footer rule for spaghetti/meatballs.
  • Add that recipe to the meal plan, push the meal plan to the mock display, and verify the meal-plan footer detection and Noun Project icon artifact.
  • Use an imported Paprika fixture recipe for the Let’s Cook session when available, rather than relying only on synthetic generated recipes.
  • Create a managed Blackcap recipe share from the primary regression org to the paired -B org.
  • Open the managed share manifest as the paired org admin.
  • Import the managed share into the paired org.
  • Verify the imported recipe carries shared-import metadata used by the Recipe Library shared badge.
  • Return to Default from a regression org context now redirects directly to the Regression Tests page #test-organizations section.

Emoji validation is now tested through the real recipe edit save path. The harness attempts to save unsupported emoji metadata on an editable recipe and expects the server to reject the save. Server-side edit save validation now rejects unsupported Blackcap emojis and reports unsupported_emojis so invalid emoji handling is enforced even when a client bypasses the browser-side emoji picker validation.

2026-06-24 Regression Harness Follow-up

  • Run detail rows now keep the pass/fail status immediately next to the timestamp and show the attributed organization immediately after the request path.
  • Regression steps without an explicit organization now default to the primary -A test organization when available.
  • The Noun Project footer check treats the display preview path as diagnostic only; current-day footer detection plus a cached spaghetti icon artifact is considered a pass.
  • Managed Blackcap recipe share imports now preserve as-written core recipe metadata such as total time, servings/yield, raw time metadata, nutrition metadata, and cleanup metadata. They do not copy translated render-unit values.
  • The paired-org recipe sharing coverage now verifies that shared import metadata for total time/core metadata is preserved in the recipient organization.

2026-06-24.8 user-flow expansion

Suite 2026.06.24.8 expands default coverage around regular organization-user workflows:

  • Mobile smoke checks now load the main mobile page plus mobile add recipe, recipe discovery, meal planner, Let’s Cook, shopping, kitchen inventory, and the shared About/status endpoint.
  • Admin smoke checks include the About/System surface where the acting user can reach it; redirects/403 responses are accepted as long as the route does not 500.
  • Shopping list coverage now completes and reopens both a fully checked list and a not-fully-checked list.
  • Shopping list coverage now removes a recipe from the recipe cart, re-adds it, creates a recipe-backed shopping list, moves a manual item to a department, and loads Recipe, Department, and None grouping views.
  • Meal planner coverage keeps recipe entry creation, movement, and shopping-list generation anchored to the current day/current week so the suite does not fail near the end of a planner week.
  • Meal planner coverage now creates a manual non-recipe meal-plan item and includes it in the generated shopping list.
  • Display coverage now pushes week, day, and month meal-plan views to a safe display using the current day as the anchor date.
  • Kitchen Inventory coverage now creates storage locations and moves an item between storage locations in addition to on-hand/not-on-hand checks.
  • Let’s Cook coverage now opens Recipe Library before starting the session and attempts to mark every detected step and ingredient complete/used before ending the session.

Generated test user references

Run Details shows generated test users in a reference table. Multi-organization runs show the primary organization administrator, the secondary test user when created, and paired organization administrators such as Org B Admin. Generated passwords remain transient, are hidden by default in the table, and are cleared after cleanup.

Cleanup organization attribution

Manual cleanup records cleanup steps against the relevant regression test organization. For paired import/export runs, cleanup steps for Org A and Org B are attributed independently so the Test Organizations summary can count those steps correctly.

2026-06-24.12 optional backup restore mutation and usability validation

Suite 2026.06.24.12 keeps the optional Test backup restore checkbox on the Start New Run panel. Current Blackcap organization backups restore in-place to the same organization; the restore service rejects backups whose embedded source organization differs from the restore target. Because of that, the regression harness validates same-organization restore against Org A rather than creating an Org C restore target.

When enabled, the restore validation still runs at the end of the main suite, after API token revocation has been tested. The harness now makes the restore probe destructive in a controlled way:

  • creates a dedicated sentinel recipe and sentinel shopping list,
  • captures a baseline of the sentinel recipe/list before backup,
  • captures the compact pre-restore summary of core user-facing org data counts,
  • creates a local organization backup through the existing organization backup service,
  • archives the sentinel recipe after the backup,
  • checks and completes the sentinel shopping list after the backup,
  • restores that backup back into Org A,
  • compares core user-facing org counts,
  • verifies that the archived recipe is active again and the completed shopping list is back to the pre-backup active/unchecked state, and
  • runs post-restore usability checks against the restored sentinel data: recipe detail payload, recipe selection, recipe cart add, shopping-list view, shopping-list completion, persisted completion status, reopen, and reopened active/unchecked state.

Regression harness tables are intentionally excluded from organization backup exports and restore clearing. This prevents an in-progress regression restore probe from deleting or rewinding its own run report, step records, or generated organization metadata.

The Run Details generated-user area is now a table with user type down the left side and Organization, Email, Password, and User ID across the top. Passwords are hidden by default, remain copyable, and can be temporarily revealed when needed.

2026-06-24.13 regression option reconciliation

Suite 2026.06.24.13 reconciles the manual and scheduled regression controls. Manual and scheduled runs now expose the same meaningful run-option checkboxes:

  • create secondary user,
  • run secondary-user tests,
  • keep org on failure, and
  • test backup restore.

The retired slow-test and skip-external-provider checkboxes were removed from the page because the current deterministic suite does not branch on those flags. External shopping-list provider automation and Google-backed recipe discovery remain intentionally outside the default deterministic harness until dedicated fixtures or isolated provider stubs exist.

Secondary-user tests now imply secondary-user creation in both manual and scheduled runs. The UI keeps those two checkboxes in sync, and the server-side config parsing enforces the same rule so submitted forms and scheduled rows stay valid even without JavaScript.

Scheduled runs now store their own backup-restore option in regression_test_schedules.include_backup_restore_test, and their own AI-provider mode in regression_test_schedules.use_real_ai_providers (default synthetic). They still merge in the shared fake recipe/API defaults from regression_test_defaults, but schedule run mode and schedule run options are independent from the Start New Run panel. Weekly schedules now expose a day-of-week selector, and the schedule time zone is selected from the common Blackcap time-zone list instead of being a fixed text value.

The dashboard panel order is now Active Runs, Test Organizations, Historical Runs, then Run Details. Historical Runs hides the verbose Current step column to make the history table easier to scan; the final/current step remains available as a Phase tooltip and in Run Details.

2026-06-24.14 restore cache readiness validation

Suite 2026.06.24.14 tightens the optional backup-restore checks around recipe cache behavior. The restore sentinel recipe now waits for its normal Recipe Library cache build to reach Cached before the organization backup ZIP is created. That ensures the backup contains both the restored recipe row/cache metadata and the org-scoped cache files for the sentinel recipe.

After restore, the post-restore recipe usability checks first call the normal recipe cache status endpoint and expect the restored sentinel recipe to already be ready. This validates that restore brought back cache assets and the metadata links to those assets, rather than depending on a new rebuild after restore. If that immediate readiness check fails, the harness records that failure and then waits for the cache as a fallback before attempting recipe selection, so the run continues with useful downstream diagnostics instead of failing only with a /select-recipe 409.

2026-06-24.15 regression history ownership and purged-org hiding

Suite 2026.06.24.15 clarifies the split between regression history ownership and test-organization association. Regression run history is platform/Default organization data; the generated organization id on steps and marker rows is an attribution value showing where each test actually ran, not ownership that makes those history rows part of the generated org's lifecycle purge.

The Test Organizations panel now shows only generated organizations that still have a live organizations row. When organization lifecycle purge permanently removes the generated org, the marker row is retained for historical association and marked purged, but it no longer appears in the operational Test Organizations list and can no longer be opened as an organization context. Historical Runs and Run Details remain available until the separate regression history retention policy removes them.

2026-06-25 documentation coverage matrix update

The regression testing document now includes a coverage matrix showing which areas are covered by manual runs, scheduled runs, the optional backup-restore flow, and the optional secondary-user flow. The matrix also calls out deterministic suite boundaries such as external shopping-list provider automation, Shop With provider website automation, OAuth callbacks, real email delivery, and real e-ink display writes.

This is a documentation-only update; it does not change the regression suite version or the number of executed test steps.

2026-06-26 test performance view

The Regression Tests page now includes a collapsible Test Performance panel. It analyzes existing regression run/step records to show how automated test duration changes from run to run without adding a new persistence layer. Manual diagnostic/test activities and diagnostic copy actions remain visible in Run Details and exports, but they are excluded from performance trends because they are not deterministic automated steps.

The performance table is keyed by stable test identity instead of visible step number. The page exposes search, group, status, run type, and max-test filters; the max-test selector lets System Admins raise the returned test count beyond the default 100-record query limit for larger suites. Selecting a test updates the sticky trend chart, and the selected pill remains highlighted, including the first auto-selected test when the panel loads.

This is a UI/reporting update only. It does not change the regression suite version, the number of executed test steps, normal Performance/Job Status isolation, or regression history purge behavior.

2026-07-02 Let’s Cook workflow and matcher regression coverage

Additional lightweight pytest regression coverage was added for the Let’s Cook matching and setup services because these behaviors are highly text-sensitive and easy to regress while tuning recipes:

  • Worcestershire sauce uses the distinctive term and does not overmatch unrelated generic sauce steps.
  • bell peppers does not match a salt and pepper seasoning step.
  • generic fallback still handles ingredients where directions use only cheese, salt, or flour.
  • collective instructions such as sauce ingredients can gap-fill otherwise-unmentioned sauce components without matching unrelated white rice serving steps.
  • generated Setup/Step 0 uses scaled quantities, ignores non-prep measurements and packaged product forms such as canned diced tomatoes, includes useful drain/rinse prep, and avoids numeric sub-step prefixes.

A small API/database visibility test also guards that the API Tester registry and Database Admin registry continue to expose the current-step, Step 0, full-recipe review, recipe units, and voice access-location fields introduced during this work.

2026-07-07 Database Admin diagnostics coverage

Suite 2026.07.08.3 includes the Database Admin diagnostics coverage first added in 2026.07.07.3 for the new saved-query and column-preference resources, async column preference save/reset, async saved query save/run/delete, on-demand Data Problems loading, the standalone Data Problems tab/fragment endpoint, and recipe purge regression coverage for archived recipes that still have meal-plan references. Run creation now reads the suite version and label from the executable runner metadata so newly queued runs show the same current suite version that the worker uses during execution.

Database Admin / SQL Console coverage

The regression suite includes System Admin SQL Console checks in addition to the normal Org Admin page-load smoke tests. These checks are run with the user that started the regression run, pinned to the Default organization context so platform-only SQL Console permissions are available.

Coverage includes:

  • SQL Console tab fragment loads without a full Database page reload and still shows the inline Export Schema button and rows-per-page control.
  • Schema metadata loads from /admin/database/api/sql-console/schema and includes protected-field tables such as display_content_assignments while omitting protected columns.
  • Table sample values are lazy-loaded from /admin/database/api/sql-console/schema/{table_name}/samples rather than being pulled during the initial schema request.
  • Schema export downloads Markdown that identifies the active SQLite/PostgreSQL provider and SQL dialect, then includes SQL Console limitations, display-assignment guidance, production-query guidance, and regression/diagnostic clone organization filters.
  • Explicit safe-column queries against protected-field tables are allowed.
  • alias.* / SELECT * against protected-field tables is blocked with a clear protected-field message.
  • WITH / CTE queries are blocked with a clear SQL Console limitation message.

These tests intentionally validate the guardrails and export text rather than trying to prove every possible SQL editor autocomplete interaction. Browser-only behaviors such as field-list scrolling, resource-click SQL insertion, and anchored horizontal result scrolling remain manual UI checks unless they are later moved into a browser automation layer.

Run detail performance

Run details are display-only diagnostics, so they are loaded independently from the main Regression Tests dashboard. Clicking a run's detail action now fetches only the run-detail panel instead of rebuilding the whole page, performance panel, and historical tables. Direct links with ?run=<public_id> still work: the page renders a lightweight placeholder and then loads the detail panel asynchronously.

The on-screen detail query avoids loading large sanitized request/response payload columns for passing rows. Failure and warning rows still include request/response detail so the expanded failure diagnostics and Copy issues action remain useful. Full CSV export intentionally reloads the complete payload set because that action is explicit and download-oriented.

Migration 183_regression_detail_indexes.sql adds indexes for the detail display path:

  • regression_test_steps(run_id, created_at, id) for run-detail step ordering
  • regression_test_steps(run_id, status, id) for issue payload lookup
  • regression_test_organizations(run_id, created_at, id) for generated-org detail lookups

Dashboard load performance

The Regression Tests dashboard now defers the Test Performance panel. The initial page load fetches active runs, recent history, schedules, and test organizations, then loads performance timing history asynchronously when the panel is viewed or explicitly loaded. This avoids making the main Regression Tests page wait on the heavier step-history trend query.

The Test Organizations panel no longer runs multiple correlated step-history subqueries for every displayed organization. It loads the visible marker rows first and aggregates all matching step counts/groups in a single indexed pass.

Migration 184_regression_dashboard_cleanup_performance.sql adds indexes for this path:

  • regression_test_runs(status, created_at, finished_at, updated_at)
  • regression_test_runs(public_id)
  • regression_test_steps(run_id, organization_id, status, sequence, id)
  • regression_test_organizations(run_public_id)
  • regression_test_organizations(run_id, organization_id)

Run-detail tables remain horizontally scrollable inside the run-detail card. The detail panel and slot are constrained to the page width so a wide step table does not create a page-level horizontal scrollbar.

Additional dashboard performance tuning keeps the initial Regression Tests page from doing per-run organization-count and diagnostic-duration queries. Recent active/history rows are annotated from grouped aggregate queries, and the async performance endpoint is now represented in the API Tester registry and regression endpoint coverage.

Regression run history can also be manually purged from Data Cleanup after the associated environment has been purged or removed. Manual purge can happen before the configured retention age with explicit confirmation, but scheduled purge still waits for purge.regression_test_runs.after_days.

Latest purge coverage

Regression/unit coverage now includes the newer dependency-aware cleanup domains in addition to the older operational-history groups. It verifies that terminal AI history can be selected early without deleting live Seed balances, successful Social Recipe Imports keep their compact provenance while evidence is removed, abandoned terminal Social Recipe Imports can be deleted completely, Support Chat cleanup removes Sources → Messages → Session, and orphan Regression Test Organization markers are only exposed after the real generated organization/run protection is gone. Database Admin also has a contract check that the detail drawer uses the same foreign-key relationship resolver as list view.

Regression suite 2026.07.08.3 continues to cover manually purging organization-scoped audit rows plus display assignment batches, setting audit rows, Data Cleanup run history, voice assistant command logs, and recipe share audit rows.

Home dashboard and display regression coverage

The regression suite now includes coverage for the Home/System display work:

  • Home dashboard smoke check for Recipe Quick Actions, Today & Tomorrow Meal Plan, Shopping Lists, and Kitchen Inventory panels.
  • Displays page check for Assign Content, Return to Default, and Today & Tomorrow options.
  • Display Details check for header Return to Default and Today & Tomorrow assignment option.
  • Display assignment endpoint check for the Today & Tomorrow meal-plan view.
  • Return-to-default endpoint check that first configures a renderable recipe default, temporarily assigns non-default meal-plan content, and then restores the configured default. This avoids coupling the test to whether a last rendered menu image exists in the test organization.
  • Here Let’s Cook start/end check through the same API used by the Home/Displays Assign Content flow.
  • API Tester registry check for Assign Display Content, Return Displays to Default, and Start Let’s Cook Session.

These tests intentionally exercise service/API behavior and page markup with the Flask test client. They do not simulate browser drag-and-drop JavaScript; the browser-specific Home interactions should still be validated during manual UI passes when the meal planner drag behavior changes.

Restart recovery boundary and warning summary

Application-start recovery derives the last contiguous fully completed regression result. Passed, warning, and skipped rows are complete; failed or non-terminal rows are retry boundaries. Resume markers and diagnostic/manual activity rows are excluded from the boundary calculation.

The runner uses persisted completed-step keys to control the resumed path rather than merely reporting a checkpoint. Generated identifiers needed by the next action, including manual and secondary-user recipe IDs, are saved with completed results and recovered after restart. Transient login and API-token state can be recreated silently.

Suite 2026.07.27.1 adds phase-aware resume sentinels for the major mutation-heavy sections. A completed Paprika, display, inventory, settings, shopping, meal-planner, voice, Let’s Cook, backup/purge, or backup-restore phase is skipped as a unit instead of being replayed from its first HTTP request. The secondary-user account setup is separate from secondary-user test execution, so finding an existing generated user no longer skips unfinished tests. Within that phase, completed page checks and recipe creations are skipped individually. An interruption after Create manual recipe: Secondary Regression Recipe 2 therefore resumes with Wait for secondary recipe 2 cache build rather than replaying Paprika tests or recreating the recipe.

The resume activity marker uses the retry-boundary sequence without advancing the formal test counter or warning totals. The first substantive resumed test consequently retains the expected next sequence. A newly-created transient API token is still revoked quietly when the original token-revocation phase had already completed.

Before any regression Let’s Cook start, the harness checks the exact target and ends only an orphan session explicitly owned by the regression harness. Non-regression user sessions are preserved. Cleanup is recorded as a warning.

Run Details displays Success, Warnings, and Failures summary tiles in that order, followed by Cleanup.

Suite 2026.07.13.2 adds focused coverage for Recipe Library add-to-plan slot/multiplier options, Home Other date enforcement, transactional display deletion safeguards, editable-content-preserving mass cache refresh, and mass-delete meal-plan reference handling. Restart recovery itself remains covered by focused service tests and real restart validation rather than by a regression test that restarts its own runner.

Suite 2026.07.13.3 corrects the editable-content refresh test so it submits the same string-based ingredients and instructions payload as the Recipe Library editor. It also validates the corrected mass-delete dependency path after the PostgreSQL-safe shopping-list dependency query was fixed. Recipe deletion now fails closed if dependency verification cannot complete, so an inspection error can no longer silently archive a recipe.

2026-07-14 Remote display-client lifecycle coverage

Suite 2026.07.14.1 adds a server-side Remote Display Client lifecycle check using a temporary remote_pi_client display. It generates a short-lived pairing code, exchanges it for a client token, reads state with bearer authentication, downloads a generated PNG preview, records a heartbeat containing four-level grayscale physical-panel acknowledgement metadata, explicitly unlinks, and verifies that the revoked token can no longer read state. The temporary display is removed during the test flow.

Focused Python tests—not the standard regression runner—cover attached-client failure and timing behavior: unchanged previews do not refresh the panel again, invalid/partial downloads preserve the last valid file, network and hardware errors preserve the physical image and report Client Error, a later successful update clears the error, Client mode suspends local schedules, expired windows are not replayed, and applicable schedules become eligible after local ownership returns.

The suite does not attempt to restart its own regression runner. Restart-checkpoint behavior remains covered by focused service tests and real restart validation.

Remote Let’s Cook client cadence and restore coverage

Suite 2026.07.14.2 extends the Remote Display Client lifecycle test. After pairing and validating preview/heartbeat behavior, the test starts a Let’s Cook session on the temporary remote_pi_client display and verifies that the state API advertises a 5-second pull interval. It then ends the session through the normal authenticated API and confirms that the display returns to non-Let’s-Cook default content and the client returns to its normal configured pull interval. The temporary client token and display are still revoked/deleted during cleanup.

The remote Let’s Cook refresh watcher now uses a PostgreSQL-safe timer-session lookup without SQLite-only relaxed GROUP BY behavior. The Remote Display Client lifecycle also verifies self-unlink through the durable client token. /api/display-client/unlink is handled by the remote-client token service rather than the normal Blackcap user/API-token middleware, and the revoked credential must then receive HTTP 401 from the state endpoint.

Code & Schema Verification option

The Start New Run and Schedule Runs panels include Code & schema verification. It is checked by default. When selected, the regression harness starts the same tracked jobs exposed on the System page and waits for each terminal result:

  • Direct SQL Audit;

Direct SQL Audit treats generated/dependency directories as excluded path components rather than only nested glob patterns. In particular .venv, venv, node_modules, Python cache directories, and the Extension source are excluded whether they appear directly at the scanned project root or below another directory. This keeps a source-package runtime such as <project-root>/.venv from consuming the audit's file limit or reporting third-party SQL as Blackcap findings, while applying the same rule on Pi and every GCP deployment mode.

  • read-only Database Parity and Schema Validation.

Each job receives its own regression step with the tracked job ID and concise result counts. Unchecking the option omits both jobs without changing the SQL Console smoke tests or other database regression coverage. The option is part of saved shared defaults and has a schedule-specific persisted value.

Scheduled recipe end timing regression

The display regression uses a two-phase real-time probe on a dedicated hosted receiver:

  1. It configures Weekly Meal Plan as the display default, pushes a recipe immediately with an end instant five minutes in the future, verifies the recipe assignment/render state, invokes the scheduled-end path once, and confirms that the first pass leaves the recipe active.
  2. A much later suite step reloads the saved assignment/display timestamps. If needed, it polls until the five-minute end instant, runs the established display refresh path, and records the actual default-restoration timestamp. The step fails if the display reverted before the scheduled end, if it reverted within the separate two-minute early-reversion guard, if the transition was not performed by display_assignment_expiration, or if the restored default is not the configured Weekly Meal Plan. Reaching this later verification after the nominal five-minute window is informational and does not turn a correct return-to-default into a false failure.

The probe uses a dedicated display so unrelated Let’s Cook, deletion, and default-display tests do not overwrite its state. It polls durable assignment state rather than using a single fragile sleep as the assertion.

Regression completion email

Migration 203_regression_completion_email_tracking.py adds idempotent delivery tracking to regression_test_runs. A terminal run atomically claims its completion notification so worker retries and restart recovery cannot send the same completion email twice.

Recipient rules are:

  • manual run: the initiating user and the primary organization admin, deduplicated by email;
  • scheduled/auto run: the primary organization admin only;
  • diagnostic clone: the same rule, using the source organization rather than the temporary clone organization.

The email records the suite, final status, start/end timestamps, duration, totals, success/warning/failure/skip counts, environment-left-open state, and an absolute Run Details URL. The external URL uses the normal [auth] external_base_url / External Base URL setting (with the deployment environment value as a fallback) before considering the local Admin bind host and port. This prevents background emails from linking to localhost:8080 when Blackcap is hosted through nginx, stunnel, or external port forwarding. A CSV issue export is attached only when the run contains an actual failed/warning step, cleanup failure/incomplete state, run-level failure, or a genuinely left-open environment. A completed cleanup is authoritative even if the generated organization remains queryable briefly while deletion or purge scheduling settles. SMTP failure is shown in Run Details and activity logging but never changes the regression run result.

The Run Details panel displays completion-email status, delivery time, recipients, or a safe failure summary. Normal application startup applies migration 203 on both SQLite and PostgreSQL.

Meal Planner guidance and Kitchen Inventory archive coverage

Regression suite 2026.07.29.1 strengthens Kitchen Inventory lifecycle coverage with an explicitly on-hand item. The flow marks the item on hand, deletes/archives it, verifies that it leaves the On hand view and appears in Recently Deleted, opens the Database Admin lifecycle safe edit, changes Lifecycle Status to Active / Restored, and verifies that the same item returns on hand. It then archives the item again through the Database safe edit and restores it through the normal Kitchen Inventory route, confirming the same public ID and pre-delete on-hand state are preserved. The secondary-user role phase still validates that System Admin, Org Admin, and Data Admin are allowed to archive, while Operator and Contributor are denied.

The emoji regression phase was updated because the former page-level emoji action moved beside each eligible field. HTTP regression now checks the stable data-blackcap-emoji-field targets on Recipe Library Add Recipe, Manual Recipe, and Edit Content surfaces. The actual 🙂 buttons are injected by the shared JavaScript decorator, so focused contract tests verify the stable data-blackcap-open-emoji-picker selector, field association, non-submit button type, cursor insertion behavior, and dynamic-field MutationObserver. Tests should not depend on the button's visual position in the DOM.

Focused tests separately cover displayed-week shopping-list fallback, field-specific emoji contracts, Home/Meal Planner PDF links, archive/restore uniqueness, the 14-day organization-scoped purge threshold, Database lifecycle safe-edit routing, API Tester lifecycle metadata, migration idempotence, and legacy endpoint aliases.

AI recipe-image regression coverage

Regression suite 2026.08.09.19 retains the AI Recipe Images phase after the manually entered recipe has verified editable content. The phase registers a temporary in-process regression provider and generates a deterministic PNG through the same provider-independent contract used by the configured AI provider adapters. Because AI image profiles are editable platform configuration, the runtime regression does not assume the original medium/high seed names or prices remain unchanged. It selects the least-expensive enabled positive-cost profile to minimize temporary test grants while still exercising reservation and settlement; an intentionally free-only configuration is also supported. The phase verifies the settled AI Seed terminology and fruit explanation, organization and global personal balances, transactional reservation and one-time settlement at the selected profile's authoritative cost, provider usage recording, and protected candidate preview access.

Image selection follows the current two-stage page workflow. The POST /api/ai/jobs/<job_id>/accept action means Use This Image: it returns pending_save=true and leaves both job and candidate in awaiting_review. The regression then reloads the recipe's current editable content and submits ai_image_candidate_id through POST /save-edit-recipe-content/<recipe_id>. That normal recipe save promotes the candidate to accepted, updates the recipe image, and queues one combined cache rebuild without creating another usage event or AI Seed charge. This distinction prevents the regression harness from mistaking a successful pending selection for a failed acceptance.

Migration/unit tests continue to verify the original seeded defaults separately. The regression adapter implements the same submission callback contract as production adapters, including provider request and response identifiers, so changes to the shared provider lifecycle are exercised by the deterministic phase. The adapter never calls an external provider and cannot incur OpenAI or Gemini charges. The temporary provider connection is encrypted while present and removed after the phase. The accepted image and accounting rows remain part of the test organization and are removed by normal regression cleanup. Restart resume uses the completed recipe-save acceptance step as the phase sentinel so a completed AI generation is not replayed. Failed generation steps include the persisted job error category and message to make provider-contract regressions diagnosable from the exported regression CSV.

AI administration, API Tester, and Database coverage

Regression suite 2026.08.09.19 extends the deterministic AI phases and System Admin tool checks without calling OpenAI or Gemini.

The suite now verifies:

  • the organization-scoped AI Usage JSON API returns the generated usage event;
  • Org Admin usage responses omit provider request identifiers and provider input/output/image token counts;
  • the dedicated AI Usage administration page renders the filters, server-side paging controls, and privacy-appropriate columns;
  • a System Admin can open the all-organization AI Usage JSON report and receives paging and summary metadata;
  • the API Tester spec exposes the AI Usage report, Support Chat transcript, retained generated-image route, reconciliation CSV, organization balance, on-demand adjustment history, and personal balance endpoints;
  • Database Admin lists all 16 AI/support configuration, accounting, job, candidate, usage, Support Chat metadata, and recipe-image-state resources;
  • the AI Usage Events Database resource loads through normal server-paged resource handling;
  • Support Chat Sessions, Messages, and Message Sources load as metadata-only Database resources;
  • protected provider usage/context/transcript/hash metadata stay hidden;
  • the existing deterministic provider flow still reserves, settles once, records provider usage, exposes the protected candidate, and accepts the image through the normal recipe/cache workflow.

Focused tests remain responsible for exhaustive filter combinations, SQLite/PostgreSQL query contracts, organization balance facets, on-demand adjustment-dialog behavior, System Admin-only provider diagnostics, global-personal privacy, and the immediate billing-anchor overlap/double-grant fix. The immediate anchor tests assert that the current bucket is retimed in place, spent/reserved amounts are preserved, a repeated save does not add another allotment, and an existing overlap rolls back with a useful error.

GCP regression correction — 2026.08.09.4

The GCP/PostgreSQL validation pass corrected three environment-specific regression assumptions. System Information rendering checks now authenticate as the run-starting System Admin because /admin/system is protected by system.configure; the generated Org Admin remains intentionally unable to open that page. Support Case numbering now exercises the same backend-neutral settings service on both databases, with SQLite BEGIN IMMEDIATE serialization and a PostgreSQL transaction-scoped advisory lock instead of sending SQLite-only transaction syntax to Cloud SQL. Extension cross-organization rejection accepts either HTTP 401 at token authentication or HTTP 403 at organization membership binding; both outcomes reject the request before Support Chat continuation or transcript access.

The Support Chat escalation checks remain chained intentionally: once the Support Request succeeds, the suite verifies terminal escalated state, cleared Get Help attention after refresh/navigation, duplicate-case replay protection, failed-send claim release/retry, Extension escalation, and Support Case transcript bubble rendering.

Support Chat lifecycle regression coverage — 2026.08.09.4

The full regression harness includes a deterministic Support Chat phase. In synthetic mode, it reuses an eligible configured Support Chat Quality Profile if available; when none exists (including when the use case is disabled), it temporarily provisions an isolated, run-scoped synthetic provider/profile and removes it afterward. The private profile never becomes the global default. Only the owning active synthetic regression organization can select it. This prevents missing real-provider configuration from failing the synthetic tests. Real-provider mode does not provision a synthetic fallback and skips a Support Chat test when no eligible configured real Quality Profile exists. Synthetic Support Chat uses a deterministic in-process response fixture through the normal durable AI job and AI Usage flow; no paid external provider request is made. The phase verifies:

  • opening Get Help returns page-specific suggestions without creating message activity;
  • reopening an untouched session preserves its empty state and returns the initial suggestions again;
  • the first submitted question creates the real conversation and records correlated AI Usage with zero Support Chat AI Seed charge;
  • a second question continues the same bounded session and increments its turn count;
  • That solved it produces terminal resolved / resolved_without_support state;
  • the authorized 🛟 View support chat AI Usage API returns the retained user-visible conversation;
  • Extension Support Chat continuation is proven to be scoped to the same authenticated user_id + organization_id, not browser/IP identity;
  • a second user in the same organization cannot read the first user's Extension chat and receives a separate session;
  • an API token scoped to an organization the user cannot use is rejected before Extension Support Chat access, while repository lookup under the wrong organization also returns no session;
  • Extension status reports the active session before resolution, then clears active/attention state after both resolution and successful Support Case escalation;
  • a resolved Extension chat can invoke the terminal transcript-email endpoint without relying on live SMTP in the deterministic suite;
  • Extension Support Case submission includes the correlated user/assistant transcript and passes it through the same visual chat-bubble email renderer used by Admin escalation;
  • the API Tester spec contains the complete eight-endpoint Extension Support Chat/Support Request sequence;
  • a deterministic provider failure records failed AI Usage, moves the session to provider_failed, and returns the normal human Support Request fallback;
  • early purge of that terminal failed chat removes transcript content while AI Usage remains and the transcript API reports history_available=false.

Restart/resume uses the purged-history AI Usage check as the Support Chat phase sentinel so a fully completed phase is not replayed after an interrupted regression run.

Support Chat backup/restore regression coverage — 2026.08.09.4

Optional same-organization backup/restore validation now creates a resolved Support Chat sentinel containing one user message, one assistant message, and one documentation-source row before the backup. After the ZIP is created the harness purges that session, restores the backup, and verifies that the exact session → ordered messages → documentation source relationship and terminal resolution state return. This protects the explicit-column backup registry and child-row ordering on both SQLite and PostgreSQL rather than merely proving the top-level session table is recognized.

Social recipe isolation and evidence regression coverage — 2026.08.03.2

The automated harness now uses the normal Chrome Extension API-token routes to verify that:

  • a 19-digit TikTok video ID remains exact through deterministic preview and resolves complete supplied evidence without AI
  • an explicitly unmatched Pinterest active pin is rejected instead of importing stale metadata from another pin
  • Database Admin exposes social import and AI media diagnostics while protected payload/hash/path fields remain hidden
  • the social recipe capture resource loads on both SQLite and PostgreSQL when the integer schema_version filter options are derived
  • API Tester publishes the complete social recipe preview, capture, status, AI preflight/reuse/options/job/import/cancel contract

Browser-only pixel behavior remains covered by focused extension contract tests for exact-pin DOM scoping, editor-state clearing, rendered video-content geometry, narrow portrait video handling, thumbnail display, and suppression of blank/manual/import/retry actions until automatic resolution is exhausted.

API reconciliation retirement — 2026.08.09.12

Suite 2026.08.09.12 is the first intentional API-retirement release after the compatibility-safe reconciliation was proven by the full runtime regression suite on Raspberry Pi/SQLite and GCP/PostgreSQL. Blackcap now removes 13 primary duplicate API routes from the Flask route map while preserving their canonical replacements and keeping the previously reconciled shared business services.

The retired duplicate URLs are no longer Flask handlers. A centralized retirement guard returns HTTP 410 Gone before normal authentication/CSRF dispatch and identifies the canonical replacement through X-Blackcap-Retired-Endpoint: true, X-Blackcap-Successor, a successor Link, and a warning. This prevents a retired mutating URL from accidentally executing while still giving older clients a deterministic migration target.

The 13 retired routes are:

  • POST /recipes/discovery/preview-url → POST /recipes/discovery/preview;
  • POST /api/recipe-shares/<recipe_id>/link → POST /api/recipe-shares/link;
  • POST /share-recipe/<recipe_id> → POST /api/recipe-shares/email-pdfs;
  • POST /api/displays/bulk-content → POST /api/displays/assign-content;
  • the nine /api/displays/<display_id>/lets-cook/... duplicates → the matching /api/kiosk/<display_id>/lets-cook/... routes.

Blackcap deliberately keeps 25 retained compatibility aliases for higher-risk Admin/configuration/OAuth paths in this release. Those routes remain callable and continue to advertise Deprecation: true, X-Blackcap-Deprecated-Endpoint: true, X-Blackcap-Successor, a successor Link, and a warning. They are not assigned a Sunset date yet because bookmarks, older administrative clients, and externally configured OAuth callbacks require a separate retirement decision.

The reconciled shared-service boundaries remain unchanged:

  • Admin URL Add, Extension Quick Add, Mobile URL Add, Extension structured capture, and Recipe Discovery Save cross the shared Recipe Write metadata boundary while preserving their different authentication/capture workflows. Manual and photo capture remain source-specific workflows.
  • Recipe PDF email delivery has two active Recipe PDF email HTTP surfaces: canonical browser POST /api/recipe-shares/email-pdfs and Extension POST /api/extension/recipes/<recipe_id>/share-pdf; both use recipe_pdf_email_share_service. The retired /share-recipe/<recipe_id> URL receives 410 and must not render or send a PDF.
  • POST /api/displays/assign-content is the only active public display-assignment command in this family; the historical also_default bulk adapter is removed from the active command service.
  • Kiosk Let’s Cook URLs under /api/kiosk/<display_id>/lets-cook/... are the only active display-scoped Let’s Cook routes in this duplicate family.
  • Extension ping and defaults still share the defaults builder while retaining their intentionally different external contracts.
  • Assisted Shopping keeps distinct action URLs (skip, complete, already-have, review/uncheck) over shared transition/progress logic.

The API Reconciliation runtime regression group now verifies the retirement boundary as well as the canonical behavior:

  • all canonical replacement routes are registered;
  • all 13 retired duplicate route patterns are absent from the Flask URL map;
  • every retired URL returns 410 with the expected successor and cannot execute the old mutating handler;
  • the six recipe-creation workflows still work;
  • Recipe Discovery canonical preview works while /preview-url is retired;
  • canonical Recipe Share Link creation remains idempotent while the path-ID route is retired;
  • both active Recipe PDF email surfaces use the shared mocked service and the retired legacy URL performs no render/send work;
  • canonical display assignment works while bulk-content is retired;
  • all canonical Kiosk Let’s Cook actions work and all nine retired /api/displays/<display>/lets-cook/... aliases return 410 with the matching Kiosk successor;
  • Extension ping/defaults and all four Assisted Shopping state-transition APIs retain their reconciled behavior and never browse or purchase from a retailer;
  • API Tester 1.14 omits the retired duplicate endpoint definitions while still exposing deprecation/successor metadata for retained compatibility aliases;
  • the remaining 25 compatibility aliases and their canonical successors stay registered.

The Extension PDF-sharing regression continues to mock recipe_pdf_email_share_service for rendering/email behavior while mocking the Extension cache-readiness gate at the blueprint boundary. This keeps the test aligned with the reconciled architecture instead of patching renderer/email names that no longer belong to the Extension blueprint.

Raspberry Pi cache-wait and recipe-source fixture rules are unchanged. A running Pi cache build may use the bounded active-state extension when either heartbeat or durable started_at demonstrates liveness, and recipe-fetch regressions use either the authorized deterministic Blackcap fixture or the operator-configured real recipe URL. Fixture workflow variants use blackcap_regression_variant query markers without altering the authorized regression-run path.

The group runs in both normal and diagnostic-clone profiles and remains resume-aware so completed mutating probes are not unnecessarily replayed after an interrupted run. Any future retirement of the remaining 25 aliases should be a separate explicit phase with caller audit, API Tester/documentation updates, and matching runtime regression coverage.

Retirement caller correction — 2026.08.09.13

Suite 2026.08.09.13 corrects two Blackcap-owned callers that were still using APIs retired in the first reconciliation retirement release. The managed Recipe Sharing regression now creates its one-recipe managed share through canonical POST /api/recipe-shares/link with a one-element recipe_ids array. The hosted web receiver and its Let’s Cook display regression now use canonical /api/kiosk/{display_id}/lets-cook/... state/action routes. The receiver still uses /api/displays/{display_id}/... for non-Let’s-Cook receiver/display state where those APIs remain canonical.

The API Reconciliation regression group continues to call the 13 retired URLs intentionally and expects HTTP 410 with successor metadata. Normal feature regressions and Blackcap-owned UI/runtime callers must use canonical routes instead. This distinction prevents retirement probes from masking stale production callers.

API reconciliation retirement Phase 2 — 2026.08.09.14

Suite 2026.08.09.14 retires 21 additional internal compatibility routes after the Phase 1 build and its caller-correction build passed the full runtime regression suite. Together with the 13 Phase 1 removals, Blackcap now has 34 total retired method/path aliases protected by the centralized HTTP 410 successor guard.

Phase 2 retires six /admin/background-operations/jobs/... action aliases in favor of /admin/job-status/jobs/..., nine /admin/system/ai/... action/download aliases in favor of /admin/config/platform/ai/..., two /admin/meal-planner/... API aliases in favor of /api/meal-planner/..., plus /back-to-menu, the Access Activity flag-update alias, the login-provider config-push alias, and the ingredients-only Database Admin canonical-cleanup alias, whose canonical successor is POST /admin/database/resources/recipe_ingredients/canonical/cleanup. Blackcap-owned callers use the canonical replacements before the old route registrations are removed.

The API Reconciliation group verifies both registration and runtime retirement behavior. It confirms every canonical successor remains registered, all 34 retired route patterns are absent from the Flask URL map, and each of the 21 new Phase 2 retirements returns HTTP 410 Gone with the expected X-Blackcap-Retired-Endpoint and X-Blackcap-Successor metadata before authentication, CSRF, or mutation logic can execute. The Phase 1 feature regressions continue to exercise recipe creation, Recipe Discovery, sharing, display assignment, Kiosk Let’s Cook, Extension defaults, Assisted Shopping, cache timing, and regression-fixture behavior. Existing Meal Planner, Displays, AI, Database Admin, and System Admin tests continue to exercise the canonical Phase 2 successors.

Only four compatibility routes remain callable after Phase 2: the legacy Dropbox and Google OAuth callback URLs and GET/POST /admin/shop-with-providers. OAuth callbacks may be stored in external provider applications, so Blackcap will not retire them until there is an explicit callback migration plan. The Shop With provider URL remains a user-facing bookmark/form compatibility surface while the canonical Configuration route is /admin/config/platform/shop-with-providers.

API Tester 1.15 removed the nine curated Phase 2 endpoint definitions that referred only to retired aliases. At that checkpoint, the Shop With provider compatibility definition still exercised retained-deprecation metadata before Phase 3 retirement.

API reconciliation retirement Phase 3 — 2026.08.09.15

Phase 3 retires the final two Blackcap-internal/user-facing compatibility method/path aliases: GET and POST /admin/shop-with-providers. The canonical page remains /admin/config/platform/shop-with-providers. Both retired aliases are absent from Flask registration and return HTTP 410 Gone with X-Blackcap-Successor pointing at the canonical Configuration route.

After this phase, 36 total method/path aliases are retired. Only two callable compatibility aliases remain: /oauth/dropbox/callback and /oauth/google/callback. These are intentionally retained because OAuth callback URLs can be stored in Dropbox or Google provider application configuration outside Blackcap. New/provider-configured integrations should use /cloud-backup/dropbox/callback and /cloud-backup/google/callback; the legacy OAuth callback aliases remain an explicit external-compatibility exception until an operator intentionally migrates provider callback configuration.

The API Reconciliation regression group verifies the Phase 3 410 behavior, all 36 retired route removals, canonical successor presence, and exact handler equivalence for the two retained OAuth callback aliases. API Tester 1.16 removes the old Shop With provider compatibility definition, so no active API Tester operation is deprecated.

Chrome Extension launch bridge and Get Help timing — 2026.08.09.19

Suite 2026.08.09.19 adds explicit regression protection for two browser-facing contracts that API-only coverage could miss. Blackcap-owned Sync External List, Shop With, External Shopping List open/capture, and assisted-shopping launch points must remain wired to the Chrome Extension content bridge. The Extension now re-injects that bridge into already-open Blackcap tabs after an install/update or token configuration change, opens shopping side panels against the initiating tab when available, and avoids resetting recipe-popup mode while an organization-token switch is occurring inside the same user click.

The same suite now verifies progressive Get Help controls across Admin/Mobile and the Chrome Extension. Did that solve it? stays hidden until the first completed assistant response. Create Support Case stays hidden until at least the second completed assistant response, except for the existing provider-failure fallback where automated help could not produce a usable response and human support must remain reachable. The Admin/Mobile stylesheet also explicitly honors the HTML hidden attribute for both rows so display:flex cannot accidentally force them visible on a brand-new chat.

Meal Planner specific-slot people-served contract — 2026.08.09.19

Suite 2026.08.09.19 adds a direct runtime regression for POST /admin/meal-planner/slot-people. The single-slot operation now passes only the parameters accepted by meal_planner_db_service.set_slot_people: plan date, slot id, people served, recalculation/manual-overwrite flags, and actor id. start_time and end_time remain exclusive to the bulk people-served date/time-window operation. This protects both SQLite and PostgreSQL deployments from service/DB signature drift such as TypeError: set_slot_people() got an unexpected keyword argument 'start_time'.

Meal item modal progressive disclosure — 2026.08.09.19

Suite 2026.08.09.19 adds rendered-page regression checks for the Add/Edit Meal Item dialog on the Admin Meal Planner, the Home Meal Planner surface, and Mobile Meal Planner. The common path keeps Meal item name and Include in shopping list immediately visible while Quantity, Unit / measurement, Canonical ingredient, Notes, and reconciliation guidance live inside a native Advanced details section that is collapsed by default. The Advanced summary is intentionally compact, the redundant header Close action is omitted, and the footer uses ↩️ Cancel, ➕ Add Meal Item, and 💾 Save Meal Item action labels.

The Admin and Mobile dialog code resets Advanced to closed every time the editor opens, including edit mode. The dialog remains content-sized in the collapsed state and grows naturally when Advanced is expanded, bounded by the existing viewport-safe maximum so the body scrolls only when the expanded content requires it.

Mobile capture Regression fixtures

The Regression Fixtures popup exposes the transient inputs used by the optional browser/mobile capture scenarios:

  • Mobile URL Text Image renders the run's configured Extension recipe URL as readable photographed-style text. The scenario verifies that Mobile Add Recipe extracts and normalizes that exact URL and stops before recipe import.
  • Mobile QR URL Image generates a standards-compliant QR code containing that same configured Extension recipe URL. The scenario verifies QR decoding/normalization and likewise stops before recipe import.
  • Mobile Recipe Photo / OCR uses a known imported Paprika Regression recipe, Blackcap's real Recipe PDF Share renderer, and the recipe's fixture image; the resulting PDF is rasterized into a transient image and submitted through the production Mobile Add Recipe photo/OCR flow.

The URL-text and QR fixtures are generated dynamically for the active run and are unavailable when Extension recipe URL is not configured; Blackcap does not silently substitute another expected URL for these two tests. Extracted visual/QR URLs remain untrusted input and pass through the same public HTTP/HTTPS and SSRF validation boundary used for typed recipe URLs. The recipe-photo fixture is available only after the deterministic Paprika phase has created the known source recipe. All generated fixture files are transient and are removed with normal request/run teardown.

The browser/mobile scenarios reuse the run's existing Playwright/Chromium context and remain part of the optional browser group. Standard Regression therefore does not require a physical camera, third-party login, or uncontrolled external OCR/AI spend.

Browser / Chrome Extension Integration — 2026.08.13.20

Suite 2026.08.13.20 adds a dedicated Browser / Chrome Extension Integration group. The group is intentionally off by default and is enabled explicitly from Regression Test configuration. Disabled runs do not import the browser-group service, start Playwright, launch Chromium, create a Chromium profile, or allocate browser failure artifacts. The setting is authoritative on every host: Raspberry Pi is allowed when the administrator enables it and the prerequisites are installed; GCP is not enabled automatically merely because it is faster.

Optional Support Chat dependency: Browser / Chrome Extension Integration does not require a configured Support Chat AI provider to run unrelated scenarios. If no enabled Support Chat quality profile has a usable configured provider, the Extension Get Help scenario is recorded as Skipped with an actionable configuration reason; shopping, recipe capture, photo OCR, QR scanning, and other browser scenarios continue, and browser orchestration can still pass. An unexpected preflight/setup exception is a real failure, not a skip. The independent deterministic Support Chat regression can still fail on that missing provider because it explicitly verifies the configured Support Chat contract. Configuring a provider allows Get Help to run during the next pass.

The browser group complements the existing deterministic API/service/source-contract coverage. It owns one Playwright runtime and one temporary Chromium persistent context for the entire group/run, loads the unpacked Manifest V3 extension, discovers the real extension service worker and extension ID, connects the regression API Token through the Extension's actual Options/Test & Save UI, verifies the resolved regression organization, and reuses the authenticated browser session across scenarios. For headless server execution, the harness stages an ephemeral copy of the exact Extension source and adds only the active run's Blackcap origin to that temporary manifest's required host permissions. This avoids a Chrome-owned optional-permission prompt that Playwright cannot reliably drive in headless mode while leaving the production manifest unchanged; the real Options API ping, token validation, storage, and content-script registration still run. Scenario reset closes only scenario-created pages and clears transient extension shopping state. A resumed run starts a fresh browser/profile and treats the browser group as the safe recovery boundary rather than attempting to reconnect to a prior Chromium process.

Before Chromium starts, preflight verifies Playwright/Chromium availability, the unpacked extension and MV3 service-worker entry point, writable temporary profile/artifact storage, the active-run fixture authorization, and bounded reachability of the Blackcap login page. The Regression page shows Playwright, Chromium, extension, manifest, and fixture readiness informationally; readiness never turns the group on by itself. Preflight validates the exact managed Chromium revision required by the installed Playwright package rather than accepting any older cached Chromium. When a mutable/source installation already has an older managed Chromium revision, regression preflight performs one bounded playwright install chromium repair before failing the group; this keeps routine Python package upgrades from silently stranding browser regression on an obsolete Playwright browser revision.

Chromium startup does not assume a warm Manifest V3 runtime. After launching the persistent context, the harness checks Chromium's current service-worker list between short event waits for up to 30 seconds. Fast Pi, GCP Original, and GCP Compiled runs return as soon as the Extension worker exists, while a cold GCP Source Python/runtime start gets the same bounded allowance without environment-specific branching. A worker that never appears still fails the browser group as a real startup error.

Browser execution reports progress through the ordinary Regression Results rows rather than waiting for the entire Chromium group to finish. Preflight, fixture preparation, Chromium/Extension startup, and each scenario are inserted as Running before work begins, then finalized in place as Passed/Failed/Warning/Skipped. Longer scenarios update the same persisted running row with owned stages such as opening the Regression Org A list, waiting for deterministic External Shopping List extraction, waiting for recipe cache generation, or waiting for AI social extraction. Every enabled external-authentication provider is expanded into its own browser result row so one provider's initiation result is independently visible. Run Details does not poll or auto-refresh. The runner persists each running/stage/final change immediately, while the Admin UI fetches a fresh detail snapshot only when the user chooses Refresh, reopens Run Details, or reloads the page. This avoids hidden 2.5-second request traffic and keeps dialogs/Fixtures stable. The run's Current Step is updated at the same runner boundaries. Aggregate pass/fail/skip/warning counts are changed only when a running row reaches a terminal state.

Manual/API activity generated while a formal regression step is running remains persisted with database sequence=0 so it cannot affect resume boundaries or performance identities. Run Details, CSV/error export, reports, and email render those activity rows as numeric child steps of the most recent formal step using two digits, for example 430.01, 430.02, ... 430.99. Two digits are used consistently because the harness does not define a permanent nine-substep ceiling; the validated browser run peaked at six subordinate activities, but future/expanded scenarios may add more. The declared display range is therefore x.01 through x.99, and no three-digit .001 formatting is used.

The browser selectors intentionally follow Blackcap-owned DOM contracts. Shop With targets the visible direct provider control when one provider is configured, otherwise opens #shop-with-provider-dialog before selecting the provider. Connections-based External Shopping List actions first open the exact connection edit dialog containing the run-scoped fixture URL. External-list sync asserts the durable chrome.storage.session.assistedShoppingState bridge result instead of the generic #mode-banner. After any shopping/side-panel scenario, the harness restores the Extension's normal popup.html action mode before recipe capture, social import, or Get Help. Blackcap-owned route navigation waits for the committed route plus an exact page marker rather than a generic document-load event.

Deterministic browser scenarios use the active-run fixture namespace and the existing regression organizations/users. Browser connection setup uses regression organization A rather than creating a browser-only organization. It always creates or restores an enabled run-scoped External Shopping List connector for the deterministic fixture. Shop With provider for browser regression is an optional Regression Test setting populated only from Shop With providers that are enabled, fully configured, and use the Chrome Extension runtime in the current environment. When a provider is selected, the harness creates or restores that provider's normal Shop With connection on regression Org A and creates a real Blackcap regression shopping list through the ordinary shopping-list API, including real manual list items. The browser then opens the real Shop a List page and clicks the exact selected provider action. If no compatible provider is configured/selected, only the Shop With scenario is intentionally skipped; the rest of the browser group continues.

Current coverage includes extension/service-worker/Blackcap bridge startup, Shop With, Sync External List, External Shopping List open, real Extension recipe capture, deterministic social/video evidence capture, Extension Get Help, and existing-page bridge recovery after deliberate MV3 service-worker termination/restart. Recipe capture uses the configured safe public recipe URL when valid; otherwise it uses the active Blackcap recipe fixture. Social/video capture follows the same rule with its own optional public URL. Fixtures are available only while the regression run is active and are not third-party hostname impersonations.

External authentication is a smoke test only. Fixture preparation first discovers the actual platform providers that are both enabled and configured through a request-independent registry/configuration query; it does not construct OAuth URLs outside a Flask request. Chromium then opens Blackcap's generic Login page and requires its exact data-auth-provider controls to match that discovered set before attempting every enabled provider. Each click follows the normal /auth/<provider>/start route and must land on a provider-owned authorization host allowed by that provider's registry host patterns, not necessarily the single hostname used to initiate authorization. For example, Login with Amazon may start at www.amazon.com and legitimately continue at a regional account host such as na.account.amazon.com. The test stops at that provider-owned page: it never enters credentials, solves CAPTCHA, grants consent, or completes login. Network/provider failures are categorized separately from deterministic Blackcap browser failures, and one unavailable provider does not prevent the remaining enabled providers from being attempted.

Browser scenarios use bounded observable waits instead of multi-second sleeps. Failures record bounded, secret-sanitized browser/page/extension/network diagnostics and take one private screenshot where practical. Optional Browser deep diagnostics records a Playwright trace only when the group has a failure/warning. Browser artifacts use the regression run namespace, are available only through the existing System Admin regression permission, and are deleted when their retained regression run is purged. Raw API Tokens, authorization headers, cookies, OAuth state/nonces/codes, passwords, and provider secrets are never persisted in browser diagnostics.

Cancellation is checked between scenarios and inside bounded polling loops; teardown closes only the context/process owned by the run, stops Playwright, and deletes the temporary user-data directory in finally handling. A corrupt browser after a scenario produces one session-health/root failure and stops dependent scenarios instead of generating a cascade. Existing non-browser resume, cleanup, API, recipe, Support Chat, and database-parity coverage remains independent of Chromium.

See Chrome Extension, Authentication Providers, GCP Deployment, and Raspberry Pi Deployment for feature-specific details.

Browser social import and fixture inspection — 2026.08.13.21

Suite 2026.08.13.21 strengthens the browser group around the actual Regression Org A identity and the full social-recipe lifecycle. Browser startup logs in through Blackcap's owned #login-email / #login-password form as Regression Org A Admin, waits explicitly for the post-login route instead of relying on Playwright's implicit click-navigation wait, and verifies both the signed-in Admin email and BLACKCAP_CURRENT_ORG_ID before scenarios begin. The Extension is configured with the run-scoped API Token created for that same Org A Admin and Org A; the Options/Test & Save ping now returns the token user id as well as organization id, and startup verifies both values before continuing.

Scheduled GCP browser runs also tolerate one bounded transient post-login navigation delay. The owned Login form remains authoritative and explicit Login/MFA errors still fail immediately, but the harness waits for the /admin navigation commit rather than requiring the full DOMContentLoaded event inside a fixed 30-second window. If Chromium/server navigation bookkeeping times out without an owned Login error, the harness retries the generated Regression Org A Admin login once and then performs the same signed-in email and active-organization assertions. The startup result records browser_login_attempts and browser_login_recovery, so a recovered timing fluctuation is visible without turning an otherwise healthy nightly run red. Two failed bounded attempts still fail the browser group with the current path and navigation error in the result.

Social recipe coverage is split into three intentionally different scenarios. Deterministic social recipe full import always uses the active Blackcap social fixture, runs the actual Extension Capture Page/review/import flow through cache completion, verifies the resulting recipe is readable by the logged-in Org A Admin, and proves that no AI social-review job was submitted and the expected AI Seed cost is zero. AI-assisted social recipe full import uses a second active fixture whose written page evidence is deliberately incomplete while its short bounded video contains the complete recipe in both readable on-screen text and narration. The Extension must reach AI review, enumerate the enabled usable Social Recipe Extraction profiles, select the lowest-cost usable profile by its exact published AI Seed cost, use fixture preparation to grant Regression Org A only the exact organization AI Seed shortfall needed for that profile, explicitly confirm funding, capture/upload the bounded media and sampled frames, wait for the real AI-reviewed editable draft, import it, and verify the resulting cached recipe through the Org A browser session. This scenario intentionally consumes the selected lowest AI Seed amount. A depleted disposable regression balance is not a skip condition; only the absence of an enabled usable Social Recipe Extraction profile skips the AI-assisted scenario. Configured public social/video smoke remains optional and stops at capture evidence so third-party availability cannot replace the deterministic Blackcap fixture as the regression gate.

While a run's fixture authorization is active, Run Details shows a 🪑 Fixtures button beside Run Details: rgr_…. The button opens a native dialog listing the run-specific Quick Add Recipe, External Shopping List, Shop Search, Social Media Recipe, and AI-Assisted Social Media Recipe URLs; every link opens in a new tab. The list is produced by the same active-run authorization service that protects the fixture endpoints, so the button is absent for inactive/completed/expired runs and the URLs themselves stop serving fixture content when the active-run guard closes.

GCP browser validation, API Tester multipart, and concise substeps — 2026.08.14.1

Suite 2026.08.14.1 records the first complete GCP pass of the optional Browser / Chrome Extension Integration group after the hosted-fixture, AI-media, Shop With observation, MV3 lifecycle, and external-auth host-pattern fixes. Run rgr_69683f792aca43 completed the real Chromium/Extension path with Playwright 1.57.0 and Chromium 143: preflight/fixture setup/startup passed; Target Shop With, External Shopping List sync/open, Extension recipe capture, deterministic social import, AI-assisted social import, configured public social smoke, Extension Get Help, all enabled external-auth initiation checks, and existing-page MV3 service-worker recovery all reached their expected outcomes. The AI-assisted fixture selected the configured Economy/OpenAI gpt-5-mini profile at 2 AI Seeds and fixture setup granted only the exact 2-Seed shortfall required for the disposable Regression Org A balance.

The browser diagnostics keep real failures visible while removing only narrowly identified expected noise: missing regression recipe-thumbnail 404 console entries, Target link-preload warnings associated with the retailer page, and ERR_ABORTED GETs for the active run-scoped AI fixture video.webm during page/media teardown. Diagnostic snapshots report how many expected console/request entries were filtered, deduplicate repeated retained entries, and continue retaining unknown Blackcap API failures, worker/page errors, and unrelated retailer errors.

The MV3 lifecycle contract deliberately terminates the unpacked Extension service worker, wakes the background runtime from a surviving Extension page, waits for owned Blackcap content-script registration/reinjection, and then verifies the already-open Blackcap page can continue the real Extension shopping action/state flow without a page refresh. A successful chrome.runtime.sendMessage after termination is authoritative evidence that Chrome revived/dispatched the background runtime; Playwright exposing a replacement Worker target is useful diagnostics but is not itself a correctness requirement.

API Tester registry 1.17 adds request-temporary multipart/file execution for curated binary fields. Recipe import-file preview/import and Social Recipe AI media upload can now be executed from the browser tester with schema-driven file pickers, repeated files such as frames, normal metadata form fields, MIME/size visibility, mutation confirmation where required, and the target endpoint's existing size/permission/AI Seed guardrails. API Tester does not persist an additional copy of selected files. Postman export emits binary schema fields as file form-data rows, including repeated-file guidance.

Visible/exported regression activity substeps now use two decimal digits (x.01 through x.99) instead of three. The underlying activity rows remain sequence=0; formal integer step numbers, resume behavior, and performance identities are unchanged.

Run status summary

Run Details reports four authoritative status totals for the current run/filter scope: Success, Warnings, Failures, and Skipped. Skipped uses the regression run's recorded skip count rather than inferring a skip from empty result fields. A skipped scenario therefore remains visibly distinct from both a pass and a warning, including optional browser/AI scenarios that cannot execute because the environment intentionally lacks a required provider configuration.

The Run Details results table participates in Blackcap's reusable configurable-column system. The semi-transparent ✏️ control in the table header opens the per-user column editor, and visible headers can be dragged to reorder them. Hidden columns retain their saved position, while fixed control/Actions columns remain non-configurable. These preferences are stored in user_data_table_preferences under a stable table key, so the Run Details layout follows the user across sessions without changing the regression data itself.

Meal Planner ordering and Extension Support Chat race safeguards

The standard regression suite must make ordering preconditions explicit. Earlier groups may intentionally leave Meal Plan entries behind for later cross-feature coverage, so a test that expects a particular Meal Item to be the slot anchor must first exercise the normal move API with target_order=0 rather than assuming a newly-created item is first. Day-detail and shared Meal Slot summary checks then validate that the stored display_order is preserved.

The Chrome Extension Support Chat keeps its progressive resolution/escalation buttons disabled while a send or resolution request is still pending. Browser regression waits for both the completed assistant message and the request lock to clear before selecting Need more help. This prevents the UI from exposing a clickable action during the small interval between rendering the assistant response and clearing the asynchronous request state.

Browser regression owned-page navigation resilience

Browser-group startup reuses the already-authenticated /admin page when its owned Home marker is present. Blackcap-owned scenario navigation waits for navigation commit plus an owned DOM marker rather than requiring DOMContentLoaded; this avoids false failures when Chromium has already rendered a usable Blackcap page but Playwright's navigation bookkeeping or unrelated page resources are slow.

Suite 2026.08.26.3 removes the literal Shop This List wording assertion from the executable regression suite; the shopping-list regression continues to exercise the actual assisted-shopping behavior without treating button copy as a functional contract. Diagnostic-clone runs now always exclude Code & schema verification. Browser / Chrome Extension Integration is available as an optional, default-off choice both when creating a diagnostic clone and when manually running diagnostic tests against an existing clone. When selected, the diagnostic runner reuses the same browser/Chrome Extension integration group as the normal regression suite against the diagnostic organization.

Suite 2026.08.26.4 moves the optional choices for rerunning tests on an existing diagnostic clone into a Run Diagnostic Tests dialog instead of placing Browser / Chrome Extension state directly in the Run Details action row. The core diagnostic-safe profile always runs; the dialog can optionally add Browser / Chrome Extension Integration and secondary-user coverage. If the clone does not yet have a generated secondary user, the dialog can create one and choose its role; selecting secondary-user tests implies creation. On later passes, Blackcap reuses the existing generated secondary user rather than creating duplicates, provided its transient regression credential is still available. Code & schema verification remains excluded from all diagnostic-clone passes.

Suite 2026.08.26.5 hardens Browser / Chrome Extension Integration for repeated diagnostic-clone passes. Recipe Photo scenarios re-establish their deterministic Paprika source fixture immediately before each photo scenario so OCR deduplication cannot consume the fixture needed by the following AI-photo scenario. The AI social scenario recognizes retained conversion/audit state and can retry from scratch without deleting that history. Provider-owned OAuth pages that refuse headless Chromium after Blackcap demonstrably initiated the correct external authorization request are treated as external-provider information rather than Blackcap failures.

Suite 2026.08.26.6 restores the shared Blackcap login after external-auth smoke tests and explicitly exercises the Extension's existing-conversion Retry from Scratch path during repeated AI social runs. This prevents an OAuth /logout step or a retained successful AI conversion from leaking state into later browser scenarios.

Suite 2026.08.26.7 refreshes the short-lived active-run fixture authorization immediately before every opted-in Browser / Chrome Extension group. A second or third diagnostic pass therefore gets its own bounded two-hour fixture window instead of depending on the diagnostic run's original creation time; the authorization still expires normally and is disabled when the run is no longer active.

The browser fixture preparation is also self-contained for diagnostic clones. Because the diagnostic profile intentionally skips the Paprika import/export regression group, Browser / Chrome Extension Integration now ensures the deterministic Emoji Lemon Squares Paprika fixture exists before the Recipe Photo scenarios. It reuses an existing copy when present and otherwise imports only that fixture through the production recipe-file API as browser fixture setup. This removes the previous Paprika Regression recipe has not been imported yet skip/failure from the built-in OCR and AI-assisted Recipe Photo browser scenarios.

AI Usage evidence and profile-summary coverage — suite 2026.08.27.1

Suite 2026.08.27.1 extends the AI Usage administration checks for the Recipe Photo/Social Recipe evidence work. System Admin regression verifies the Recipe Photo quality-profile summary endpoint returns current AI Seed pricing plus historical average latency/estimated cost, Database Admin continues hiding the private input_evidence_json, output_evidence_json, and evidence_artifacts_json fields, and API Tester publishes the profile-summary, What Happened JSON, and documented retained-photo artifact routes. The corresponding focused source tests also verify the Recipe Photo summary card, System Admin-only cost/profile controls, Org Admin-safe What Happened privacy boundary, provider-output capture contract, social URL/iframe evidence structure, migration 227 schema declaration, and matching purge/backup lifecycle.

Component Recipe workflow coverage — suite 2026.08.28.1

Suite 2026.08.28.1 adds a deterministic Component Recipes group to the deployed regression harness. The group creates isolated Component and parent fixtures, verifies exact-name/type direct discovery and one-level-only behavior, checks quantity recommendations including parent multipliers, and exercises both keep-as-ingredient and make-from-scratch Shopping List behavior. It also verifies the Meal Planner uses the same Component planning result, Let’s Cook persists the selected Component multiplier and before/after staging snapshot across a session read, and deliberate sharing returns the Component with parent recipe context.

The group intentionally does not test rename, archive, delete, or stored dependency behavior because Component relationships are opportunistic and runtime-only. Component yield persistence and Blackcap file round-trip metadata are covered by focused tests alongside the deployed workflow group. The optional Browser / Chrome Extension Integration group remains reserved for UI/browser smoke coverage rather than owning the deterministic Component behavior.

AI Meal Plan, Meal identity/image, and retention coverage — suite 2026.09.02.1

Suite 2026.09.02.1 extends the deterministic Plan Meal / Let’s Cook Meal regression lifecycle without calling OpenAI, Gemini, or another paid provider. The regression registers a temporary local adapter for both structured Plan Meal text and a generated PNG, and temporarily enables the applicable AI use cases only when necessary before restoring their original configuration.

The deployed lifecycle now verifies a multi-Recipe + Meal Item Meal Slot can generate and persist one validated AI Meal Plan with exact AI Seed accounting and AI Usage correlation; a custom whole-Meal name can be saved without invalidating that plan; a whole-Meal AI image can be generated, reviewed, accepted, and persisted through the normal image-job pipeline; Review Plan JSON and the generated PDF carry the custom Meal grouping/image while retaining the actual Recipe/Meal Item cards; and an intact same-Meal-Slot date move can carry the custom name, Meal image, and current AI Meal Plan without another provider request or AI Seed charge. Material multiplier/content changes still require explicit invalidation confirmation, explicit replan makes exactly one new deterministic provider request, deleting the saved plan removes the protection boundary, and sequential Meal-level Let’s Cook remains available without AI.

Real-provider regression accounting (October 20): The Plan Meal image check measures new AI Usage relative to its pre-image baseline, rather than requiring the whole regression organization to have exactly one image event. A prior AI Recipe Images scenario can legitimately generate its own image. A whole-Meal date move is verified to create no additional AI Usage; once the move succeeds, subsequent Plan Meal/Let’s Cook checks follow the target date even if a separate post-move assertion fails. This prevents an image counter mismatch from cascading into false missing-Meal errors, without reducing Seed settlement or saved-plan validation coverage.

The purge phase also creates an old dated Meal domain and executes the normal Meal Plans cleanup path. It verifies that historical Meal Plan entries, Meal identity metadata, people-served override, saved AI Meal Plan, and AI Meal Plan Recipe references are removed together. Focused tests continue to own provider-parser/token-limit edge cases, exact PDF rendering details, filesystem cleanup, SQLite/PostgreSQL parity, and UI layout contracts.

Claude provider, failure guard, and provider-correlation coverage — suite 2026.09.02.2

Suite 2026.09.02.2 extends the System Admin and Database Admin portions of the deployed regression harness for the Anthropic Claude rollout without making a paid Claude, OpenAI, or Gemini request. The AI Configuration smoke check verifies that Anthropic Claude is offered as a provider, the Quality Profile table exposes Disable Reason, the profile-to-AI Usage diagnostic action is rendered, and the standard ↩️ Cancel action remains present.

The API Tester regression now requires registry specification 1.28 and verifies that the AI Usage provider filter explicitly offers openai, gemini, and anthropic. It also verifies that the System Admin AI Usage contract documents the user-friendly failure_reason and the Provider ID correlation rule: prefer the provider HTTP request identifier and fall back to a durable provider response/message identifier when needed.

Database Admin regression now opens AI Profiles, AI Jobs, and AI Usage Events specifically for the new diagnostics. AI Profiles must expose disable_reason and consecutive_failure_count while hiding provider options; AI Jobs must expose both provider_request_id and provider_response_id while hiding provider snapshots/raw prompts; AI Usage Events must expose failure_reason while keeping provider/private evidence JSON hidden. Focused provider tests continue to exercise Claude nullable-schema normalization, streamed request-ID capture, response-ID fallback behavior, the two-consecutive-failure circuit breaker, and historical Drupe-to-Seed activity display normalization entirely with deterministic local fixtures.

Kitchen Inventory density, Made It, and API/database close-out coverage — suite 2026.09.11.1

Suite 2026.09.11.1 extends the existing deterministic Let’s Cook execution instead of creating a separate density-only regression group. After the ordinary session/history checks, the regression updates and rereads the Recipe-level Made It data with a rating of 4, notes, and medium cleanup difficulty, then opens the Recipe resource in Database Admin to verify those shared fields are visible. The test deliberately validates the existing Recipe record; Made It is not modeled as a per-user or per-Let’s-Cook-session record.

The same Let’s Cook execution now contains a specific ingredient-consumption scenario that creates a unique all-purpose-flour Kitchen Inventory item with a 5 lb physical package and a Recipe that uses 1 cup. The completion preview must resolve the platform All-purpose flour density (0.507 g/mL) rather than the water fallback, estimate about 4.7355 lb remaining, and the normal Let’s Cook completion path must persist approximately that same physical-package quantity. This exercises the real volume↔weight bridge through session completion and Inventory reconciliation rather than only testing the conversion helper in isolation.

The close-out checks also require API Tester registry 1.30 coverage for Kitchen Inventory Product/Overview Photo workflows, Expected Runout, package/Product mutations, Recipe-level Made It, and platform Ingredient Density list/add/save APIs. Database-only Package, remaining-quantity, Product-detail, and Ingredient Density mutations are audited as transactional Multi API operations; multipart Inventory/Made It uploads remain executable directly in API Tester but are excluded from Multi API because file uploads cannot participate in the outer database rollback. Database Admin coverage includes Ingredient Density Profiles, Inventory Products/packages/activity/monthly summaries/photo staging, Recipe Made It fields, and Recipe Edit Content notes.

Kitchen Inventory image fixtures

The active-run 🪑 Fixtures popup now exposes three durable Kitchen Inventory image fixtures in addition to the existing recipe/browser fixtures: Inventory Barcode / UPC, Inventory Product / Half Full, and Inventory Overview / Multiple Products. The links are run-scoped and use the same active-run authorization/expiry rules as the other public regression fixtures.

The barcode PNG contains a known valid retail code and the Kitchen Inventory regression sends that actual image through /mobile/kitchen-inventory/{inventory_item_id}/barcode/decode-frame; the expected value is decoded by Blackcap's production server-side image decoder rather than injected as test JSON. The half-full Product fixture shows a front label and visibly half-full package with no barcode, so Product-photo regression can verify that a realistic front-of-package image stages correctly for optional AI identity/fullness review without pretending a barcode is present. The multi-product overview fixture contains two visibly different packages/fullness levels and replaces the previous blank gray staging image, so the top-level Photo Inventory regression uses a fixture that can also be inspected or submitted during provider-specific AI diagnostics.

AI fullness itself remains provider-assisted. The deterministic core regression verifies the real image staging/evidence/review boundary and keeps the fixture available from Run Details; provider-specific image-understanding diagnostics can use the same durable image without making the core suite depend on a paid model or on a model returning an identical percentage every run.

Plan Meals and Recipe Discovery close-out coverage — suite 2026.09.15.1

Plan Meals regression coverage exercises the asynchronous quote/job/application/AI Usage path with a deterministic regression provider so the test remains stable while still validating real AI Seed accounting. The deterministic provider also verifies that the trusted snapshot contains the run-specific Dietary Preferences and Planning Request, so prompt-context regressions are caught without making a paid external request. The harness inspects enabled Plan Meals Quality Profiles, calculates the two-slot workload price, selects the least-expensive enabled positive-cost profile when one exists (falling back to an intentionally free profile only when that is the configured reality), and mirrors that profile's actual Plan Meals pricing fields into the deterministic regression profile. Before execution, Regression Org A is topped up only by the Seed shortfall needed for both bounded runs. Each run must reserve/settle the configured price and AI Usage must report that price; Find & Add remains off during this paid planning assertion so no additional Find & Add surcharge is introduced.

The deterministic Plan Meal / Let's Cook Meal regression follows the same rule instead of hard-coding a one-Seed test price. It selects the least-expensive enabled configured Plan Meal profile and the least-expensive enabled configured image-generation profile, mirrors each authoritative Seed cost into the temporary deterministic providers, and ensures Regression Org A can fund two Plan Meal operations plus the one whole-Meal image operation before starting. Balance deltas, reservations, settlements, and AI Usage are then checked against those configured costs. AI regressions that are genuinely free by product policy (for example Support Chat in this release, or deterministic non-AI capture paths) remain zero-Seed tests; the regression harness does not globally force AI profile prices to zero.

Recipe Discovery/Find & Add coverage is otherwise deterministic and network-free. It validates provider-URL preference, include/exclude enforcement, Meal Item bypass, detailed Recipe Discovery search, name-only retry, non-slow preference, slow-source last-resort fallback, source/capture handoff, and mixed responses where some Recipes use provider URLs while others use Agent Search. Separate static/runtime contracts verify that non-search Plan Meals profiles can use configured Agent Search, while grounded-search-capable profiles remain encouraged to return real provider URLs.

Close-out contracts also verify API Tester registry 1.35, Transactional Multi API exclusions for provider/background/cross-identity operations, generated Postman collection 1.10.4, Database schema 253, retired Recipe Discovery credential/master-toggle removal, and current Database Admin Plan Meals/Recipe Search resources.

Initiating System Admin organization-context coverage — suite 2026.09.16.2

Suite 2026.09.16.2 includes a dedicated System Admin Organization Context regression group. It is enabled by default for normal manual and scheduled runs and can also be run by itself with System Admin context tests only. The isolated-only mode creates Regression Org A but does not require the paired Regression Org B, so license preflight requires one temporary organization instead of two.

API Tester is validated as a standalone context-aware surface: while the initiating System Admin is inside Regression Org A, /api-tester/context must expose org_admin as the effective role, retain system_admin as the base identity, advertise only Regression Org A, disable the API Tester organization switcher, and /api-tester/spec must omit every platform-only operation. The API Tester header also displays the effective role and a locked organization-context indicator so the support/test context is visible to the user.

Regression organizations are hidden from the normal organization picker, so this group deliberately uses the same special Open Context action as the Regression Tests UI (/admin/regression-tests/orgs/<organization_id>/open-context). It exits with the matching regression-only Return Default action (/admin/regression-tests/return-default), not the ordinary organization switcher. Once the hidden context is established, ordinary page navigation is what the group tests for persistence and effective Org Admin isolation.

System Admin context tests only is mutually exclusive with the other manual/scheduled test-selection checkboxes. Selecting it clears the other test options immediately; selecting any other test option while it is active clears the only-mode flag. Keep org on failure is intentionally independent and remains available in context-only mode so a failed Regression Org A can still be left open for inspection. The server repeats the same normalization so stale or crafted form submissions cannot combine the isolated mode with secondary-user, backup/restore, code/schema, browser, or deep-diagnostic test options. Schedule Enabled and the manual Save shared defaults control are also operational controls rather than test selections and remain independent.

The group runs against the real started_by_user_id identity that initiated the regression run. Blackcap creates a separate Flask regression session for that identity so the background worker never changes the operator's live browser cookie/session. Authentication is seeded only to establish the initiating identity in Default context; the behavior under test then uses the production regression-specific Open Context action to enter Regression Org A and the regression-specific Return Default action to exit it.

While inside Regression Org A, the group verifies that ordinary Admin, Account, Configuration, API Tester, and Mobile navigation never silently restores Default; the rendered effective role is Org Admin; AI Usage, Performance, and Job Status remain locked to the selected organization; platform Configuration and Database SQL Console controls are unavailable; API Tester cannot independently change organization context; and direct platform-only pages/APIs return 403. The final steps explicitly use the regression Return Default action and verify Regression Tests plus System Admin-effective presentation are restored.

The same release adds a focused source contract for the security template context processor after a missing current_user import caused login and error-page rendering to fail with HTTP 500. The contract requires current_user to be imported from permissions_service whenever it is exposed to templates.

Database parity and Extension Support Chat stability — suite 2026.09.16.3

Suite 2026.09.16.3 advances the application database schema marker to migration 247, matching the regression System Admin context schedule migration. This ensures PostgreSQL startup does not incorrectly treat schema 246 as current while the source migration set already includes 247. When the parity audit does fail, the regression result now records the finding category, resource, and finding text instead of only the finding count.

The Chrome Extension Get Help flow now releases its request/composer lock as soon as the owned Support Chat message response has rendered. Refreshing the launcher attention indicator is non-blocking, so a slow Support Chat status refresh cannot freeze progressive actions after a successful response. Extension package version 1.8.12 contains this change.

Support Chat provider work is now durable and intentionally asynchronous: the message POST returns a queued waiting_for_ai assistant row, while Admin and Extension clients poll the stored session until the assistant response reaches a completed or terminal state. The deterministic Support Chat regression follows the same contract instead of assuming the POST contains the final answer. It verifies the queued response first, then polls session history before checking AI Usage, progressive controls, resolution/escalation, and provider-failure fallback. This prevents false regression failures after the durable AI-job migration and verifies that the Chrome Extension's Get Help UI progresses from the working bubble to the completed deterministic assistant response without paid AI usage.

Kitchen Inventory Package freshness dates — suite 2026.09.17.1

Suite 2026.09.17.1 advances the application database schema marker to migration 248 for optional physical-Package freshness dates and advances API Tester to registry 1.31 / generated Postman metadata 1.10.0. Focused coverage verifies full-day and month/year persistence without a fabricated day, month-end expiration calculation, organization warning-threshold overrides and timezone-local calendar behavior, explicit edit/clear, preservation during unrelated Product edits, independent expiration filters, aggregate logical-item warnings, and the rule that an expired Package remains on hand until explicitly changed. API Tester reconciliation also requires the new Package freshness mutation endpoint and classifies it as database-only / Multi API eligible.

USDA FoodKeeper Estimated Best By — suite 2026.09.17.2

Suite 2026.09.17.2 advances the application database schema marker to migration 249, API Tester to registry 1.32, and generated Postman metadata to 1.10.1. Focused FoodKeeper coverage verifies the editable platform catalog, numeric range normalization, preference for numeric from-purchase guidance, Pantry/Refrigerator/Freezer storage-zone selection, creation of system-only Estimated Best By, automatic conversion to ordinary Best By on manual edit, preservation of customized rows across USDA refreshes, and the rule that non-numeric guidance does not manufacture a date. Existing Package freshness warning/filter/on-hand behavior is reused without a second expiration-status implementation.

USDA FoodKeeper refresh and ingredient-preparation follow-up — suite 2026.09.17.3

Suite 2026.09.17.3 changes the Kitchen Inventory FoodKeeper regression from a synthetic manual shelf-life rule to a real USDA-backed catalog item. The regression adds an avocado Kitchen Inventory item, places it in a refrigerator location, creates a physical Package without supplying a freshness date, and verifies that the persisted Package automatically receives Estimated Best By from an active usda_foodkeeper or bundled usda_foodkeeper_fallback profile. It then retains the existing check that a user edit converts that display-only estimate to ordinary Best By.

The FoodKeeper refresh parser now identifies product collections by the FoodKeeper shelf-life row schema rather than accepting the first list of objects in the JSON response. This keeps category/metadata arrays from being mistaken for product rows when the public USDA/FoodSafety.gov wrapper changes. Ingredient normalization also treats compound preparation wording as one instruction: finely chopped thyme normalizes to the Kitchen Inventory identity thyme while recipe ingredient metadata retains finely chopped as the preparation note.

Multi-provider Recipe Discovery and provider-neutral Plan Meals close-out — suite 2026.09.21.1

Suite 2026.09.21.1 extends the deterministic close-out coverage for the four Recipe Discovery provider choices and the recent Plan Meals fixes. Runtime API Tester validation requires the Recipe Discovery search contract, Agent Search image-enrichment operation, and Plan Meals generation/job operations to be present in registry 1.35; external/provider-backed Recipe Discovery operations must remain excluded from Transactional Multi API. The generated Postman collection contract is 1.10.4.

The Plan Meals group now performs a network-free classifier/title guard before its deterministic provider run. OpenAI, Gemini, and Anthropic must all use the conservative classifier; representative simple foods such as steamed jasmine rice and string cheese with grapes must remain Meal Items; clear preparations such as chicken piccata and chicken enchiladas must classify as Recipes; and a generic extracted title such as Recipe must yield to a specific planned title hint without overriding a genuinely specific source title. This verifies the behavior that Find & Add changes source acquisition rather than classification without spending AI Seeds or search-provider credits.

Database Admin regression additionally requires Recipe Search Rules and Recipe Source Websites to be exposed, loads the Recipe Search Rules resource with its key columns, and directly verifies the migration-250 contract that recipe_discovery.search_provider_enabled no longer exists in active settings or setting_definitions. Historical setting-audit rows are intentionally not treated as an error.

Adapt Recipe: synthetic or real provider regression

The Adapt Recipe and Bulk Adapt Recipe groups create disposable source recipes in the isolated regression organization. The same sources, dietary instructions, and expected source IDs are used in either provider mode.

With Use real AI providers unchecked (default), a run-scoped deterministic adapter returns synthetic adaptations through the ordinary durable AI job. With it checked, Blackcap selects the lowest-cost eligible enabled real Quality Profile and submits the identical source fixtures to OpenAI, Gemini or Anthropic. Missing real-provider credentials, use-case configuration or an eligible Quality Profile are reported as Skipped. Actual submitted provider failures and incorrect results are reported as Failed, without a synthetic fallback.

Both modes exercise regular Organization AI Seed funding/reservation/settlement, AI Usage evidence, idempotency, user/organization isolation and source links. Single-recipe saves verify both the canonical Recipe Adaptation identity and saved editable content; the cache's output-file type can change as rendering progresses and is not used as a proxy for editability. The bulk synthetic adapter accepts both the legacy heading and the current compact batch prompt, which includes the full list of expected source recipe IDs. Incomplete bulk results are not saved or charged in Blackcap Seeds.

The test saves linked adaptations and verifies that metadata survives a cache build and that deleting the source puts it on Hold while its descendants exist. A resumed regression reuses the existing durable job instead of issuing another provider request. Cleanup/keep-on-failure rules apply as usual. Real-provider execution can incur external AI provider costs even when Blackcap releases a failed request's reserved AI Seeds.

Recipe editors: browser end-to-end coverage

When Include browser / Chrome Extension integration is enabled, the existing isolated Regression Org A Chromium group now runs four independent production UI journeys. Each test creates its own disposable recipe(s), submits through the same page controls and authenticated endpoints as a user, reloads the persisted result, and archives its recipes before returning. Failure screenshots and step results are reported by the normal Browser / Chrome Extension Integration group.

  • Manual Add Recipe · save and select: both Save Only and Save and Select, ingredient/instruction/notes persistence and Cookbook Category assignment. If PDF caching is still queued, selection follows Blackcap's normal deferred selection behavior rather than requiring a cached result prematurely.
  • Recipe Library · Add Recipe: adds via the Add New Recipe form and checks saved name, description, layout, Cookbook Category and a matching Library result.
  • Edit Recipe Content · full editor: creates a manual source, edits the title, description, ingredients, steps, notes, layout and Cookbook Category, uploads a replacement image, then reloads the full editor and validates the saved result.
  • Edit Recipe Metadata · quick editor: updates metadata/category through the Recipe Library Edit action, verifies original ingredients and instructions are unchanged, and follows the Edit Content handoff for the same recipe.

These are true browser journeys, not merely template/JavaScript assertions. They do not submit AI image-generation requests and behave identically with synthetic or real AI provider regression mode. The browser/Extension prerequisite check remains authoritative: Chromium and Playwright must be available, and the optional browser group must be selected. Existing non-browser recipe API tests continue to run without that checkbox. All test fixtures remain confined to the regression organization; automatic organization cleanup is unchanged.

Browser AI provider mode (including Extension social import and mobile recipe photos) — 0.7.2-alpha

The Use real AI providers option applies to every AI request submitted by the regression organization, including Browser / Chrome Extension Integration. The fixtures (video, image, recipe, and user-entered content) do not change with the mode.

  • Unchecked — synthetic (default): The AI-assisted social-video import and AI-assisted Recipe Photo Capture run through their normal UI, durable AI job, review, save, and cache flows using a private run-scoped synthetic media adapter. It returns deterministic output from the same active regression fixtures and does not contact OpenAI, Gemini, Claude, or another external AI provider. The synthetic media profiles have a 0 AI Seed price and are available only to the active regression organization. Blackcap removes them after the browser group. The normal deterministic social/photo scenarios are still tested separately.
  • Checked — real: The same fixtures are submitted to the cheapest eligible configured Social Recipe Extraction and Recipe Photo Extraction providers. Real-provider usage and AI Seed charges are recorded. An unconfigured AI use case/profile is Skipped; a request submitted to a real provider which then fails is Failed.

As a second safeguard, the durable AI interaction boundary refuses to submit OpenAI, Gemini or Claude requests for a regression organization whose persisted run configuration is synthetic. A browser or Extension API payload cannot override the mode. Historical run records without a real-AI selection remain synthetic.

October 16 regression compatibility fixes: The durable provider boundary continues to block paid requests in synthetic mode, but permits the server-verified Support Chat fixture for the same regression organization/session/job, even when its configured profile references OpenAI, Gemini or Claude. This avoids false provider failures in both Pi and GCP Support Chat checks and the Extension Get Help progressive flow. Private browser-media profile ownership SQL now lives in the regression data-access service, avoiding an unclear Direct SQL Audit finding. The browser recipe-editor checks open the real Recipe Library ⋯ action drawer before choosing Edit Metadata. Edit Content verification tolerates a successful JSON save body being discarded during immediate browser navigation, but still requires the expected redirect and persistence of recipe content, metadata, categories, and the uploaded image; an HTML/error response must never count as a successful save.

October 19 follow-up (synthetic Support Chat and Recipe Editor browser checks): The Support Chat test itself owns a run-scoped temporary synthetic Quality Profile when no usable profile is configured, so normal, resumed, and diagnostic runs behave alike and never require a paid provider in synthetic mode. The profile is removed after the test. Real mode never falls back to synthetic. Recipe-editor browser checks refresh Cookbook Categories through the normal organization-scoped Recipe Library endpoint when cached category options are stale, cancel pending search debounce before opening the action drawer, and wait for its animation to finish; a redraw causes a bounded retry. Category diagnostics now distinguish a missing editor option from a category missing in the signed-in organization.

0.7.3-alpha testing status

New focused local tests verify original-generation structured output, comparison compatibility validation, synthetic provider routing, basic two-stage UI contracts and prevention of automatic recovery submission for held combination jobs. They do not replace an integrated database and browser test suite. Before deployment, test Seed reservation and settlement on both SQLite and PostgreSQL; both synthetic and real providers; compare/generate cancellation, concurrent confirmation, retries, page refresh and expiry; original/secondary Recipe relationships; and modal focus/backdrops. Real-provider regressions must not silently fall back to synthetic. Automated Plan Meals original generation and its end-to-end tests remain unimplemented.

Optional Admin mobile responsiveness smoke check (manual runs)

The Start New Run → Manual run options → Admin mobile responsiveness checkbox enables a small, separate browser check for the legacy Admin console. It is off by default and is not tied to Browser / Chrome Extension integration. It runs only on manually initiated normal regression runs; the scheduled run builder explicitly disables it. System Admin context tests only is mutually exclusive with this checkbox, as with other general test options.

When opted in, the existing disposable regression organization and generated Org Admin credentials are used for a read-only Chromium/Playwright session. The group checks authenticated navigation and the presence of the Admin shell and mobile menu on Home, Recipe Library, Meal Planner, Shopping Lists, Kitchen Inventory, Games, Organization, and Users, where available to the generated user. It also checks 320px portrait, 390px portrait, and 844px landscape samples, touch navigation opening/closing, and visible Recipe Library action placement. Missing permission-dependent links produce Skipped; minor page-level horizontal overflow produces Warning because some Admin tables intentionally scroll. Missing/hidden main content, inaccessible navigation, failed page loading/authentication, and recipe action buttons extending outside the viewport produce Failed. Missing Playwright/Chromium is recorded as Skipped, not a worker crash. Results appear as ordinary steps under Admin Mobile Responsiveness in run details.

This is a high-level smoke check, not a mobile redesign, screenshot-diff baseline, deep dialog workflow test, or substitute for Android Chrome/iPhone Safari testing. It does not write application data, execute destructive Admin actions, load the Chrome Extension, or access paid AI providers. A full normal regression run still creates the usual disposable test organizations, so usual lifecycle/license requirements apply. If external browser access is not possible, configure the existing BLACKCAP_REGRESSION_FIXTURE_BASE_URL / BLACKCAP_INTERNAL_BASE_URL or Admin external base URL to a reachable Blackcap origin as for other browser tests.

On this page