Voice Assistants

Audience: User, Org Admin, System Admin, Developer, Support Related: Overview · External Shopping · Users Permissions And Authentication

Blackcap supports Alexa Custom Skill, Alexa Smart Home, and Google Home integrations for supported Let’s Cook, display, timer, and household shopping actions. Voice connections are associated with a Blackcap user and organization and can be linked to an appropriate display.

Provider capabilities differ. The Custom Skill supports richer conversational commands; Smart Home and Google Home expose the actions supported by their respective device and scene models.

Voice connection architecture and commands

Blackcap Voice is the working design for connecting Blackcap to Alexa and Google Home so voice commands can control Let’s Cook sessions without duplicating Let’s Cook logic inside provider-specific code.

Status: Alexa and Google Home voice assistant integration complete for current development use. This repository includes the provider-neutral schema, display resolution service, meal-plan recipe selection helper, voice command service foundation, Let’s Cook step/timer helpers, Alexa account linking/custom-skill fulfillment, Alexa Smart Home microwave-style display devices, Google Home microwave-style Cloud-to-cloud devices, voice-link emails, and example Alexa interaction-model files.

Alexa custom skill implementation status

Alexa is now the first enabled provider path. The implementation is intentionally scoped to a custom skill because Let’s Cook commands are custom cooking intents rather than standard smart-home device traits.

Blackcap endpoints for Alexa:

GET  /voice/alexa/authorize
POST /voice/alexa/token
POST /voice/alexa/skill
GET  /api/voice/alexa/status

The account-linking endpoints implement OAuth authorization-code linking for Alexa. The Alexa skill endpoint accepts signed Alexa custom-skill requests, maps Alexa intents to the provider-neutral Blackcap voice command model, validates the linked Blackcap access token, and then uses the shared Let’s Cook/display services. For HTTPS-hosted skills, Blackcap verifies the request timestamp, configured skill id, and Alexa signature headers unless the explicit unsigned-debug setting is enabled for local testing.

Configuration required before live Alexa testing lives in the database-backed Admin UI, not inky_menu_config.ini:

Admin → Configuration → Platform → Voice Assistant Providers
/admin/config?scope=platform&group=voice_assistant_providers

Configure these Alexa provider fields there:

Enable Alexa Custom Skill
Alexa OAuth Client ID
Alexa OAuth Client Secret
Alexa Skill ID
Verify Alexa Signatures
Allow Unsigned Alexa Debug Requests
Alexa Timestamp Tolerance Seconds
Allowed Alexa Redirect URLs
Alexa Access Token Seconds
Alexa Refresh Token Days
Alexa Authorization Code Seconds

Use Generate / Replace Alexa Client Credentials on that page to create the Alexa OAuth Client ID and Client Secret. In this flow Blackcap is the OAuth provider and the Alexa skill is the OAuth client, so these credentials are generated by Blackcap and copied into the Alexa Developer Console. They are separate from Amazon Login credentials. Use the small 📋 buttons beside the Client ID and Client Secret fields to copy them. Replacing them can require existing linked Alexa accounts to re-link.

Keep Allow Unsigned Alexa Debug Requests disabled outside local testing. The OAuth client secret is stored as a DB-backed encrypted setting.

For Alexa Developer Console account linking:

Authorization URI: https://YOUR_BLACKCAP_HOST/voice/alexa/authorize
Access Token URI:  https://YOUR_BLACKCAP_HOST/voice/alexa/token
Client ID:         the configured Alexa OAuth Client ID
Client Secret:     the configured Alexa OAuth Client Secret
Authorization Grant Type: Authorization Code Grant

Paste the full Alexa Redirect URLs shown by the Alexa Developer Console into Allowed Alexa Redirect URLs, one per line, for example:

https://layla.amazon.com/api/skill/link/...
https://alexa.amazon.co.jp/api/skill/link/...
https://pitangui.amazon.com/api/skill/link/...

Blackcap validates the exact URL when this allow-list is configured. Legacy host-only entries are still accepted as a fallback when they match a standard Alexa account-linking path.

HTTPS / stunnel note

Alexa account linking requires the Authorization URI and Access Token URI to be public HTTPS URLs on normal port 443. If the Pi uses stunnel to terminate TLS and forwards to Gunicorn over local HTTP, keep the Alexa URLs portless:

https://YOUR_BLACKCAP_HOST/voice/alexa/authorize
https://YOUR_BLACKCAP_HOST/voice/alexa/token
https://YOUR_BLACKCAP_HOST/voice/alexa/skill

Then configure Blackcap so the backend knows those requests arrived through external HTTPS:

Admin → Configuration → Platform → Security
External Base URL: https://YOUR_BLACKCAP_HOST
Trust External HTTPS Base URL: Yes

The setting is stored as security.external_base_url so it appears on the Platform → Security page, but Blackcap still applies it to the legacy [auth] external_base_url runtime config key.

For the current stunnel + Gunicorn PROXY protocol deployment, the public base URL must not include the internal :8443 or :9443 listener port once Alexa is configured for port 443. Gunicorn should remain bound to localhost when this trust option is enabled.

The Alexa status API includes diagnostics for this setup:

GET /api/voice/alexa/status

Check current_request_effective_secure and external_base_url_matches_current_host if account linking reaches /voice/alexa/token but Blackcap logs Blocked external HTTP request.

For Alexa+ mobile app linking, the observed development skill path is:

Alexa+ Store → Settings → Original Alexa Skills → Dev → Blackcap → Settings → Link Account

For the Alexa custom skill endpoint:

Default endpoint: https://YOUR_BLACKCAP_HOST/voice/alexa/skill

Example files are provided in:

integrations/alexa/interaction-model.en-US.json
integrations/alexa/skill-manifest.example.json

The first Alexa pass supports these custom intents:

Alexa intent	Blackcap intent	Example utterance
StartLetsCookIntent	START_LETS_COOK	“Alexa, ask Blackcap to start cooking.”
NextStepIntent	NEXT_STEP	“Alexa, ask Blackcap for the next step.”
PreviousStepIntent	PREVIOUS_STEP	“Alexa, ask Blackcap for the previous step.”
StartTimerIntent	START_TIMER	“Alexa, ask Blackcap to start timer.”
PauseTimerIntent	PAUSE_TIMER	“Alexa, ask Blackcap to pause timer.”
ResumeTimerIntent	RESUME_TIMER	“Alexa, ask Blackcap to resume timer.”
StopTimerIntent	STOP_TIMER	“Alexa, ask Blackcap to stop timer.”
CancelTimerIntent	CANCEL_TIMER	“Alexa, ask Blackcap to clear the timer.”
AddTimerMinutesIntent	ADD_TIMER_MINUTES	“Alexa, ask Blackcap to add one minute to the timer.”
AddOneMinuteTimerIntent	ADD_TIMER_MINUTES	“Alexa, ask Blackcap to add one minute to the timer.”
AddFiveMinutesTimerIntent	ADD_TIMER_MINUTES	“Alexa, ask Blackcap to add five minutes to the timer.”
SubtractTimerMinutesIntent	SUBTRACT_TIMER_MINUTES	“Alexa, ask Blackcap to subtract one minute from the timer.”
SubtractOneMinuteTimerIntent	SUBTRACT_TIMER_MINUTES	“Alexa, ask Blackcap to subtract one minute from the timer.”
SubtractFiveMinutesTimerIntent	SUBTRACT_TIMER_MINUTES	“Alexa, ask Blackcap to subtract five minutes from the timer.”
TimerStatusIntent	GET_TIMER_STATUS	“Alexa, ask Blackcap how much time is left.”
EndLetsCookIntent	END_LETS_COOK	“Alexa, ask Blackcap to end Blackcap cooking.”

Alexa may route phrases containing reserved stop/cancel words to built-in intents instead of the custom intents. Blackcap handles those built-ins defensively when they reach /voice/alexa/skill: AMAZON.CancelIntent cancels the active Blackcap timer, and AMAZON.StopIntent ends the active Let’s Cook session. Prefer custom phrases such as “clear the Blackcap timer” and “end Blackcap cooking” during testing because Alexa may otherwise try to manage its own local timers or session. Literal one-minute and five-minute add/subtract commands are modeled as separate Alexa intents so they do not depend on Alexa returning a numeric slot for words such as “one” or “five”. Timer add/subtract commands operate only on the active timer, never start a timer as a side effect, and subtract commands clamp at zero.

Managing Alexa links from Connections

When Alexa account linking succeeds, Blackcap records a provider-neutral voice assistant connection for the selected organization. The normal Connections page shows those Alexa links in the existing Configured Connections panel under a Voice Assistant group. This group is read/manage-only: users cannot create Alexa links from Connections because the official creation flow is still Alexa account linking.

The Voice Assistant group is organization-scoped. Org admins can see Alexa links for users in the current organization, including inactive links. Dates in this group use the same Blackcap UI format as other admin pages: m/d/yy h:mm am/pm. The table shows:

• provider/account summary, with the Alexa account id masked when captured;
• linked Blackcap user;
• connection surface: ask Blackcap Custom Skill or Display Actions;
• explicit linked display for Custom Skill links, or discoverable display-device/action count for Alexa Smart Home links;
• active, disabled, user-disabled, display-problem, or discovery-readiness status;
• last linked, last command, and Smart Home discovery timestamps where applicable.

Available actions:

• Disable Alexa link: disables only the selected voice_assistant_connections row. Access-token and refresh-token lookups require the row to be connected, so disabled links cannot be used by Alexa. The token hashes are not shown or deleted.
• Re-enable Alexa link: marks the selected row connected again. This can make the same link usable again if the tokens are still valid.
• Change linked display: available only for ask Blackcap Custom Skill links. It updates the explicit default display mapping for that Alexa link. Alexa Smart Home links intentionally do not have this action because discovery returns one device for each active display in the linked organization.

If the linked Blackcap user is disabled globally, Alexa access-token and refresh-token lookups fail even if the Alexa link row is still connected. Alexa receives the generic relink response. This is intentionally different from disabling one Alexa link: disabling the user invalidates all of that user's Blackcap access paths, while disabling one Alexa link only blocks that selected voice assistant connection.

Alexa custom-skill session policy

The most reliable Custom Skill pattern is an explicit launch followed by immediate in-session commands:

Alexa, open black cap.
start cooking.
next step.
start timer.
pause timer.
resume timer.
clear the Blackcap timer.
add one minute to the timer.
add five minutes to the timer.
subtract one minute from the timer.
subtract five minutes from the timer.
end Blackcap cooking.

LaunchRequest, help, fallback, and normal cooking-control responses should keep the Alexa skill session open with a reprompt so Alexa keeps follow-up phrases scoped to Blackcap instead of reinterpreting them as Alexa-native cooking or timer commands. The voice session is still short-lived and should not be treated as the durable cooking state. The durable state remains the active Let’s Cook session on the linked Blackcap display.

Full one-shot commands are still useful, but Alexa has shown less reliable routing for cooking/timer phrases outside an already-open skill session:

Alexa, ask black cap to start cooking.
Alexa, ask black cap to go to the next step.
Alexa, ask black cap to start the timer.
Alexa, ask black cap to pause the timer.
Alexa, ask black cap to resume the timer.
Alexa, ask black cap to clear the Blackcap timer.
Alexa, ask black cap to add one minute to the timer.
Alexa, ask black cap to add five minutes to the timer.
Alexa, ask black cap to subtract one minute from the timer.
Alexa, ask black cap to subtract five minutes from the timer.

Blackcap only supports adding or subtracting one or five minutes at a time. If Alexa routes a request such as “add two minutes,” Blackcap should reply that only one or five minutes are supported and must not change the timer.
Alexa, ask black cap to end Blackcap cooking.

End-cooking commands should end both the Let’s Cook session and the Alexa skill session. Timer/step commands, including previous-step, pause, resume, add-one-minute, add-five-minutes, subtract-one-minute, subtract-five-minutes, and cancel/clear timer, should keep the Alexa session open for the next immediate action.

Current display resolution for Alexa custom-skill commands:

1. Spoken display/room name from the DisplayName slot, if included.
2. The explicit default display mapping saved on the linked Alexa connection.
3. First enabled Blackcap display in the linked organization as a fallback only.
4. Friendly setup response if no display is available.

Alexa Custom Skill device IDs identify the Alexa speaker/app and are not treated as Blackcap display endpoints. Smart Home endpoint IDs will be handled separately when Alexa display devices/scenes are added.

Alexa custom skills do not reliably provide room context for arbitrary skill invocations, so room-quality behavior will likely require either explicit display names in the utterance or a later Smart Home discovery layer.

Alexa awareness after Blackcap ends

After END_LETS_COOK returns shouldEndSession: true, Blackcap has ended the Alexa skill session and its own Let’s Cook session for the linked display. There is no provider API hook in the custom-skill response model that clears Alexa’s broader local/runtime awareness or prevents Alexa from answering a later bare phrase such as next step locally. Treat any post-end response without Skill I/O JSON as Alexa-local behavior. The practical mitigation is to avoid bare commands after ending and reopen Blackcap before additional controls.

Hosted receiver Let’s Cook updates

Hosted web receivers, display detail previews, and other passive/non-touch display surfaces still render live Let’s Cook state. Even when a receiver does not support touch interaction, it polls the Let’s Cook state endpoint while that display owns an active session and shows completed steps/ingredients with strike-through styling, active or paused timers, and the current step progress bar. The hosted receiver uses one shared web Let’s Cook layout for interactive and read-only surfaces based on the graphical preview style: a large recipe header, recipe thumbnail, progress rule, side/additional meal item chips, plain two-column-capable ingredients, flowing instructions, and a timer tray anchored at the bottom. Touch controls remain hidden/disabled on non-interactive receivers; Admin, Mobile, Kiosk, or Alexa commands drive the state changes. This web receiver layout is separate from the e-ink render surface.

Alexa Smart Home display-action direction

The Alexa Smart Home foundation now exposes one Alexa endpoint per enabled Blackcap display. The endpoint now uses an appliance-style model: each Blackcap display is discovered with displayCategories: ["MICROWAVE"], native cooking/time/hold/power interfaces, and the existing Alexa.ModeController fallback instance named Blackcap.Action. This mirrors the Google Home microwave-style approach while keeping the existing room-aware Blackcap Home connection.

Example discovered endpoints:

Kitchen Blackcap
Dining Room Blackcap
Basement Kiosk Blackcap

Each discovered endpoint exposes native appliance controls where Alexa supports them, and keeps these Blackcap action modes as a fallback/control surface:

Lets Cook
End Cooking
Next Step
Previous Step
Start Timer
Resume Timer
Pause Timer
Stop Timer
Cancel Timer
Add One Minute
Add Five Minutes
Subtract One Minute
Subtract Five Minutes

This replaced the earlier first-pass SceneController discovery shape that returned one scene endpoint per action per display. Alexa treats SceneController scenes as virtual endpoints, so five displays and eleven actions appeared as fifty-five Alexa devices/scenes. The display-action model is easier to assign to rooms because Alexa sees one Blackcap device per display.

Current Blackcap route for local relay/testing:

POST /voice/alexa/smarthome

Production certification still needs a proper Alexa Smart Home skill deployment. Official Alexa Smart Home skills are typically implemented as AWS Lambda skill code. This Blackcap route and alexa_smart_home_provider adapter are reusable by a thin Lambda relay or local HTTPS development testing, but the Alexa Developer Console Smart Home endpoint/certification path should be validated before submission.

Alexa Smart Home linking is driven by the Smart Home skill requesting the blackcap_smarthome scope in addition to the normal Alexa scopes, for example:

profile blackcap blackcap_smarthome

That scope creates/updates a distinct Alexa Smart Home connection row using the same Alexa OAuth client id and secret as the existing custom skill. The Alexa Smart Home link is organization-scoped and does not select a default display during linking.

During account linking, Blackcap now pre-caches:

• one Smart Home display mapping per active display in the linked organization;
• stable display endpoint ids such as blackcap_display_vmap_...;
• command/action mappings for every supported interaction on each display.

Discovery then does a fast cache-read and returns one endpoint per display. If displays were added after linking and the cache is stale, discovery performs a small repair pass before returning. The Connections page shows a compact Smart Home summary: display-device count, interaction count, last discovery timestamp, and discovery duration. It intentionally does not expand every display/action row.

The earlier voice_assistant_scene_mappings rows are still maintained as an internal command/action cache and to preserve compatibility with any legacy SceneController endpoints already discovered during the first pass. New discovery responses no longer return one scene endpoint per action.

Smart Home command behavior

Smart Home display-action commands flow through the same provider-neutral voice_command_service as the custom skill. Alexa native appliance directives now map as follows:

• Alexa.PowerController.TurnOn, non-OFF Alexa.Cooking.SetCookingMode, and Alexa.Cooking.TimeController.CookByTime with no active session map to START_LETS_COOK.
• Alexa.PowerController.TurnOff and Alexa.Cooking.SetCookingMode with OFF map to END_LETS_COOK.
• Alexa.TimeHoldController.Hold maps to PAUSE_TIMER.
• Alexa.TimeHoldController.Resume resumes a paused timer if possible, otherwise falls back to starting the current-step timer.
• Alexa.Cooking.TimeController.CookByTime during an active session maps to START_TIMER.
• Alexa.Cooking.TimeController.AdjustCookTime maps to the supported Blackcap +1, +5, -1, and -5 timer changes when Alexa sends those deltas.
• Alexa.ModeController.SetMode remains the fallback for specific Blackcap actions, including next/previous step and timer adjustments.

Discovery includes Alexa.Cooking, Alexa.Cooking.TimeController, Alexa.TimeHoldController, Alexa.PowerController, Alexa.ModeController, Alexa.EndpointHealth, and Alexa. ReportState returns endpoint health plus current cooking/power/mode context derived from the active Let’s Cook session for the resolved display.

Smart Home success responses are limited to Alexa's normal Smart Home acknowledgement behavior. Blackcap still logs and executes the same rich provider-neutral command result used by the ask Blackcap skill, but Alexa Smart Home Alexa.Response payloads do not provide the same custom outputSpeech surface as a Custom Skill response. Use ask Blackcap when a rich spoken response is required; use Alexa Smart Home commands for short room-aware control.

Supported action modes:

Action mode	Provider-neutral intent	Notes
LETS_COOK	START_LETS_COOK	Uses the next meal-plan recipe helper; no alphabetical recipe fallback.
END_COOKING	END_LETS_COOK	Ends the active Let’s Cook session and restores default display content.
NEXT_STEP	NEXT_STEP	Same Let’s Cook next-step behavior as custom skill.
PREVIOUS_STEP	PREVIOUS_STEP	Moves back one step by reopening the previous completed step; does not change ingredient checked state.
START_TIMER	START_TIMER	Starts the first available timer in the current incomplete step only.
RESUME_TIMER	RESUME_TIMER	Resumes a paused active timer. Native appliance resume falls back to start timer when no paused timer exists.
PAUSE_TIMER	PAUSE_TIMER	Active timer only.
STOP_TIMER	STOP_TIMER	Active timer only.
CANCEL_TIMER	CANCEL_TIMER	Active timer only.
ADD_TIMER_1_MINUTE	ADD_TIMER_MINUTES	Active timer only.
ADD_TIMER_5_MINUTES	ADD_TIMER_MINUTES	Active timer only.
SUBTRACT_TIMER_1_MINUTE	SUBTRACT_TIMER_MINUTES	Active timer only; clamps at zero.
SUBTRACT_TIMER_5_MINUTES	SUBTRACT_TIMER_MINUTES	Active timer only; clamps at zero.

When a command succeeds, Blackcap returns Alexa.Response with current EndpointHealth, cooking, power, and action mode context properties where applicable. Blackcap also handles Alexa.ReportState with a StateReport containing EndpointHealth.connectivity=OK, Alexa.Cooking state, Alexa.PowerController.powerState, and the fallback action mode. Legacy SceneController.Activate requests are still accepted for old endpoints and return ActivationStarted on success. When Blackcap cannot safely perform the action, it returns Alexa.ErrorResponse, commonly NOT_SUPPORTED_IN_CURRENT_MODE for no active session, no active timer, no current-step timer, or no upcoming meal-plan recipe.

Schema additions

Migration 174_alexa_smart_home_scenes.sql adds:

• connection capability flags: custom_skill_enabled, smart_home_scenes_enabled, last_discovered_at;
• display mapping exposure fields: expose_to_smart_home, smart_home_friendly_name, is_fallback_eligible;
• command log fields: skill_type, provider_endpoint_id, provider_scene_endpoint_id;
• voice_assistant_scene_mappings for cached Smart Home command/action mappings, legacy scene endpoint ids, display mapping, friendly name, enablement, discovery/activation timestamps, and safe error summary.

Certification risks

Alexa Smart Home cooking/microwave-style capabilities have certification constraints. Blackcap is using the appliance model to get a better room-aware control/status surface for a cooking display workflow, not to control a physical heating appliance. If this ever moves toward public certification, review the cooking-appliance policy and UX requirements carefully. The discovery model intentionally limits discovery to one endpoint per enabled display rather than one scene per action per display.

Feasibility summary

Alexa

Alexa now uses two complementary provider surfaces:

• ask Blackcap Custom Skill for rich spoken Let’s Cook feedback and a selected/default display.
• Alexa Smart Home / Blackcap Home for room-aware display targeting, exposed as one microwave-style Smart Home endpoint per enabled Blackcap display.

Both use Blackcap OAuth account linking and call the same provider-neutral command service. The Smart Home skill requests the blackcap_smarthome scope and does not ask the user to choose a single display during linking.

Google Home

Google Home is implemented as a Cloud-to-cloud integration. It uses one microwave-style Google Home device per enabled Blackcap display, with native appliance traits plus a Modes fallback for Blackcap-specific actions. Google Home linking asks for organization only; display targeting is handled by Google device/room assignment.

Google Conversational Actions were sunset in 2023, so Blackcap does not attempt a custom conversational Google Assistant skill. The supported Google path is Cloud-to-cloud smart home fulfillment.

Recommended architecture

Use a provider-neutral core:

provider request
  -> provider adapter
  -> voice_command_service
  -> voice_display_resolution_service
  -> lets_cook_selection_service / lets_cook_service
  -> existing display assignment and render pipeline
  -> centralized DB services

Provider adapters should only translate and validate provider payloads. They should not implement cooking logic.

New foundation modules:

• inky_admin/services/voice_command_service.py
• inky_admin/services/voice_display_resolution_service.py
• inky_admin/services/voice_assistant_db_service.py
• inky_admin/services/lets_cook_selection_service.py

Extended existing modules:

• inky_admin/services/lets_cook_service.py
• inky_admin/services/lets_cook_db_service.py
• inky_admin/repositories/display_repository.py

Provider model recommendation

Option A: Smart Home Device Integration

Use one virtual Blackcap device per controllable display, such as:

Kitchen Blackcap
Dining Room Blackcap
Basement Blackcap

Advantages:

• better room/device assignment;
• provider discovery can map endpoints to displays;
• endpoint ids can become reliable display-resolution inputs.

Concerns:

• smart-home traits do not naturally represent cooking-session commands;
• custom intents like next step and start timer may not fit standard traits.

Option B: Custom Skill / Conversational Integration

Use a provider-specific custom interaction model for cooking commands.

Advantages:

• best fit for next step, start timer, add five minutes, and end Let’s Cook;
• can return friendly setup or no-recipe responses;
• supports a provider-neutral internal command model.

Concerns:

• room context may be weaker;
• invocation phrasing may be less natural than smart-home commands.

Option C: Hybrid

Long-term target:

• use endpoint/device discovery where provider support is useful for display identity;
• use custom command handling for Let’s Cook-specific behavior.

Initial implementation should be Alexa custom skill first, with the schema already prepared for endpoint mappings.

Account linking design

Account linking must use official OAuth flows only.

Rules:

• no Amazon or Google passwords are stored;
• the provider token maps to a Blackcap user;
• the linked user must have active membership in the organization;
• account linking presents only organizations where the user has active membership, including for system admins;
• each provider connection belongs to exactly one organization;
• each Alexa link creates or updates a default display mapping for the selected display;
• display mappings are organization-scoped;
• disabling an Alexa link from Connections sets only that voice assistant connection inactive; token hashes remain stored so the same link can be re-enabled if still valid, or replaced by relinking;
• provider-facing endpoints must not rely on browser session cookies.

Proposed provider-facing routes when implemented:

/voice/alexa/authorize
/voice/alexa/token
/voice/alexa/skill
/voice/alexa/smarthome
/voice/google/authorize
/voice/google/token
/voice/google/fulfillment

The OAuth pages should be mobile-friendly and should reuse Blackcap authentication instead of adding provider passwords.

Organization/user/display scoping

Every voice command must carry or resolve:

organization_id
linked_user_id
provider
provider endpoint/room context if supplied
display_id
intent

Rules:

• a linked user may belong to multiple organizations, but a voice connection record is scoped to one organization;
• mappings never cross organizations;
• fallback display selection only checks displays in the current organization;
• if the linked user loses org access, provider commands for that org must fail;
• display token-only access cannot configure voice assistants.

Permissions added:

voice_assistant.connect
voice_assistant.manage
voice_assistant.control_lets_cook
voice_assistant.view_mappings

Org Admins can connect/manage. Cooking-capable roles can control Let’s Cook through already-linked mappings.

Display and room mapping

Schema foundation:

voice_assistant_connections
voice_assistant_display_mappings
voice_assistant_command_log

A provider account may expose multiple displays. A Blackcap display may be exposed to more than one provider.

Example mapping:

Provider: Alexa
Voice name: Kitchen Blackcap
Blackcap display: Kitchen E-Ink
Enabled: yes

Display resolution rules

Voice commands resolve a display in this order:

1. Provider endpoint/device id mapped to a Blackcap display.
2. Explicit Blackcap display id or display/room name in the command.
3. Provider room name if supplied and mapped.
4. Default display mapping for the linked provider connection.
5. First added enabled Blackcap display in the organization.
6. Exactly one enabled voice mapping for that organization/provider.
7. Friendly setup response.

The first-added fallback is intentionally stable:

organization-scoped enabled displays
ORDER BY created_at ASC, display_id ASC

Disabled displays are skipped. Displays from another organization are never eligible.

Voice commands and intent model

Provider-specific adapters should translate utterances into these internal intents:

START_LETS_COOK
NEXT_STEP
START_TIMER
PAUSE_TIMER
STOP_TIMER
CANCEL_TIMER
ADD_TIMER_MINUTES
GET_TIMER_STATUS
END_LETS_COOK

Initial Alexa utterance examples:

Alexa, open Blackcap.
Alexa, ask Blackcap to start cooking.
Alexa, ask Blackcap to go to the next step.
Alexa, ask Blackcap for the next step.
Alexa, ask Blackcap to start the timer.
Alexa, ask Blackcap to pause the timer.
Alexa, ask Blackcap to stop the timer.
Alexa, ask Blackcap to cancel the timer.
Alexa, ask Blackcap to add one minute to the timer.
Alexa, ask Blackcap to add five minutes to the timer.

Unsupported timer extension amounts, such as two minutes, are rejected with a friendly response instead of being rounded up to five minutes.
Alexa, ask Blackcap how much time is left.
Alexa, ask Blackcap to end Let’s Cook.
Alexa, ask Blackcap to end the cooking session.

Preferred real-use flow is to open Blackcap and keep the short Alexa skill session scoped to Blackcap for immediate cooking controls. Use full one-shot commands when the Alexa session has already timed out. Do not expect the Alexa skill session to last for the whole recipe; the durable cooking state stays in Blackcap on the linked display.

Clear end-session keyword

The explicit end-session command is:

end Let’s Cook

Additional accepted phrases should map to END_LETS_COOK:

end the Let’s Cook session
end cooking with Blackcap
finish cooking with Blackcap
close the Blackcap cooking session

Avoid treating bare stop as an end-session command because stop timer is a timer command. stop cooking can be supported only when mapped clearly to END_LETS_COOK and ideally when the skill is already in a cooking-session context.

END_LETS_COOK ends the active Let’s Cook session for the resolved display and returns that display to its configured default content. This is intentionally stronger than the normal restore path, which may restore the pre-session current assignment for manual web/kiosk flows.

Starting Let’s Cook from voice

Voice starts are implicit; the user did not press a specific recipe. Therefore voice start uses only the next applicable meal-plan recipe.

Rules:

1. Use the currently active timed meal slot when it contains a recipe-backed item.
2. Otherwise use the next upcoming timed meal slot with a recipe-backed item.
3. Untimed meal slots remain eligible for older/simple meal plans and are ordered by date, slot order, and entry order.
4. Include additional recipe-backed entries from the selected slot as side/additional session recipes through the existing Let’s Cook slot logic.
5. If no recipe-backed meal-plan item exists within the upcoming window, do not start a session.
6. Never fall back to the first recipe alphabetically, first recipe id, or a random recipe.

The current implementation uses a 7-day upcoming window. This matches the rolling meal-planner behavior better than a 24-hour window and avoids surprising failures when the next planned recipe is later in the week.

No-recipe response:

I couldn't find an upcoming meal plan recipe for this display. Add a recipe to your meal plan first, then try again.

Explicit recipe starts

Admin/Mobile explicit recipe starts still pass a recipe id and can start that recipe directly.

Meal-plan card starts still pass a meal-plan entry id and preserve side reminders.

Kiosk-style implicit starts now use the shared meal-plan selector and no longer fall back to saved recipes.

Context-sensitive timer behavior

Voice start timer uses the first incomplete step only:

1. Find the first recipe step in the active session that is not completed.
2. Inside that step, find the first detected timer candidate that is not already completed/cancelled.
3. Start that timer.
4. If a timer candidate in that step is already active or paused, report that a timer is already running for this step.
5. If earlier timers in the step are completed/cancelled, start the next timer in the same step.
6. Do not skip to a later step just to find a timer.
7. If the current incomplete step has no timer, respond clearly.

No-timer response:

I don't see a timer in the current step.

Friendly voice responses

Common responses:

Blackcap is not connected to an available display yet. Open Blackcap Connections to choose a display.
I’ll use your default Blackcap display.
There is no active Let's Cook session on this display.
I don't see a timer in the current step.
There isn't an active timer to change.
Added one minute to the timer.
Added five minutes to the timer.
Moving to the next step.
Ending the Let's Cook session.

Responses should remain short and should not read recipe content aloud unless a future command explicitly requires it.

Internal API Tester endpoint

Until provider-specific Alexa and Google Home fulfillment routes are implemented, the safest way to test the provider-neutral command service is through the API Tester.

Endpoint:

POST /api/voice/test-command

This endpoint is intentionally internal and test-only. It requires:

• a logged-in browser session;
• the API Tester request header;
• CSRF validation;
• mutating API Tester confirmation;
• voice_assistant.control_lets_cook;
• the matching Let’s Cook permission for the chosen intent.

Supported API Tester fields include:

provider
intent
organization_id
display_id
room_or_display_name
provider_endpoint_id
provider_room_name
connection_id
minutes
raw_utterance
slots
include_state

The API Tester exposes intent as a dropdown and uses typeahead-backed fields for organizations, displays, voice assistant connections, provider endpoint mappings, and room/display names. Use internal_test as the provider for normal service testing. Use alexa or google_home only when testing provider-specific mapping rows.

Useful first tests:

START_LETS_COOK
NEXT_STEP
START_TIMER
PAUSE_TIMER
STOP_TIMER
CANCEL_TIMER
ADD_TIMER_MINUTES
GET_TIMER_STATUS
END_LETS_COOK

For the clear end-session keyword, test END_LETS_COOK with sample utterances such as:

end Let’s Cook session
end cooking with Blackcap
finish cooking with Blackcap

Provider adapters should not call this test endpoint. Real provider routes should validate provider requests and then call the shared voice_command_service through an Alexa or Google Home adapter.

Security and privacy

Requirements:

• validate provider signatures/requests in provider adapters;
• validate OAuth access tokens and linked user permissions;
• store token references, not raw tokens in logs;
• never store Amazon or Google passwords;
• do not store raw provider payloads by default;
• do not store raw utterances unless explicitly enabled later;
• command logs store summaries only;
• rate-limit provider endpoints if practical;
• sanitize errors before returning voice responses;
• enforce organization and display scoping for every command.

Phased implementation plan

Phase 1 — Design and safe foundation

Completed in this package:

• provider feasibility documented;
• schema foundation added;
• provider-neutral voice command service added;
• display resolution helper added;
• first-added-display fallback added;
• shared meal-plan recipe selector added;
• kiosk implicit Let’s Cook fallback to saved recipes removed;
• context-sensitive current-step timer helper added;
• explicit END_LETS_COOK intent added.

Phase 2 — UI shell and internal testing

Partially completed now:

• internal POST /api/voice/test-command endpoint added;
• API Tester registry exposes the endpoint with an intent dropdown and dynamic typeaheads;
• API Tester option sources include voice connections, voice display mappings, and voice room/display names.

Remaining follow-up:

• add automated regression coverage for Alexa Smart Home and Google Home fulfillment payloads;
• add Google Home Report State / Request Sync if certification is pursued;
• review Alexa and Google certification fit for the microwave-style non-heating display/controller model before public submission.

Phase 3 — Alexa custom skill

Partially completed now:

• OAuth authorization-code account linking;
• Alexa custom skill fulfillment endpoint;
• example interaction model and manifest files;
• Alexa request timestamp, skill id, and signature validation helpers;
• account-link access token validation;
• provider adapter to shared Blackcap voice commands.

Remaining:

• real-device Alexa Developer Console testing;
• richer display-name slot values generated from actual organization displays;
• certification hardening and privacy/terms URLs.

Phase 4 — Alexa endpoint discovery / hybrid test

Optional after custom skill works:

• expose virtual Blackcap displays as endpoints;
• test whether endpoint ids improve room/display resolution;
• keep cooking commands in the custom intent path unless a standard Smart Home API supports them cleanly.

Phase 5 — Google Home design continuation

• continue tracking official Google Home custom-command options;
• consider Cloud-to-cloud only if a standard device type/trait set can represent useful Blackcap display behavior;
• avoid unofficial APIs.

Testing checklist

Display resolution:

• mapped endpoint resolves correct display;
• explicit display id resolves correct display;
• explicit display/room name resolves correct display;
• mapped provider room resolves correct display;
• unmapped room falls back to earliest-created enabled display in the same organization;
• disabled displays are skipped;
• no display produces setup response;
• command log records resolution source.

Recipe selection:

• active timed meal slot wins;
• next upcoming timed recipe wins when no active slot exists;
• untimed meal slots are still eligible;
• side/additional recipes from the same slot are included;
• no recipe-backed upcoming meal means no session starts;
• no alphabetical fallback occurs;
• explicit recipe start still works.

Timer commands:

• start timer uses first incomplete step;
• first timer in current step starts;
• second timer in current step starts after first is completed/cancelled;
• timer does not skip to later steps;
• no timer in current step returns the no-timer response;
• pause/stop/cancel/add time operate on the active timer;
• resume operates on the paused timer and reports a friendly message if no paused timer exists.

End session:

• end Let’s Cook ends the session;
• display restore still runs;
• bare stop timer does not end the session;
• no active session returns a friendly response.

Alexa+ account-linking navigation

In the Alexa+ app, the development skill link path used during testing was:

Alexa+ Store → Settings → Original Alexa Skills → Dev → Blackcap → Settings → Link Account

This path may vary by Alexa app version, but it is the known working route for reaching the development skill account-linking action.

Account-linking approval page troubleshooting

If Alexa opens the Blackcap login page and then account linking fails with a server error immediately after sign-in, check the Blackcap logs for TemplateNotFound: base.html on /voice/alexa/authorize. The Alexa approval page should use the lightweight auth layout and include a CSRF token so the approval POST can complete inside the Alexa app/web view.

The default development invocation name in the Alexa interaction model is black cap because Alexa custom invocation names generally need at least two spoken words.

Alexa token endpoint redirects to login

If account linking reaches the approval page and then fails with Unable to link the skill at this time, check the Blackcap log for this pattern:

POST /voice/alexa/token HTTP/1.1 302
POST /login?next=/voice/alexa/token HTTP/1.1 200

That means Alexa successfully reached the token endpoint, but Blackcap's normal browser-session security guard treated the provider callback like a web page and redirected it to login. /voice/alexa/token and /voice/alexa/skill must remain public provider callbacks; the routes perform their own OAuth client-secret, bearer-token, Alexa signature, and effective-HTTPS validation.

A successful account-linking exchange should show:

GET  /voice/alexa/authorize
POST /voice/alexa/authorize
POST /voice/alexa/token 200

Alexa skill utterance reaches Alexa locally but not Blackcap

If the Alexa Developer Console or Alexa app answers a cooking command without showing Skill I/O JSON and without a Blackcap /voice/alexa/skill request log entry, Alexa did not invoke Blackcap. Server-side Let’s Cook, display mapping, OAuth token, and permission logic cannot run until Alexa sends the skill request.

Use these gates:

Alexa, open black cap

This should create a LaunchRequest and Blackcap should keep the skill session open with a reprompt. Then test immediate in-session action phrases:

start cooking
next step
start timer
pause timer
resume timer
clear the Blackcap timer
add one minute to the timer
add five minutes to the timer
end Blackcap cooking

If the skill session has already timed out, use the full invocation again or reopen Blackcap. Full one-shot commands are still supported, but timer/cooking phrases have been less reliable when Alexa must route them from scratch:

Alexa, ask black cap to start cooking
Alexa, ask black cap to go to the next step
Alexa, ask black cap to start the timer
Alexa, ask black cap to pause the timer
Alexa, ask black cap to resume the timer
Alexa, ask black cap to clear the Blackcap timer
Alexa, ask black cap to add one minute to the timer
Alexa, ask black cap to add five minutes to the timer
Alexa, ask black cap to end Blackcap cooking

Google Home Cloud-to-cloud final working model

Google Home uses a Cloud-to-cloud fulfillment endpoint rather than the Alexa Lambda relay pattern. Blackcap exposes one Google Home device per enabled Blackcap display, one organization-scoped Google Home connection per link, and no default display selection during linking. Display mappings and action mappings are pre-cached during the OAuth token exchange so SYNC can return devices quickly.

The Google Home provider uses these public endpoints:

/voice/google-home/authorize
/voice/google-home/token
/voice/google-home/fulfillment

The generated Blackcap OAuth client credentials live under Platform Configuration → Voice Assistant Providers → Google Home. Use the generated Client ID and Client Secret in the Google Home Developer Console account-linking settings. The recommended Google Home scope is:

profile blackcap_google_home

Google Home microwave appliance model

The working Google Home device model uses a kitchen appliance shape because it produced the best app UI and natural command routing:

action.devices.types.MICROWAVE

Blackcap currently exposes these traits:

action.devices.traits.OnOff
action.devices.traits.StartStop
action.devices.traits.Cook
action.devices.traits.Timer
action.devices.traits.TransportControl
action.devices.traits.Modes

Native Google Home controls map to Blackcap as follows:

Google command/trait	Blackcap behavior
OnOff / StartStop / Cook start	Start Let’s Cook on the display.
OnOff / StartStop / Cook stop	End the active Let’s Cook session.
TransportControl next/previous	Move to next/previous Let’s Cook step.
TransportControl pause/play or TimerPause/TimerResume	Pause the active timer; resume a paused timer or start the current-step timer when none is paused.
TimerAdjust	Add/subtract supported one- or five-minute increments.
TimerCancel	Cancel the active timer.
Modes fallback	Directly invoke any Blackcap action mode when the Google app/voice grammar routes through SetModes.

The Google Home app microwave page gives useful controls: a general play/stop action for start/end cooking, next/previous controls for recipe steps, and timer play/pause for the current-step timer. The app decides which timer adjust/cancel buttons it renders; Blackcap cannot force exact +1, +5, -1, -5, or cancel buttons onto the Google Home device page. Those actions remain available through the Timer trait when Google routes them and through the Modes fallback.

Useful tested voice examples:

Hey Google, start cooking on Office Blackcap
Hey Google, stop cooking on Office Blackcap
Hey Google, next on Office Blackcap
Hey Google, previous on Office Blackcap
Hey Google, pause the timer on Office Blackcap
Hey Google, add five minutes to Office Blackcap
Hey Google, set Office Blackcap to mode next step
Hey Google, set Office Blackcap to mode end cooking

Google Home state

QUERY returns current Blackcap display state regardless of whether the last command came from Google Home, Alexa, the ask Blackcap skill, Mobile UI, or the display. It reports the device online, current cooking/start-stop state, current mode fallback, and timer-related state where available. Blackcap does not yet implement Home Graph Report State / Request Sync; those are recommended before any public certification effort.

Google Home account-linking behavior

Google Home linking asks for the organization only. It never asks for a default Blackcap display because display targeting is determined later by Google Home device identity and room assignment.

A successful account-linking exchange should look like:

GET  /voice/google-home/authorize
POST /voice/google-home/authorize
POST /voice/google-home/token 200
POST /voice/google-home/fulfillment 200  # SYNC

When linking/relinking succeeds, Blackcap sends the voice-link reference email to the linked user and active org admins if SMTP is configured.

Google Home limitations and certification notes

Google Home is working for development use, but before external certification Blackcap should add or verify:

• Home Graph Report State / Request Sync support;
• production-grade Google fulfillment validation and logging expectations;
• whether the microwave model is acceptable for a non-heating cooking display/controller;
• Google Home app UX across Android, iOS, and Nest Hub surfaces;
• regression tests for SYNC, QUERY, EXECUTE, DISCONNECT, and cross-org isolation.

Final voice-assistant operating model

Blackcap now has three complementary voice surfaces:

Surface	Best for	Targeting model	Feedback model
Alexa ask Blackcap Custom Skill	Single/default display and rich spoken feedback.	Linked organization + selected/default display.	Custom skill speech with detailed responses.
Alexa Smart Home / Blackcap Home	Multi-display room-aware control in Alexa rooms/groups.	One microwave-style Smart Home endpoint per enabled display.	Alexa Smart Home acknowledgements plus app state.
Google Home Cloud-to-cloud	Multi-display room-aware control and strong appliance-style app UI.	One microwave-style Google Home device per enabled display.	Google Home acknowledgements plus app/device state.

All three paths call the shared provider-neutral voice_command_service, so Let’s Cook, step, timer, and end-cooking behavior should remain consistent across providers.

Voice country / locale access tracking

Blackcap now records a compact, organization-scoped voice assistant access rollup for Alexa Custom Skill and Alexa Smart Home requests. The Connections page shows a compact 🌍 button on voice assistant links. Hovering the button shows a high-level summary such as Voice assistant access seen from 3 countries/locales: US-14, CA-2, GB-1; clicking it opens a Blackcap-styled details dialog with provider, skill type, linked display, country/locale, request counts, first/last seen, and whether the signal came from provider locale, marketplace, or server-observed callback IP.

The country signal is intentionally conservative. Alexa custom-skill requests reach Blackcap through the Alexa service, and the Blackcap HTTPS endpoint observes the callback peer that reached the server, not a guaranteed end-user household IP. For that reason the UI labels the summary as country/locale rather than pure country. Provider locale values such as en-US, en-GB, or fr-CA are usually more meaningful for Alexa Custom Skill usage than IP-derived country. Alexa Smart Home has less conversational locale detail in ordinary control directives, but the same storage model can capture provider marketplace/locale later and is ready for Google Home provider metadata.

The tracking model stores only a sanitized rollup: organization, provider, skill type, connection/display/mapping identifiers, command category, provider locale/marketplace when available, server-observed IP metadata following the existing access-activity privacy pattern, request count, first/last seen, and purge eligibility. It does not store raw access tokens, refresh tokens, authorization codes, full provider payloads, or provider payloads with sensitive details.

Alexa read recipe steps aloud

ask Blackcap Custom Skill display mappings now have an Alexa-specific Read recipe steps aloud preference. It is available during account linking and later from the Connections page display-selection popup for the Alexa Custom Skill link.

When enabled, Alexa speaks the current step for these Custom Skill commands:

• start Let’s Cook: reads the first current recipe step;
• next step: reads the newly current step after advancing;
• previous step: reads the newly current step after moving back.

When disabled, Alexa gives a short confirmation such as Starting Let’s Cook. or Moving to the next step. while still updating the Blackcap display normally. The setting is intentionally separate from display rendering; a display can show the full recipe while Alexa reads steps, or show only the current step while Alexa stays concise.

The speech formatter strips markup and excess whitespace, includes step numbers, and avoids very long spoken responses. Steps over roughly 260 characters or 45 words are summarized as too long and sent to the display instead. This keeps the service ready for a future start-of-session readout that may read prep guidance or ingredients before the first recipe step.

Alexa Smart Home display actions do not receive the same rich spoken readout because Smart Home directives are appliance/control responses rather than full Custom Skill conversations.

Let’s Cook current-step display mode

Each display has a Let’s Cook Current Step Display Mode setting in Display Rendering Settings:

• Off (full_recipe) — the existing behavior and the default for existing displays.
• Only show current step for voice-initiated Let’s Cook (voice_current_step) — applies only to sessions started by a voice assistant. The legacy stored value current_step is treated as this option.
• Only show current step for all Let’s Cook sessions (all_current_step) — applies to voice, Admin, Mobile, and Kiosk starts on that display.

Current-step mode applies when the session moves into step-by-step cooking. If the organization preference Show full recipe before step-by-step cooking is enabled, the render surface intentionally starts in a full-recipe review phase first, then switches to Setup/Step 0 or Step 1 after the cook says or presses next.

In current-step mode, Blackcap shows the normal header/progress area, then all session ingredients, then the current step as the main visual focus, and then the timer footer/tray. The renderer intentionally shows all ingredients for the active session above the current step. Auto-checking uses the shared ingredient matcher, including distinctive-term, generic fallback, and collective gap-fill behavior, but the visual ingredient list is not yet reduced to only the active step's matched ingredients.

For meal-plan sessions with sides/additional recipe-backed entries, the renderer uses the existing Let’s Cook step order rather than inventing another ordering. The current step comes from whichever session recipe owns the current step, while the header and progress continue to show overall session progress.

Full-recipe review before current-step cooking

The organization-level cooking preference lets_cook.show_full_recipe_review_before_current_step defaults to true. When the target display will render in current-step mode, voice starts, kiosk starts, timed starts, and Admin/Mobile starts can initially show the full recipe review on the render surface. The review phase does not include generated Step 0, does not check off ingredients, and does not edit the saved recipe. Alexa says I sent the full recipe to your display. Say next when you’re ready to start cooking.; the next command leaves review and reads Setup/Step 0 or Step 1 when read-aloud is enabled.

Session-only Setup / Step 0

Current-step cooking can generate a session-only Setup/Step 0 from scaled ingredients with prep cues such as chopped, diced, minced, melted, room temperature, divided, or separated. Step 0 is not counted in the recipe step total and completing it does not check off ingredients. Full-recipe Let’s Cook render surfaces include Step 0 only when the display setting show_setup_step_in_full_lets_cook is enabled.

📖 Show Entire Recipe takeover

When a Let’s Cook session is currently rendered in current-step mode, including non-voice sessions on displays configured for all-session current-step mode, interactive surfaces expose this header control:

📖 Show Entire Recipe

The control is a per-session takeover, not a display-setting change. Pressing it sets the active session render override to full_recipe, re-renders the display, keeps the session active, and preserves checked steps, checked ingredients, active timers, and the current step/focus. The display’s default current-step setting remains unchanged for future sessions.

The button is allowed in public/token-only kiosk or hosted receiver contexts when those views already have valid control authority for that display/session. It remains hidden on non-touch/read-only receivers, static rendered output, and local e-ink/PDF-style render targets where a button cannot be pressed. Admin and Mobile Let’s Cook controls also expose the same takeover action so a non-touch display can be taken over from another device; Admin and Mobile themselves remain full-step control surfaces even when the target display renders one step at a time.

Voice household shopping

This document captures the feasibility review and Blackcap design for organization-level household shopping-list integrations with Alexa/Amazon and Google Home/Google Keep.

The implementation must remain modular and conservative:

• no unofficial scraping or private endpoints
• no stored Amazon or Google passwords
• no provider-specific logic in inky_admin_app.py
• all runtime DB access through the service/repository layer
• external provider items are not checked off remotely until the Blackcap shopping list is completed
• uncompleted external items remain open remotely
• external items keep their own source-of-truth identity even when Shop With presents a duplicate merge

Current feasibility summary

Provider	Current official path	Read open items	Check off / update item	List selection	Push/webhooks	First implementation recommendation
Alexa / Amazon	Not currently safe. Amazon deprecated List Skills and the List Management REST API for Alexa Shopping/To-Do lists on July 1, 2024.	No supported official app/skill API path.	No supported official app/skill API path.	No supported official app/skill API path.	No supported official app/skill API path.	Design only. Do not expose a connection flow.
Google Home / Google Keep	Google Home shopping lists are currently represented through Google Keep list notes. Google Keep has an official API, but the documentation frames it as an enterprise/Workspace administration API and its public REST surface exposes create, get, list, delete, attachment download, and permission operations.	Possibly, if the target account can authorize the Keep API and the shopping note can be identified.	Blocked until verified. The official Keep REST reference does not expose an update/patch method for mutating an existing note or checking an existing list item.	Possible by listing Keep notes and selecting a list note. There is no Blackcap-safe default-list assumption yet.	Not exposed in the reviewed Keep API. Polling would be required.	Gated design only until OAuth/account eligibility and safe update/check-off semantics are verified.

Recommended first provider

Do not implement either provider as a live end-user connection yet.

The earlier assumption that Google Tasks might be the easier provider is not correct for the current Google Home shopping-list behavior. The user-facing Google Home shopping list is managed in Google Keep, not Google Tasks. Keep is therefore the correct target, but the currently documented API is not enough for Blackcap's required completion behavior because Blackcap must be able to check off selected external list items when a Blackcap shopping list is completed.

A future Google Keep implementation can start only after a local feasibility spike proves all of the following against the actual account type Blackcap will support:

1. the OAuth consent flow can request https://www.googleapis.com/auth/keep for that account type
2. notes.list can see the Google Home shopping-list note
3. the note body exposes stable list item identifiers and checked state
4. an official update path exists for setting a specific list item checked without rewriting unrelated note content unsafely
5. provider terms allow this use case for personal/household Blackcap users

Alexa remains blocked until Amazon provides an official replacement for Alexa List Management.

User-facing model

Use Household List as the primary user-facing section label. It describes the purpose better than provider names and still allows provider badges inside the section.

Examples:

Household List
  [Google logo] Milk
  [Amazon logo] Paper towels

If both providers are eventually supported, group by one Household List section with provider badges, not separate main sections, unless the list becomes too noisy.

Provider logos and badges

External list items and connection rows should show the provider logo:

• Alexa/Amazon: reuse /static/auth_provider_logos/amazon.svg
• Google Keep/Google Home: reuse /static/auth_provider_logos/google.svg

Every external item should retain:

• provider / connection type
• provider badge, such as Amazon or Google
• provider_logo_url
• selected external list name
• external item ID
• local Blackcap link/check state

The Connections page should also render the same logo on the connection entry. The current connection-row template already supports provider_icon_url; the foundation service now centralizes the external shopping-list provider logo paths for future connection enrichment.

Connection model

Use organization-scoped connection types:

alexa_shopping_list
google_keep_shopping_list

These are seeded as connection types by migration 143_external_shopping_list_foundation.sql, but they are intentionally hidden from the Add Connection UI until they are live-safe.

Connection metadata should include:

• connected account identity
• selected external list ID/name
• sync enabled/disabled
• last sync / last success / last error
• token reference only, never raw tokens
• feasibility/provider status metadata

External item data model

Use separate external-item tables plus a link table rather than folding external items into shopping_list_items as normal ingredients.

Reasoning:

• recipe/manual items remain owned by Blackcap
• external items remain owned by the provider until Blackcap list completion
• the same external item may appear in more than one active-shopping context over time
• failed remote check-off state can be retried without confusing recipe item completion

Foundation tables:

• external_shopping_lists
• external_shopping_list_items
• shopping_list_external_item_links
• external_shopping_sync_runs

The link table is where a Blackcap shopping list records blackcap_checked, already_have_it, completed_remotely, and sync_error for each external item.

Active shopping list behavior

Define an active shopping list as the current open Blackcap shopping list for the organization: status not completed/deleted and selected via the current is_current behavior.

When opening an active list:

1. refresh external open items if the provider connection is enabled and not stale
2. attach open external items to the list view through shopping_list_external_item_links
3. render them under Household List with provider logo/badge
4. keep their Blackcap checked state local until the list is completed

Recommended first sync triggers:

• connection setup / selected-list change
• opening/viewing an active shopping list
• manual refresh
• immediate pre-completion refresh
• remote check-off during Blackcap list completion

Avoid scheduled sync until provider limits and rate behavior are known.

Completion semantics

When completing a Blackcap shopping list:

• checked external items: attempt provider check-off
• unchecked external items: leave open remotely
• failed provider updates: complete the Blackcap list, show a warning, and retain retry state
• deleted/externally completed remote items: do not fail the whole list

Do not mutate the external provider when an item is merely checked during active shopping. The provider mutation happens at Blackcap list completion.

Shop With duplicate merge behavior

During Shop With, Blackcap can present duplicate needs together without merging their underlying records.

Example:

• recipe item: 1 cup milk
• Google Keep item: Milk

Shop With can display one merged presentation group for milk, but internally it must retain:

• the recipe/manual Blackcap item identity
• the external item identity
• the external connection/list/item IDs
• the local link row state

If the user marks the merged group Got It, both child items can become checked locally. The external child still waits for Blackcap shopping-list completion before remote check-off.

Already Have It should be treated as locally resolved for the active Blackcap list and should be eligible for remote check-off at list completion, because the household need has been resolved. If that later feels too aggressive, make it an organization setting before changing behavior globally.

The foundation helper external_shopping_merge_service.build_shop_with_merge_groups() implements conservative presentation-only matching by normalized grocery need. It does not write to the DB and does not complete remote items.

Conflict handling

Completed externally before Blackcap completion

On next sync, mark the external item as externally completed. In the active Blackcap list, show it as completed/unavailable with a provider badge and do not attempt another remote check-off.

Renamed externally

If the external item ID is stable, update the displayed title on sync and preserve local blackcap_checked state.

Deleted externally

Mark the item unavailable/removed on sync. Do not fail the shopping list. If it was already checked locally, show a warning only if Blackcap later cannot confirm remote completion.

Remote update failure

Complete the Blackcap list, retain sync_status=failed, retain sync_error, and surface a warning. A future retry action can attempt only failed external updates.

Service architecture

Recommended modules:

route/blueprint
  -> external_shopping_connection_service
  -> external_shopping_list_service
  -> provider adapter
  -> repository/database service

Provider adapter interface:

list_available_lists()
get_default_shopping_list()
sync_open_items(list_id)
complete_item(list_id, item_id)
refresh_token()
disconnect()

Foundation modules added in this pass:

• external_shopping_provider_service.py: feasibility-aware provider registry, labels, badges, logos
• external_shopping_merge_service.py: non-destructive Shop With presentation merge helpers

Future provider modules should be separate, such as:

• google_keep_shopping_list_provider.py
• alexa_shopping_list_provider.py only if Amazon adds a safe official API again
• external_shopping_list_db_service.py
• external_shopping_connection_service.py

UI plan

Connections

When a provider becomes live-safe, add a connection card/row with:

• provider logo
• connected/disconnected status
• connected account
• selected list
• last sync
• last error
• Connect / Select List / Sync Now / Disconnect

Blocked providers should not expose broken Connect buttons. A System Admin-only feasibility note is acceptable, but normal org admins should not see half-built flows.

Shopping list

Add a Household List section with rows containing:

• checkbox
• provider logo
• item title
• provider badge
• optional selected-list tooltip
• sync status only when noteworthy

Completion

Completion confirmation should include concise copy:

Checked Household List items will be checked off in Google/Amazon.
Unchecked Household List items will stay open there.

Audit logging

Audit these events without secrets:

• provider connection created/connected/disconnected
• selected external list changed
• manual sync started/completed/failed
• external items imported/synced
• remote check-off attempted/succeeded/failed
• Blackcap shopping-list completion attempted provider updates

Data retention

Recommended retention:

• open external sync rows: retain while active/open
• link rows: retain with completed shopping-list detail retention
• sync run detail/errors: retain with background job/error retention
• raw provider metadata: minimal and purged with sync details

External shopping integration validation

• Hidden provider types do not appear as live Add Connection options.
• Provider registry returns Amazon/Google logos.
• Existing Connections UI still shows Shop With/provider icons.
• External merge helper groups milk with 1 cup milk without losing either item ID.
• Existing shopping lists, manual items, recipe lists, Shop With providers, Chrome extension flows, and Kitchen Inventory remain unchanged.
• No raw provider secrets are logged or exposed.
• No provider-specific logic is added to inky_admin_app.py.