Browse documentation

🧭 Start Here

Blackcap Overview ✨ Blackcap Feature Catalog 🤖 AI Features 🧱 Technology, Administration, and Reliability Installation First Run

📘 User Guides

Administration and Operations Platform Configuration Organization Configuration Email and Consent Management

🚀 Deploy Blackcap

Platform Stacks and Raspberry Pi Hardware Raspberry Pi Deployment Raspberry Pi Client Services GCP Deployment Packaged Blackcap deployment Application Updates After Your First Blackcap Login: Self-Hosted Setup Guide Blackcap Release Notes Environment Variables and Secrets Reverse Proxy and TLS Background Jobs and Schedules

🛠️ Administer Blackcap

Organizations Users, Permissions, and Authentication Configuration Workspace Backups and Restore Database Administration Regression Testing Performance and Job Status Audit, Access Activity, and Logging GeoIP and Access Location Data Retention and Purge Support Chat Administration Support Requests API Tester and Postman Instance Reporting Third-Party Packages and Attribution AI Provider Usage and Billing Portals Microwave Pie licensing for Blackcap Microwave Pie licensing for Blackcap

🍽️ Use Recipes

Recipes and the Recipe Library Recipe Import and Discovery Recipe Editing and Cache Artifacts Recipe Sharing Social Recipe Import AI Recipe Image Generation 🪄 Adapt Recipe and 🎲 Variations

📅 Plan Meals

Meal Planner

🛒 Use Shopping Lists

Shopping Lists and Shop a List External and Household Shopping 🔗 Chrome Extension Shop With

🧺 Manage Kitchen Inventory

🧺 Kitchen Inventory

🖥️ Use Displays

Displays and Connections Assigning and Scheduling Display Content Remote Pi Client E-Ink Rendering Menu Refresh and Rendering Noun Project Footer Images

🧑‍🍳 Cook with Let’s Cook

🧑‍🍳 Let’s Cook 🧑‍🍳 Let’s Cook Controls and Timers

🤖 Use and Administer AI

🤖 AI in Blackcap 🤖 AI Providers and Connections 🤖 AI Seeds and Usage Support Chat

🔗 Use the Chrome Extension

🔗 Blackcap Chrome Extension 🔗 Chrome Extension Recipe Capture 🔗 Chrome Extension Shop With 🔗 Chrome Extension Release and Privacy

🎮 Play Games

Games and trivia Bramble Harvest — Player Guide Flavor Lab — Player Guide

🔌 Integrations

Email Integration Cloud Storage Integrations Voice Assistants Shop With Integrations Authentication Providers

⚙️ Develop Blackcap

Application Architecture Database Service and Data Access SQLite and PostgreSQL Database Migrations Background Job Architecture Testing Games platform — current developer reference Bramble Harvest developer tools API Architecture Security and Organization Scoping UI, Icons, and Documentation Assets Blackcap-Safe Emoji Support Chat Architecture Extending Support Chat Support Chat Model Benchmark Documentation Standards Terminology

⚠️ Troubleshoot Blackcap

⚠️ Troubleshooting Licensing Troubleshooting Deployment Troubleshooting Display Troubleshooting Recipe Import Troubleshooting 🤖 AI Troubleshooting Backup Troubleshooting Database Troubleshooting Diagnostic Organization Clones Support Chat

Reverse Proxy and TLS

Audience: Deployment Administrator Related: Raspberry Pi · Gcp · Environment And Secrets

Raspberry Pi and GCP deployments use different front ends. The Pi deployment script recommends stunnel because it is lightweight and compatible with the established Gunicorn PROXY-protocol configuration. GCP uses Nginx as the HTTPS-only reverse proxy and Certbot for browser-trusted certificates.

Keep Gunicorn bound to localhost when a local TLS front end is enabled. Validate the proxy configuration, the forwarded client address behavior, the external base URL, and certificate renewal after infrastructure changes.

Raspberry Pi stunnel

For Raspberry Pi deployments, Blackcap should continue to use stunnel rather than moving to Nginx by default.

Browser / LAN / Alexa → HTTPS → stunnel on the Pi → Gunicorn 127.0.0.1:8080 → Blackcap

Decision: stay with stunnel on the Pi

Nginx is the preferred GCP front end, but stunnel remains the better default for the Pi because it is lighter, matches the existing Blackcap Pi deployment pattern, and works with Gunicorn's PROXY protocol mode so Blackcap can preserve real client IPs. Nginx is still a valid advanced option, but it should not be the standard Pi path unless a future Pi deployment needs more reverse-proxy features.

Script behavior

tools/deploy_pi.sh installs stunnel4 and openssl when the base Linux package step is accepted.

The script asks:

Use Gunicorn PROXY protocol for local stunnel/TLS forwarding? [Y]
Configure stunnel as the Pi TLS front-end? Recommended over Nginx for the Pi [Y]
Public stunnel HTTPS port [8443]
stunnel combined PEM certificate path [/etc/stunnel/blackcap.pem]

When stunnel is enabled, Gunicorn is bound to:

127.0.0.1:8080

The script writes:

/etc/stunnel/blackcap.conf

and enables/restarts stunnel4.

Certificate behavior

If the configured combined PEM certificate does not exist, the script creates a self-signed certificate. This is acceptable for private LAN testing. For public HTTPS, replace /etc/stunnel/blackcap.pem with a trusted certificate and private key combined into one PEM file.

A combined PEM contains the private key and certificate in the order required by stunnel. Keep the file root-protected and never place its contents in documentation or logs.

Expected stunnel config

The generated config is equivalent to:

[blackcap]
accept = 0.0.0.0:8443
connect = 127.0.0.1:8080
cert = /etc/stunnel/blackcap.pem
TIMEOUTclose = 0
protocol = proxy

The protocol = proxy line matches the Gunicorn service's PROXY protocol mode. If you disable PROXY protocol in the deploy script, do not use the generated stunnel PROXY protocol line.

Verification

sudo systemctl status inky_admin.service --no-pager
sudo systemctl status stunnel4 --no-pager
sudo ss -ltnp | egrep ':443|:8443|:8080'

Expected listener pattern:

0.0.0.0:8443      stunnel
127.0.0.1:8080    gunicorn

If using port 443 for Alexa account linking, the expected stunnel listener is 0.0.0.0:443 instead of 8443.

GCP Nginx and Certbot

Blackcap's direct GCP deployment uses this trust boundary:

Internet client → Nginx :443 → Gunicorn 127.0.0.1:8080 → Blackcap

Gunicorn remains bound to loopback. The GCP firewall exposes TCP 443 only; TCP 80 is not required by this deployment pattern.

Why DNS-01 is used

Let's Encrypt HTTP-01 validation requires inbound TCP 80. Blackcap's HTTPS-only GCP setup instead uses automated DNS-01 validation. Certbot proves control of the hostname by creating and removing an _acme-challenge TXT record through the DNS provider's API.

The focused setup supports three automation modes:

  • godaddy: native Blackcap hooks that use GoDaddy's Domains API with a scoped Personal Access Token (PAT). This is the default for the current Blackcap deployment.
  • google: Google Cloud DNS through Certbot's dns-google plugin.
  • hooks: provider-specific executable authentication and cleanup hooks for other DNS providers.

Interactive manual DNS entry is not considered auto-renewal and is not used by this setup.

Configure an existing GCP instance

From the current checkout:

cd /opt/Blackcap
sudo ./tools/configure_gcp_frontend.sh

The script:

  • installs Nginx and Certbot when missing;
  • installs python3-certbot-dns-google when Google Cloud DNS mode is selected;
  • installs root-owned GoDaddy DNS credentials and Certbot hook wrappers when GoDaddy mode is selected;
  • installs the non-secret certificate-status helper and, for GoDaddy, a narrowly scoped root-owned credential-rotation worker;
  • obtains or converts the hostname certificate to unattended DNS-01 renewal;
  • writes an Nginx site that listens on TCP 443 only;
  • enables the package-provided certbot.timer, or installs a Blackcap fallback timer when the package has no timer;
  • installs /etc/letsencrypt/renewal-hooks/deploy/blackcap-nginx-reload, which also refreshes Blackcap's certificate-expiration/issuer metadata after successful renewals;
  • records provider, timer, certificate, and non-secret credential metadata in /var/lib/blackcap/certificates/public-access.json;
  • writes trusted-local-proxy settings to /etc/blackcap/blackcap.env; and
  • restarts Blackcap so host-level proxy settings take effect.

GCP firewall requirements

Allow inbound TCP:

  • 443 for Blackcap HTTPS traffic.

Do not expose:

  • 80; DNS-01 validation does not use it.
  • 8080; Gunicorn listens only on loopback.
  • 5432; the Cloud SQL Auth Proxy listens only on loopback.

Certbot needs outbound HTTPS access to the ACME service, and the selected DNS automation method must be able to update the authoritative DNS zone.

GoDaddy DNS automation

Choose godaddy when the hostname's authoritative DNS records are managed in GoDaddy. Before running the setup, create a GoDaddy Personal Access Token (PAT) for the same account that controls the domain. Grant only domains.domain:read and domains.dns:update. GoDaddy PATs use Authorization: Bearer <token> and are the recommended Domains/DNS authentication method; legacy sso-key API key/secret credentials are retained only for migration compatibility in the hook.

The setup prompts for:

GoDaddy-managed root domain
Root-only GoDaddy credentials file
GoDaddy Personal Access Token (domains.domain:read + domains.dns:update)
GoDaddy PAT expiration date (YYYY-MM-DD; leave blank if not recorded)
DNS propagation wait in seconds

For a hostname such as blackcap.microwavepie.com, the GoDaddy-managed root domain is normally:

microwavepie.com

Credentials are stored outside the repository and outside Blackcap's application environment:

/etc/blackcap/godaddy-dns.ini

The file is owned by root and mode 0600. The PAT is never written to /etc/blackcap/blackcap.env, command-line arguments, Nginx configuration, Blackcap settings/status metadata, or the PostgreSQL database. The optional PAT expiration date is non-secret metadata and is recorded so Blackcap can assess future renewal risk.

Blackcap installs these Certbot hook commands:

/usr/local/sbin/blackcap-godaddy-dns-auth
/usr/local/sbin/blackcap-godaddy-dns-cleanup

The hooks use GoDaddy's Domains API with Bearer-token authentication to read the existing TXT records, add only the current Let's Encrypt value, and remove only that same value after validation. This preserves unrelated TXT values at the same _acme-challenge record name. The hook waits for the configured propagation period before Certbot asks Let's Encrypt to validate the record.

Verify API access without changing DNS:

sudo /usr/bin/python3 /opt/Blackcap/tools/godaddy_dns_hook.py verify \
  --credentials /etc/blackcap/godaddy-dns.ini

Inspect the saved Certbot renewal hooks:

sudo grep -E \
  '^[[:space:]]*(authenticator|manual_auth_hook|manual_cleanup_hook)[[:space:]]*=' \
  /etc/letsencrypt/renewal/<hostname>.conf

Expected GoDaddy renewal configuration:

authenticator = manual
manual_auth_hook = /usr/local/sbin/blackcap-godaddy-dns-auth
manual_cleanup_hook = /usr/local/sbin/blackcap-godaddy-dns-cleanup

A manual authenticator is unattended when both saved hook commands are present. A plain interactive manual certificate without hooks is not renewable automatically.

Google Cloud DNS permissions

For google mode, Certbot uses Application Default Credentials by default. On a Compute Engine VM, that normally means the VM's attached service account.

The service account must be allowed to discover the relevant managed zone and create/delete challenge TXT records. Prefer zone-scoped, least-privilege roles. The setup also accepts a service-account JSON file, but the VM service account is preferable because it avoids a long-lived key file on disk.

The script prompts for:

Google Cloud project containing the public DNS zone
Google DNS service-account JSON path; leave blank to use the VM service account
DNS propagation wait in seconds

If the public zone is in another project, enter that DNS project rather than the Compute Engine project. Blackcap intentionally does not assign an artificial expiration date to VM service-account/ADC authorization. The operational dependency is IAM: the service account must remain enabled and keep the required Cloud DNS permissions. Explicit service-account JSON credentials are identified separately in the status UI.

Non-Google DNS providers

Choose hooks mode and provide:

Executable DNS-01 authentication hook
Executable DNS-01 cleanup hook
DNS provider display name
DNS credential/authentication description
DNS credential expiration date (YYYY-MM-DD; leave blank if not applicable)

The authentication hook must create the requested TXT record and wait until it is resolvable before returning successfully. The cleanup hook must remove the record. Certbot stores these hook paths in the renewal configuration, allowing the scheduled renewal command to run unattended.

Protect API tokens used by hook scripts. Keep credential files root-owned and mode 0600.

Blackcap certificate and DNS status

After configure_gcp_frontend.sh completes, System Admins can use Platform → Certificates & Public Access and System → Certificates & DNS on package-managed or source installations. The configuration page controls the certificate-renewal warning window (default 14 days); the System panel shows hostname, provider, certificate expiration/issuer, renewal timer, credential type/status, and provider-specific metadata. Private installations simply report that public HTTPS/DNS automation is not configured.

The global warning is intentionally certificate-driven. An expired GoDaddy PAT or custom-provider credential is shown in the panel immediately, but it does not replace a normal Blackcap update notice until the current HTTPS certificate enters the configured warning window and Blackcap knows unattended renewal cannot succeed. While that renewal-risk warning is active, it has higher priority than the update notice. Successful certificate renewals refresh the recorded expiration automatically.

For GoDaddy, Platform → Certificates & Public Access can rotate the PAT or update only its expiration date. The web process never receives permission to edit arbitrary root files: it writes a unique, tightly scoped request for a root systemd worker. A replacement PAT is verified against the GoDaddy Domains API before the root-owned credentials file is replaced. PAT-bearing requests are removed after processing, stale requests are cleaned up, and audit/result records contain only non-secret facts. If an existing host predates this status feature, Blackcap first discovers the existing environment/Certbot configuration directly. Run sudo ./tools/configure_gcp_frontend.sh --register-existing once to write durable status metadata, refresh the renewal deploy hook, and install the GoDaddy worker when applicable. This registration mode does not issue or reissue a certificate and does not change DNS secrets.

For Google Cloud DNS, the page displays the DNS project and whether authentication uses VM service-account/ADC or an explicit JSON credential. For custom hooks, Blackcap displays the configured provider name, authentication description, and optional credential expiration; secret rotation remains the hook/provider's responsibility.

Client IP trust model

The Nginx site deliberately overwrites incoming forwarding headers:

proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header Forwarded "";

Using $remote_addr instead of $proxy_add_x_forwarded_for prevents an Internet client from injecting a fake first address into the forwarded chain. Blackcap accepts those headers only when the immediate socket peer is in:

BLACKCAP_TRUSTED_PROXY_HEADERS_ENABLED=1
BLACKCAP_TRUSTED_PROXY_CIDRS=127.0.0.1/32, ::1/128
BLACKCAP_ENFORCE_HTTPS_EXTERNAL=1
BLACKCAP_TRUST_EXTERNAL_BASE_URL_HTTPS=1

These environment settings are applied after DB-backed configuration because the proxy topology belongs to the host, not to the shared PostgreSQL database.

Renewal behavior

The Debian/Ubuntu Certbot package normally provides certbot.timer. Blackcap enables it when present. Successful renewals execute:

/usr/sbin/nginx -t
/bin/systemctl reload nginx

The reload happens only after Nginx configuration validation succeeds.

Check the schedule and certificate:

sudo systemctl list-timers --all | grep -E 'certbot|blackcap-certbot'
sudo certbot certificates
sudo certbot renew --dry-run

Inspect the renewal authenticator:

sudo grep -E '^[[:space:]]*(authenticator|manual_auth_hook|manual_cleanup_hook)[[:space:]]*=' \
  /etc/letsencrypt/renewal/<hostname>.conf

For GoDaddy, the authenticator is manual with both Blackcap GoDaddy hook paths saved. For Google Cloud DNS, the authenticator should be:

authenticator = dns-google

For custom hooks, it should be manual and include both saved hook paths. A plain interactive manual certificate without hooks is not unattended.

Verify HTTPS-only exposure

sudo nginx -t
sudo systemctl status nginx --no-pager
sudo ss -ltnp | egrep ':80|:443|:8080'

Expected:

  • Nginx listens on 0.0.0.0:443 and/or [::]:443.
  • Nothing listens publicly on port 80.
  • Gunicorn listens on 127.0.0.1:8080.

The GCP firewall should also contain no inbound HTTP rule for this VM.

Validate the real client address

After an external browser request, use Blackcap's access activity or GeoIP diagnostics. The request should show the public client IP rather than 127.0.0.1.

You can also inspect the Nginx access log:

sudo tail -n 50 /var/log/nginx/access.log

Blackcap's security and access services use the validated client address for activity records, GeoIP lookup, rate-limit keys, and local-versus-external request decisions.

Troubleshooting DNS-01

If issuance or the renewal dry run fails:

  1. Confirm the hostname's public DNS zone is managed by the provider selected in the script.

  2. For GoDaddy, verify /etc/blackcap/godaddy-dns.ini is root-owned, mode 0600, contains a Personal Access Token with domains.domain:read and domains.dns:update, and names the correct root domain. Run the GoDaddy hook's verify command shown above.

  3. For Google Cloud DNS, confirm the VM service account can list the managed zone and create/delete record changes.

  4. Confirm the Certbot plugin is installed when using Google Cloud DNS:

    sudo certbot plugins | grep -E 'dns-google|dns_google'
    
  5. Confirm the public authoritative nameservers can resolve TXT records under _acme-challenge.<hostname>.

  6. Increase BLACKCAP_CERTBOT_DNS_PROPAGATION_SECONDS when the DNS provider publishes changes slowly.

  7. Inspect Certbot logs:

    sudo journalctl -u certbot.service -n 100 --no-pager
    sudo tail -n 100 /var/log/letsencrypt/letsencrypt.log
    
  8. Validate Nginx separately:

    sudo nginx -t
    sudo systemctl status nginx --no-pager
    

Real client IP inside Flask and Gunicorn logs

For the direct GCP topology, Nginx is the only proxy and Gunicorn remains bound to 127.0.0.1:8080. The deployment environment enables trusted proxy handling only for loopback CIDRs. Blackcap applies Werkzeug ProxyFix for exactly one proxy hop so Flask's standard request.remote_addr, scheme, host, and port use the values Nginx overwrites.

Gunicorn's default %(h)s access-log token intentionally reports its socket peer, which is local Nginx. The repository gunicorn.conf.py switches the GCP access log to the Nginx-provided X-Forwarded-For value when the loopback-only trusted proxy environment is enabled. No systemd ExecStart change is required; Gunicorn loads gunicorn.conf.py from /opt/Blackcap because that is the service working directory.

After updating these files, restart Blackcap:

sudo systemctl restart inky_admin.service

New access lines should begin with the client's public address rather than 127.0.0.1. Existing Access Activity rows are not rewritten.

On this page