Guide

The operations page

What every panel on the operations page shows and what its buttons do: processes, backups, disk, alerts, logs, test restores, incidents, long-running queries and the rest.

5 min read

Opening itCopy link to this section

The operations page, /ops, is for whoever runs the workspace. It opens after signing in on the workspace's own sign-in page, and on a phone it can be installed as an app of its own. Actions that change something are written to the operator actions list.

HealthCopy link to this section

  • Processes: each service with its state and restart count. Warn people before a restart or restore shows signed-in people a notice first.
  • Uptime and version: how long this start has run and which build it is, with a link to the status history.
  • Database and audit trail: a database check every 30 seconds over the last hour, failures drawn in red, and audit write failures, each of which is a missing record.
  • Disk: free space for the database, the Bots' files and temporary files. A directory below 10% free and below 5 GB raises one alert until space comes back.
  • Memory per service: each service's memory every five minutes over up to 24 hours. A line that keeps climbing is worth a look.
  • Computer limits and Malware scan: what the host lets the Bots' computers be limited to, measured at start, and whether file scanning is ready with the date of its signatures.

AlertsCopy link to this section

A crash loop, low disk or failing backups raise an alert. The newest 50 are kept on the Alerts panel, where Mark all seen clears the count and Send a test alert checks the path. An alert can be snoozed for 15 minutes to 7 days. When the hosting settings name an alert webhook, each alert is also posted there, with secret values removed, in a form most chat and paging tools accept.

Backups and restoresCopy link to this section

  • Backups: every snapshot in backup storage with its note, size and read-back check. Back up now takes one at once, Download database saves a copy of the live database, Pin keeps a snapshot whatever the retention rule says, and Restore replaces the whole database with a snapshot after saving the current state first.
  • Restore preview and Restore progress: a snapshot loaded into a scratch database with its tables, rows and newest message, before you restore it; and each step while a restore runs.
  • Test restores: weekly, and whenever you select Run a test restore, the newest snapshot is loaded into a scratch database and its row counts compared with the live one. The live database is only read.
  • Compare snapshots: row counts of two snapshots, table by table, without touching anything live.
  • Backup storage and Backup checks and weekly summary: snapshot size over 90 days, the largest files, whether storage answers, and a weekly summary you can also send now.

The schedule, the retention rule and what pauses backups are in Backup and restore.

Sign-ins and securityCopy link to this section

  • Sign-ins: recent sign-in attempts with their result and client.
  • Sign-in security: change the password (at least 12 characters; every other session is signed out), Sign out everywhere, and two-factor sign-in with an authenticator app.
  • Operator actions: every action taken on this page, when, with its detail, result and client.

Traffic and logsCopy link to this section

  • Traffic: response times for the last hour by route group (median, 95th and 99th percentile, client and server errors), errors per hour over 24 hours, the slowest requests, and requests the workspace refused, timed out or cut off.
  • Logs: the services' output, one file per day with secret values replaced, kept for 1 to 90 days and downloadable. Search logs finds lines in a time range, with their service and level; Live log follows them as they are written.
  • Trace a request: look up a request id and see each hop the request took and how long. Recent API log shows the latest lines.
  • Errors in people's browsers: uncaught errors in the app since this start, grouped by message, with ids removed from page addresses.

Database and background workCopy link to this section

  • Database size and connections: the size with an alert threshold you set, open connections by application, and the largest tables and indexes.
  • Long-running queries: statements running for more than a minute, with values hidden. Cancel stops that one statement, not the connection.
  • Work queue: background work by kind: waiting, scheduled, running and failed, and the oldest waiting.
  • Stuck work: work running four times longer than its kind usually takes (at least 15 minutes). Cancelling stops it being handed out again.

Releases and settingsCopy link to this section

  • Deploy history: each start, its build, the database upgrades it applied and how long it took to be ready.
  • Post-release check: after every start the workspace loads its own key pages and records whether each answered as it should. Run now checks again.
  • Configuration: every setting the workspace reads, whether it is set, and what it does. Values are never shown, not even for settings that are not secret.
  • Maintenance mode: visitors and the app get a maintenance page with the expected end you give, while this page, sign-in, health checks and signed webhooks keep working. It ends when you turn it off.
  • Installed app worker: if an installed app's offline worker misbehaves, Switch off makes every installed copy remove it the next time it opens.

Incidents and the status pageCopy link to this section

Incidents are public: log one with its impact and the parts affected, post updates as you learn more, and resolve it. They appear on /status-history and in its feed. Uptime objective sets a monthly target, measured from the same five-minute checks for the current month in Japan time.

Site and inboundCopy link to this section

  • Inbound webhooks: the only paths outside services reach without signing in, each checked by its own signature or secret, with deliveries counted since this start. Forwarded addresses shows what the hosting platform's forwarding looks like, so the per-visitor limits count visitors correctly.
  • Enterprise waitlist and Newsletter: sign-ups from the site, downloadable as CSV.
  • Docs feedback and Site page views: answers to "Was this page helpful?", and views per page, language and linking domain, kept without cookies or visitor addresses.

Last updated