Background AI system

TOW runs AI work through configurable Agents and durable Agent Runs.

Runtime shape

The backend starts background AI work from the FastAPI lifespan in backend/app/main.py.

1. Built-in Agents are seeded or refreshed.
2. agent_run_queue.start() starts Agent Run workers.
3. If operating_background_enabled is true, OperatingBackgroundRunner.run_forever() checks scheduled operating work.
4. User actions such as check-ins, doc edits, chat turns, snapshots, roadmap generation, and ticket scans enqueue Agent Runs.
5. Workers load the Agent's published version and run the generic Agent runner. Built-in platform workflows can delegate to deterministic service handlers after the Agent gate is resolved.

Agent Runs are database records, so queued, running, completed, and failed status is auditable across backend restarts.

Status flow

Frontend status comes from GET /api/agent-runs. The frontend polls active queued and running runs for the signed-in user and shows recent failure counts.

Item responses that start background work include agent_runs for immediate UI indicators.

Agent gating

Agent availability resolves through server, organisation, project, and user bindings. Any disabled binding disables the Agent. Schedules come from the most specific configured binding, then the Agent version trigger defaults.

Direct AI endpoints check the effective Agent before queuing work. Mixed actions save non-AI data when appropriate and skip the disabled AI follow-up.

See Agent configuration for the admin-facing model and built-in Agent catalog.

Agent Run catalog

Agent key	Typical trigger	Target	Output
proactive_daily_review	Scheduled daily review.	daily_session	Updates proactive-review sessions and Inbox proposals.
operating_pulse	Scheduled interval pulse.	daily_session	Updates Today pulse context.
today_checkin_processor	Project Check-In submitted.	daily_session	Coaching, memory diffs, ticket automation, and proposals.
roadmap_proposal_generator	Roadmap generation request.	daily_session	Roadmap and epic proposals.
docs_memory_reviewer	Doc create/update/archive/manual extract.	doc	Doc indexing, references, embeddings, and memory diffs.
snapshot_generator	Snapshot generation request.	snapshot	Snapshot draft for review.
ticket_conflict_scanner	Manual ticket conflict scan and scheduled daily scan.	ticket_conflicts, daily_session	Duplicate/conflict proposals and nightly conflict scan summaries.
chat_memory_reviewer	Chat turn completed.	raw_event	Pending memory diffs.

Operational controls

Setting	Purpose
operating_background_enabled	Enables or disables the periodic operating background runner.
operating_background_interval_seconds	Controls how often the background runner scans for scheduled Agent work.
ticket_conflict_scan_hour_utc	First UTC hour when the nightly conflict scan may be queued.
ai_queue_max_concurrency	Maximum number of Agent Runs processed concurrently.
ai_queue_status_retention_seconds	Status retention window used by polling clients.
OPENAI_API_KEY	Required for OpenAI Responses and embedding calls.
openai_reasoning_model	Default reasoning model for Agent work.
openai_fast_model	Default model for structured JSON helper calls.
openai_embedding_model	Embedding model used for semantic retrieval and doc indexing.
EXA_API_KEY	Enables configured web search and scrape tools.

Troubleshooting

Symptom	What to check
Runs stay queued	Confirm the backend lifespan started and Agent Run workers are running. Check ai_queue_max_concurrency.
A run failed	Check /api/agent-runs, backend logs, and the persisted agent_runs.error value.
AI action is disabled	Check the effective Agent row in Server, Organisation, Project, or User Settings.
Expected output is missing	Check Agent Runs, Inbox, Memory Diffs, Today, snapshots, tickets, or roadmap proposals.
Nightly scans never run	Confirm background mode, user membership, UTC hour, and the Agent's effective daily schedule.