Troubleshooting

Gateway troubleshooting

This page is the deep runbook. Start at /help/troubleshooting if you want the fast triage flow first.

Command ladder

Run these first, in this order:

builderforce status
builderforce gateway status
builderforce logs --follow
builderforce doctor
builderforce channels status --probe

Expected healthy signals:

builderforce gateway status shows Runtime: running and RPC probe: ok.
builderforce doctor reports no blocking config/service issues.
builderforce channels status --probe shows connected/ready channels.

No replies

If channels are up but nothing answers, check routing and policy before reconnecting anything.

builderforce status
builderforce channels status --probe
builderforce pairing list <channel>
builderforce config get channels
builderforce logs --follow

Look for:

Pairing pending for DM senders.
Group mention gating (requireMention, mentionPatterns).
Channel/group allowlist mismatches.

Common signatures:

drop guild message (mention required → group message ignored until mention.
pairing request → sender needs approval.
blocked / allowlist → sender/channel was filtered by policy.

Dashboard control ui connectivity

When dashboard/control UI will not connect, validate URL, auth mode, and secure context assumptions.

builderforce gateway status
builderforce status
builderforce logs --follow
builderforce doctor
builderforce gateway status --json

Look for:

Correct probe URL and dashboard URL.
Auth mode/token mismatch between client and gateway.
HTTP usage where device identity is required.

Common signatures:

device identity required → non-secure context or missing device auth.
unauthorized / reconnect loop → token/password mismatch.
gateway connect failed: → wrong host/port/url target.

Gateway service not running

Use this when service is installed but process does not stay up.

builderforce gateway status
builderforce status
builderforce logs --follow
builderforce doctor
builderforce gateway status --deep

Look for:

Runtime: stopped with exit hints.
Service config mismatch (Config (cli) vs Config (service)).
Port/listener conflicts.

Common signatures:

Gateway start blocked: set gateway.mode=local → local gateway mode is not enabled. Fix: set gateway.mode="local" in your config (or run builderforce configure). If you are running BuilderForce Agents via Podman using the dedicated builderforce user, the config lives at ~builderforce/.builderforce/builderforce.json.
refusing to bind gateway ... without auth → non-loopback bind without token/password.
another gateway instance is already listening / EADDRINUSE → port conflict.

Channel connected messages not flowing

If channel state is connected but message flow is dead, focus on policy, permissions, and channel specific delivery rules.

builderforce channels status --probe
builderforce pairing list <channel>
builderforce status --deep
builderforce logs --follow
builderforce config get channels

Look for:

DM policy (pairing, allowlist, open, disabled).
Group allowlist and mention requirements.
Missing channel API permissions/scopes.

Common signatures:

mention required → message ignored by group mention policy.
pairing / pending approval traces → sender is not approved.
missing_scope, not_in_channel, Forbidden, 401/403 → channel auth/permissions issue.

Cron and heartbeat delivery

If cron or heartbeat did not run or did not deliver, verify scheduler state first, then delivery target.

builderforce cron status
builderforce cron list
builderforce cron runs --id <jobId> --limit 20
builderforce system heartbeat last
builderforce logs --follow

Look for:

Cron enabled and next wake present.
Job run history status (ok, skipped, error).
Heartbeat skip reasons (quiet-hours, requests-in-flight, alerts-disabled).

Common signatures:

cron: scheduler disabled; jobs will not run automatically → cron disabled.
cron: timer tick failed → scheduler tick failed; check file/log/runtime errors.
heartbeat skipped with reason=quiet-hours → outside active hours window.
heartbeat: unknown accountId → invalid account id for heartbeat delivery target.

Node paired tool fails

If a node is paired but tools fail, isolate foreground, permission, and approval state.

builderforce nodes status
builderforce nodes describe --node <idOrNameOrIp>
builderforce approvals get --node <idOrNameOrIp>
builderforce logs --follow
builderforce status

Look for:

Node online with expected capabilities.
OS permission grants for camera/mic/location/screen.
Exec approvals and allowlist state.

Common signatures:

NODE_BACKGROUND_UNAVAILABLE → node app must be in foreground.
*_PERMISSION_REQUIRED / LOCATION_PERMISSION_REQUIRED → missing OS permission.
SYSTEM_RUN_DENIED: approval required → exec approval pending.
SYSTEM_RUN_DENIED: allowlist miss → command blocked by allowlist.

Browser tool fails

Use this when browser tool actions fail even though the gateway itself is healthy.

builderforce browser status
builderforce browser start --browser-profile builderforce
builderforce browser profiles
builderforce logs --follow
builderforce doctor

Look for:

Valid browser executable path.
CDP profile reachability.
Extension relay tab attachment for profile="chrome".

Common signatures:

Failed to start Chrome CDP on port → browser process failed to launch.
browser.executablePath not found → configured path is invalid.
Chrome extension relay is running, but no tab is connected → extension relay not attached.
Browser attachOnly is enabled ... not reachable → attach-only profile has no reachable target.

If you upgraded and something suddenly broke

Most post-upgrade breakage is config drift or stricter defaults now being enforced.

1) Auth and URL override behavior changed

builderforce gateway status
builderforce config get gateway.mode
builderforce config get gateway.remote.url
builderforce config get gateway.auth.mode

What to check:

If gateway.mode=remote, CLI calls may be targeting remote while your local service is fine.
Explicit --url calls do not fall back to stored credentials.

Common signatures:

gateway connect failed: → wrong URL target.
unauthorized → endpoint reachable but wrong auth.

2) Bind and auth guardrails are stricter

builderforce config get gateway.bind
builderforce config get gateway.auth.token
builderforce gateway status
builderforce logs --follow

What to check:

Non-loopback binds (lan, tailnet, custom) need auth configured.
Old keys like gateway.token do not replace gateway.auth.token.

Common signatures:

refusing to bind gateway ... without auth → bind+auth mismatch.
RPC probe: failed while runtime is running → gateway alive but inaccessible with current auth/url.

3) Pairing and device identity state changed

builderforce devices list
builderforce pairing list <channel>
builderforce logs --follow
builderforce doctor

What to check:

Pending device approvals for dashboard/nodes.
Pending DM pairing approvals after policy or identity changes.

Common signatures:

device identity required → device auth not satisfied.
pairing required → sender/device must be approved.

If the service config and runtime still disagree after checks, reinstall service metadata from the same profile/state directory:

builderforce gateway install --force
builderforce gateway restart