BuildBuddy Invocation Troubleshoot
Overview
Use this skill to investigate a BuildBuddy invocation end-to-end: locate the invocation, identify failing targets, collect stdout/stderr, determine whether RBE or remote cache was involved, and download build artifacts (raw BES, build logs, execution profiles, cache scorecard).
Workflow (recommended)
0) Preflight: auth + base URL
- •Avoid accessing the API key value directly. Check presence by piping to
wcand treating any non-empty value as valid, e.g.git config --local buildbuddy.api-key | wc -c. - •If the API key is missing (empty output), ask the user to run
bb loginand retry. - •Confirm the base URL (default
https://app.buildbuddy.iounless self-hosted). - •Collect
group_idand (if already known)invocation_id. - •Create a temp working directory (per investigation) and store all downloaded artifacts and API responses there. Reuse cached files to avoid re-calling the API; delete a cached file or set a force flag when you want a fresh response.
Reference request templates live in references/requests.md.
1) Find the invocation ID
- •Use
SearchInvocationwith a time window (updated_after/updated_before), group ID, and optional filters (repo, branch, user, status, command). - •Sort by
UPDATED_AT_USEC_SORT_FIELDdescending to find the latest runs. - •If the user already has an invocation URL, extract the
invocation_idfrom it and skip search.
2) Fetch invocation metadata
- •Call
GetInvocationwithinvocation_idto retrieve:- •
console_buffer(fastest way to see the failure banner) - •
remote_execution_enabled,upload_local_results_enabled,download_outputs_option - •
cache_statsandscore_card(if remote cache was used) - •
structured_command_lineto detect flags (remote executor, BES, etc.) - •
invocation_statusto confirm completeness
- •
- •Note and report the git commit hash, branch name, OS, and hostname from the invocation metadata. If these match the local environment, reproduction may be feasible.
If invocation_status is not complete, expect partial logs and missing artifacts.
3) Identify failing targets + stdout/stderr
- •Use
GetTargetfor the invocation.- •Filter by
status=FAILEDorfilterto narrow labels. - •Focus on
root_causetargets when present.
- •Filter by
- •For each failing target, inspect
action_events(ActionExecuted) for:- •
stdout/stderrfile descriptors - •
exit_code,command_line,failure_detail
- •
- •Download target logs using the file
uri(if present) or by passing thebytestream_urlto/file/download.
4) If RBE is involved, drill into executions
- •When
remote_execution_enabled=true, useGetExecution(byinvocation_id) orSearchExecutionto locate the execution. - •For execution details:
- •
execute_response(useinline_execute_response=true) contains ActionResult and stdout/stderr digests. - •
executed_action_metadataprovides queue/execute timestamps. - •
execution_id,target_label,action_mnemonic, andcommand_snippethelp correlate to the failing target.
- •
- •If multiple executions failed but later retries succeeded, filter
SearchExecutionby the exact failing target label (fromGetTarget) or byprimary_outputto avoid unrelated transient failures. A single target label can map to many actions, soprimary_outputis often the best discriminator. - •Download the execution profile (if enabled) via
/file/download?artifact=execution_profile&invocation_id=...&execution_id=....
5) Download build events / build log / Bazel profile
- •Raw BES JSON/Proto:
- •
/file/download?artifact=raw_json&invocation_id=... - •
/file/download?artifact=raw_proto&invocation_id=...
- •
- •Build log (BES text chunks):
- •
/file/download?artifact=buildlog&invocation_id=...&attempt=...
- •
- •Bazel profile:
- •Usually attached as an artifact file in BES. Find it from invocation/target files, then download via
/file/downloadwith its bytestream URL or directuri.
- •Usually attached as an artifact file in BES. Find it from invocation/target files, then download via
6) Cache scorecard (server-side cache requests)
- •Use
GetCacheScoreCardwithinvocation_id. - •Apply filters for request/response type and search for a specific target/action mnemonic.
- •Use
order_byto surface slow or large transfers.
Tips and safety
- •Avoid printing API keys in outputs. Redact before sharing.
- •When making requests, use shell expansion like
$(git config --local buildbuddy.api-key)instead of pasting the API key directly. - •Prefer smaller time windows in searches to avoid large result sets.
- •For JSON mapping, proto fields are lowerCamelCase (e.g.,
requestContext,groupId). - •When unsure about fields, check these protos:
- •
proto/buildbuddy_service.proto - •
proto/invocation.proto - •
proto/target.proto - •
proto/build_event_stream.proto - •
proto/execution_stats.proto - •
proto/cache.proto - •
proto/eventlog.proto
- •