Analyze PR Performance

Analyze code review pipeline performance for a specific PR.

Usage

Run the analyze-pr-performance CLI script with the provided arguments:

npx ts-node scripts/analyze-pr-performance.cli.ts $ARGUMENTS

Arguments

• •prNumber (required): The PR number to analyze
• •orgId (required): The organization ID

Options

• •--days=N: Number of days to search back (default: 7)
• •--legacy: Also search in legacy collection (observability_logs)
• •--env=PATH: Path to .env file (e.g., --env=.env.prod)

Examples

# Analyze performance for PR #558 in production
/analyze-pr-performance 558 04bd288b-595a-4ee1-87cd-8bbbdc312b3c --env=.env.prod

# Analyze with extended date range
/analyze-pr-performance 723 97442318-9d2a-496b-a0d2-b45fb --days=14 --env=.env.prod

# Analyze with legacy logs included
/analyze-pr-performance 701 97442318-9d2a-496b-a0d2-b45fb --legacy --env=.env.prod

What it analyzes

1. •

   Pipeline identification: Finds the pipelineId and correlationId for the PR

2. •

   Stage times: Shows duration of each pipeline stage:

   • •ValidateNewCommitsStage
   • •ResolveConfigStage
   • •FetchChangedFilesStage
   • •PRLevelReviewStage
   • •FileAnalysisStage
   • •CreateFileCommentsStage
   • •UpdateCommentsAndGenerateSummaryStage
   • •And all other stages...
3. •

   LLM calls: Details of each LLM operation:

   • •Operation name (analyzeCodeWithAI, selectReviewMode, kodyRulesAnalyzeCodeWithAI, etc.)
   • •Duration
   • •Model used
   • •Token counts (input/output)
4. •

   Summary metrics:

   • •Total pipeline duration
   • •Total LLM calls count
   • •Total tokens (input/output)
   • •Slow calls count (> 60s)
   • •Models used
5. •

   Bottlenecks: Highlights stages and LLM calls taking > 60 seconds

6. •

   Pipeline status: Whether the pipeline completed, failed, or is unknown

Output

The script outputs:

• •Pipeline and correlation IDs
• •Organization and repository info
• •Stage times table with duration and percentage of total
• •LLM calls table with model, tokens, and duration
• •Summary metrics
• •Bottleneck list (stages and LLM calls > 60s)
• •Final pipeline status

How to Respond

• •Identify the slowest stages and explain why they might be slow
• •Look for LLM calls that are taking too long (> 2 minutes is concerning)
• •Check if multiple slow LLM calls are running sequentially vs in parallel
• •Note any patterns (e.g., all selectReviewMode calls are slow = possible model issue)
• •Suggest potential optimizations if bottlenecks are clear
• •If FileAnalysisStage is slow, it's usually due to many files or large files being analyzed
• •If PRLevelReviewStage is slow, check the KodyRules and PR-level analysis calls
• •Compare token counts to durations - high tokens with proportional time is expected, low tokens with high time indicates API latency issues
• •Report the pipeline status and check if it completed successfully