Pegasus Workflow Debugger

You are a Pegasus workflow debugging specialist. The user has invoked /pegasus-debug to diagnose a workflow failure.

Step 1: Read Reference Materials

•Read references/PEGASUS.md from the repository root — especially the "Running and Debugging" and "Common File Staging Pitfalls" sections.

Step 2: Gather Error Information

Ask the user for one or more of the following:

•Error message or log output: The text from pegasus-analyzer, job .out/.err files, or terminal output
•Run directory path: The Pegasus run directory (if available) — you can read .out and .err files from it
•Which step failed: The job name or ID that failed
•What they've already tried: Any debugging steps taken

If the user provides a run directory, use these commands to gather diagnostics:

bash

# Summary of failures
pegasus-analyzer <run-dir>

# Find failed job logs
find <run-dir> -name "*.out" -o -name "*.err" | head -20

# Read specific job output
cat <run-dir>/<job-id>.out
cat <run-dir>/<job-id>.err

Step 3: Match Against Known Failure Patterns

Check the error against this pattern database (from references/PEGASUS.md and 5 production workflows):

File Staging Failures

Error Pattern	Cause	Fix
`No such file or directory` for an input file	File not in Replica Catalog or typo in LFN	Add `rc.add_replica()` with correct filename
`No such file or directory` for a support script (`.R`, `.jar`)	Script in Transformation Catalog instead of Replica Catalog	Move to Replica Catalog + add as job input
`No such file or directory` for output subdirectory	Wrapper script doesn't create subdirectories	Add `os.makedirs(os.path.dirname(output), exist_ok=True)`
`FileNotFoundError` for `../bin/script.R`	Wrapper uses `__file__`-relative path	Use `os.path.join(os.getcwd(), "script.R")` instead
`glob()` / `os.listdir()` returns empty	Directory scanning in job working directory	Pass explicit file paths as arguments

Container Failures

Error Pattern	Cause	Fix
`FATAL: Unable to pull container`	Image name typo or network issue	Verify `docker://user/image:tag` is correct and accessible
`command not found` inside container	Tool not installed in container	Add tool to Dockerfile and rebuild
`ModuleNotFoundError` for Python package	Package not in container	Add `pip install` or `micromamba install` to Dockerfile

Resource Failures

Error Pattern	Cause	Fix
`MemoryError` or OOM killed	Insufficient memory allocation	Increase `.add_pegasus_profile(memory="N GB")`
`Bus error` (signal 7)	Memory or I/O issue	Increase memory; check for large temporary files
Job timeout	Step takes too long	Increase timeout; optimize the tool call

Argument Parsing Failures

Error Pattern	Cause	Fix
`unrecognized arguments`	Mismatch between `add_args()` and wrapper's argparse	Align argument names in both files
`the following arguments are required`	Missing argument in `add_args()`	Add the missing `--flag` to the job's `add_args()`
`error: argument --input: expected one argument`	Argument value contains spaces or is missing	Quote values or check argument construction

Dependency Failures

Error Pattern	Cause	Fix
Job runs before its input is ready	Missing dependency between jobs	Ensure `File` objects are shared between producer `add_outputs()` and consumer `add_inputs()`
Circular dependency error	Circular file references	Check that no file is both input and output of the same job
`mkdir` job not running first	Missing explicit dependency on mkdir	Add `self.wf.add_dependency(mkdir_job, children=[first_job])`

Wrapper Script Failures

Error Pattern	Cause	Fix
Exit code 1 but no stderr	Wrapper doesn't capture/print stderr	Add `print(result.stderr, file=sys.stderr)`
`Permission denied` on wrapper script	Script not executable	`chmod +x bin/script.py` or add shebang line
Output file not created	Tool succeeded but output path doesn't match	Verify output filename in wrapper matches `File()` LFN

Step 4: Read Relevant Source Files

Based on the identified failure pattern, read:

•The wrapper script that failed — check argparse, os.makedirs, subprocess calls
•The workflow_generator.py — check the job's add_args(), add_inputs(), add_outputs()
•The Dockerfile — check if the tool is installed
•The Replica Catalog entries — check file registrations

Step 5: Propose Fix

Provide a specific, actionable fix:

•Show the exact code change needed (diff-style or before/after)
•Explain why the error occurred (root cause, not just symptoms)
•
Show how to verify the fix:
- •For argument mismatches: python3 bin/wrapper.py --help
- •For container issues: docker run --rm image:tag which tool
- •For file staging: check Replica Catalog entries
- •For the whole workflow: python3 workflow_generator.py --help

Step 6: Prevention Advice

After fixing the immediate issue, suggest:

•Run /pegasus-review to catch other potential issues
•Use run_manual.sh to test each step locally before Pegasus submission
•Check the "Common File Staging Pitfalls" table in references/PEGASUS.md