Sync Job Expert
Expert guidance for the Frappe Tweaks Sync Job framework - a queue-based system for data synchronization between DocTypes.
Core Concepts
Sync Job Type: Template defining sync configuration (source/target doctypes, queue, timeout, retry settings)
Sync Job: Individual task instance tracking execution, status, errors, and diffs
Controller: Python module implementing sync logic (Bypass or Standard mode)
Quick Start
Creating: Navigate to Sync Job Type > New, set name and doctypes, save to generate controller
Enqueueing: See references/enqueueing.md for enqueue_sync_job() parameters and examples
Implementing: See references/implementation.md for step-by-step controller implementation
Implementation Modes
Choose one of 3 paths (see references/implementation.md for details):
- •Single Target (Standard): One-to-one sync - implement
get_target_document()+update_target_doc() - •Multiple Targets (Standard): One-to-many sync - implement
get_multiple_target_documents()+update_target_doc() - •Bypass: Full control - implement
execute()
Status Flow
- •Pending → 2. Queued → 3. Started → Finished/Failed/Skipped/Relayed/No Target
- •Canceled: Manual cancel (from Pending/Queued/Failed)
- •Failed: Retries automatically if
retry_count < max_retries
Key Methods
On sync_job:
- •
sync_job.get_context()- Parse context dict - •
sync_job.get_source_document()- Load source (even if deleted) - •
sync_job.get_trigger_document()- Load triggering document - •
sync_job.get_target_document()- Load target
Flags (control operations):
- •
sync_job.insert_enabled/update_enabled/delete_enabled - •
sync_job.update_without_changes_enabled - •
sync_job.dry_run- Calculate diff without saving
Configuration
Sync Job Type defaults (can override per job):
- •Queue: "default", "short", "long"
- •Timeout: seconds
- •Retry Delay: minutes between retries
- •Max Retries: maximum attempts
- •Verbose Logging: preserve data snapshots (disabled by default)
Context: Pass custom data to sync logic Trigger Tracking: Track what triggered the job for auditing
Dry Run Mode
Test sync without changes:
sync_job = enqueue_sync_job(
sync_job_type="SAP Customer Sync",
source_document_name="CUST-00001",
dry_run=True # Populates diff_summary without saving
)
Troubleshooting
Job stays Queued: Check RQ workers running (bench worker), verify queue name
Job fails repeatedly: Check error_message, verify documents exist, validate module path
No changes detected: Set update_without_changes_enabled=True or check diff generation
Module not found: Ensure controller exists, check naming (use scrubbed names), run bench migrate
Best Practices
- •Use Standard Mode for simple mappings, Bypass for complex operations
- •Validate source/target before syncing
- •Handle missing targets gracefully (return
target_document_type=None) - •Use context for runtime parameters
- •Set appropriate timeouts for operation complexity
- •Use specific queues for heavy operations
- •Log important decisions with
frappe.log_error() - •Test retry logic with intentional failures
Source Code References
- •
tweaks/utils/sync_job.py- Core utilities - •
tweaks/tweaks/doctype/sync_job_type/- Sync Job Type DocType - •
tweaks/tweaks/doctype/sync_job/- Sync Job DocType - •
tweaks/tweaks/doctype/sync_job_type/boilerplate/controller._py- Controller template