Rapidata Use

Overview

Translate user requests into Rapidata SDK workflows: authenticate, submit orders, apply filters/settings/selections, create validation sets, monitor progress, and retrieve results. Use MRI when the user wants model ranking/benchmarking.

Quick start (auth)

•Initialize with rapi = RapidataClient().
•If the machine is not authenticated, ask the user to complete the browser login prompt; credentials are saved locally.
•If the user has client ID/secret, pass them to RapidataClient(client_id=..., client_secret=...).

Task routing (open references only as needed)

•API surface + method signatures: references/api.md
•Order/validation workflows + prompt design + early stopping: references/workflows.md
•Results schema + pandas export: references/results.md

Default assumptions (confirm before submit)

Ask the user to confirm each default when they did not specify it:

•responses_per_datapoint=10 (or responses_per_comparison=1 for ranking)
•data_type="media"
•filters=[UserScoreFilter(0.55, 0.95)]
•settings=[]
•selections=[]
•validation_set_id=None
•confidence_threshold=None
•contexts=None, media_contexts=None, private_notes=None
•Compare: a_b_names=None; Ranking: random_comparisons_ratio=0.5

If the user rejects the default user-score filter, remove it entirely or apply their requested bounds. Treat the defaults above as preferences and always confirm them with the user before submission.

Core workflows

1) Submit a labeling order from a dataset

•Ask for: order type, instruction/question, datapoints, optional contexts/media_contexts/private_notes, responses_per_datapoint, and any filters/settings/selections.
•Map the dataset into the expected Rapidata inputs (lists of strings or pairs) and verify all list lengths match.
•Create the order with rapi.order.create_*_order(...), then call order.preview() (if requested) and order.run().

2) Pairwise tasks (true compare)

•Use create_compare_order with datapoints=[ [left,right], ... ] (two unique items per datapoint).
•If the user provides pre-concatenated left/right images, prefer a true compare order by using the original two files; only use classification if the data is irreversibly concatenated.

3) Quality via validation sets

•Create validation sets with rapi.validation.create_*_set(...) and attach via validation_set_id or selections.
•If selections are provided, validation_set_id is ignored; pick one approach.

4) Early stopping

•For classification/compare, set confidence_threshold to stop early once confidence is reached.
•Keep responses_per_datapoint as the maximum cap.

5) Monitor or manage orders

•Use order.display_progress_bar(), order.get_results(preliminary_results=...), order.pause(), order.unpause(), order.delete().
•Use rapi.order.find_orders(...) or rapi.order.get_order_by_id(...) for retrieval.

6) MRI benchmarks

•Use rapi.mri.create_new_benchmark(...), benchmark.create_leaderboard(...), and benchmark.evaluate_model(...).
•Retrieve standings from leaderboard.get_standings() or benchmark.get_overall_standings().

Notes and guardrails

•Prefer writing a script to disk and running it for multi-step actions (avoid inline Python).
•Ask the user for any clarifications before submitting orders when requirements are ambiguous.
•Compare datapoints must be pairs of two unique items; ranking requires lists of >=2 unique items per group.
•API/platform constraints (must enforce): Rapidata rejects contexts longer than 400 characters (truncate to <= 380), and only SFW content should be submitted.
•Compare orders only allow a_b_names for the A/B labels; the optional "I can't tell" choice comes from AllowNeitherBoth() and cannot be renamed. If you need exact label text like "I can't tell," use a classification order with stitched images instead.
•Preferences (confirm with user): batch compare/classification orders to <= 100 datapoints each; add resume flags (--start, --total-batches) to avoid re-uploading when rerunning. Reruns create duplicate orders, so prefer an explicit cleanup step (find-by-name + delete) before resubmitting. Credits are consumed on submission—start with a tiny test batch (~10 items), confirm format, then scale up.
•timestamp order is marked not fully supported; warn the user.
•SDK delete exists for orders and validation sets; for any finer-grained deletion (individual tasks/responses), ask for the API endpoint or confirm using the OpenAPI client.
•Quickstart doc notes a 100-datapoint limit per order; ask if they need batching.

rapidata-use