Examples gallery: Inference · Docs

The examples repository

examples/inference_jobs/predict holds complete predictor-mode jobs across modalities. Like the vLLM set, the models are general-purpose tools with no sector attached; each folder is a working reference for one shape of the predict(data) → dict contract.

Folder	What it serves	Min VRAM
sam2.1_segmentation	point/box-prompted image segmentation — masks out	6 GB
FLUX.1-schnell_imagegen	text-to-image generation	16 GB
Orpheus-TTS-3B_tts	text-to-speech	8 GB
image_classification	a compact CV classifier — the smallest real-model reference	—
minimal_echo	the five-line Echo predictor — the contract with nothing else in the way	none (CPU)

Every folder ships the same four files:

<model>_<task>/

inference.yamlWhich class to load, its init_args, and the hardware: workers, GPUs and free VRAM.
scripts/<name>_serve.pyThe predictor class with predict(data) → dict. This is the file you edit.
submit.pyDeploys the job with the SDK and prints the endpoint and the API key.
predict.pyA working client that sends this job’s request shape.

Run any of them

Shell

cd <job-folder>
cp .env.example .env     # RESON_API_KEY (rsk_…) + S3 keys
python submit.py         # uploads the bundle, prints endpoint + predict key
python predict.py        # job-specific smoke test

note

Or paste the serve script and yaml into the dashboard's inference wizard. Jobs that pull weights from HuggingFace Hub take a few minutes on first boot; the submit script waits for RUNNING.

Previousinference.yaml Next SAM 2.1 segmentation