Quick start · Docs · ResonTech

The idea

You upload a small bundle: one inference.yaml plus a scripts/ folder with your Python. The platform finds GPU workers that match your requirements, boots your model on them, and hands you back two things:

an endpoint URL where your model listens,
a predict API key, shown once at submit. Save it; every request needs it in the X-API-Key header.

From that point the runtime is a normal HTTP service. Your code, your weights, your endpoint. Stop the job and billing stops with it.

Two runtime types

The engine field in your yaml picks one of two contracts. Everything else about submitting is the same.

	vLLM (engine: vllm_openai)	Custom (/predict)
Built for	LLMs: chat, coding assistants, vision-language, embeddings, rerankers	Any model: segmentation, TTS, image generation, classifiers, your research code
You write	a `serve_module.py` that declares which model to serve and how	a Python class with `predict(data: bytes) → dict`
Clients call	`POST /v1/chat/completions` and the rest of the OpenAI protocol	`POST /predict` with raw bytes or JSON
Works with	openai-python, curl, LangChain, Aider, Cline, Continue. Unmodified.	any HTTP client, or the SDK's InferenceClient
Weights	pulled from HuggingFace Hub at boot	from HF Hub, or a weights file you upload

Serving an open LLM? Use vLLM mode and skip writing inference code entirely. Serving anything else, or your own architecture? Use the custom contract. Start with Serve vLLM models or Serve custom models.

Submit a job

1
Get your bundle
Fastest path: copy a folder from the examples repository (vLLM examples, custom examples) and edit it. Each folder is complete: yaml, scripts, a submit script, a test client.
2
Submit from Python
One call uploads the bundle and starts provisioning.
3
Save the endpoint and key
The submit response prints both. The key is never shown again; you can rotate it later from the dashboard.

submit.py

from resontech import InferenceConfig, ResonTech, ResonTechConfig

sdk = ResonTech(ResonTechConfig(
    platform_api_key="rsk_...",        # from /profile → Developer
    s3_access_key_id="...",
    s3_secret_access_key="...",
))

job = sdk.rt_submit_inference(
    name="my-model",
    inference=InferenceConfig(
        gpu_models=["rtx_5070", "rtx_5090"],  # ordered preference
        max_price_per_gpu_hour=2.50,          # $ per GPU per hour
    ),
    inference_yaml="./inference.yaml",
    scripts_dir="./scripts",
)

print(job.endpoint)            # → https://<subdomain>.../predict
print(job.predict_api_key)     # shown ONCE — save it now
job.wait_until_ready(timeout=3600)

note

gpu_models and max_price_per_gpu_hour are required unless you pin worker_ids. Together they bound what the platform may pick for you: which GPUs, and at what price. gpu_models is an ordered preference of catalog ids, not a set — workers are taken from the first entry before the next. With two RTX 5070s and one 5090 free and two workers needed, you get the pair of 5070s. The price is per GPU per hour, the same unit the catalog quotes, so a 4-GPU worker costs four times it. Valid ids come from the list below.

Loading GPU ids…

note

First boot downloads model weights, so expect minutes rather than seconds on a cold worker. wait_until_ready() polls until the runtime is RUNNING. You can watch the same progress on the dashboard, worker by worker.

Full parameter reference: Submit inference via SDK. Prefer the browser? The dashboard's inference wizard accepts the same two files.

Call it

A vLLM runtime speaks the OpenAI protocol. Point any OpenAI client at <endpoint>/v1:

vLLM runtime — curl

curl "$ENDPOINT/v1/chat/completions" \
  -H "X-API-Key: $PREDICT_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "assistant",
    "messages": [{"role": "user", "content": "Say hello."}],
    "max_tokens": 64
  }'

A custom runtime takes bytes on /predict and returns your class's JSON:

custom runtime — curl

curl -X POST "$ENDPOINT" \
  -H "X-API-Key: $PREDICT_KEY" \
  -H "Content-Type: application/octet-stream" \
  --data-binary @input.png

Both runtime types also expose GET /health (no auth) as a readiness check, and the dashboard playground lets you test either kind from the browser.

PreviousOverview Next Choosing hardware