Dataset format & sharding · Docs

You shard, the platform counts

The platform doesn’t split datasets. You build the shards on your machine, zip them, and upload them to jobs/<name>/shards/ in your bucket.

Rule	What it means
Only zips count	The name must end in lowercase .zip (the check is case-sensitive)
Zips = workers	Four zips means four GPUs. There is no override
Pairing	Zips go to workers in listing order; each worker downloads only its own
No content checks	A zero-byte zip passes. Shard quality is on you, so check it before uploading

Each worker unpacks its zip flat into its data root. Zip the contents, never the folder that holds them:

Shell

# right: the worker sees data_root/images/ and data_root/data.yaml
zip -r shard_0.zip images/ data.yaml

# wrong: the worker sees data_root/shard/images/ and your paths break
zip -r shard_0.zip shard/

Upload from the Files page, the upload panel in step 6 of the training wizard, or rclone copy ./shards resontech:<bucket>/jobs/<name>/shards. The SDK uploads them for you.

Splitting

Every example job splits with round-robin striding in build_shards_hf.py. Edit HF_DATASET, NUM_SHARDS and MAX_EXAMPLES at the top of that file.

build_shards_hf.py

indices = list(range(len(train_ds)))
partitions = [indices[i::NUM_SHARDS] for i in range(NUM_SHARDS)]

Property	Detail
Deterministic	No random seed; rebuilding gives identical shards
Equal sizes ± 1 sample	The first `len % N` shards get one extra example
Near class balance	A side effect of striding over ordered data. If you need it guaranteed, use the stratified splitter in the job_training_sdk example’s build_shards.py
Validation copied to every shard	Every worker validates on the same full set, so per-worker metrics compare round to round. Keep validation small: it is stored N times

Shard size sets influence

The FL server averages the workers’ weights by the sample count each one reports:

formula

global = Σ (samples_i × weights_i) ÷ Σ samples_i

Case	Result
Unequal shards	Weighted correctly: a 10k-sample worker pulls 10× harder than a 1k one. Equal shards matter only for even GPU use, since every round waits for the slowest worker
`samples` missing from the return value	That worker counts as 0 and drops out of the average. If every worker reports 0 the job fails
Equal influence regardless of size	Set `aggregation_weights` per worker and `weigh_by_local_iter: false` on the aggregator in `config_fed_server.json`
A late or duplicate result	Dropped. One result per worker per round

scripts/model_def.py

return {"weights": model.get_weights(), "samples": len(train_ds)}   # always

Thresholds for every data type

Check	Rule	Why
Steps per round	`shard_size ÷ batch_size` ≥ ~50	Fewer steps make each round a few noisy updates. Measured: job_classify runs 148 steps per round per worker and reproduces within 0.09 pp
Class coverage	Every label in every shard	A shard missing a class pulls the classifier away from it every round
Same distribution	Shards drawn from one distribution	Averaging converges best on similar shards. Per-site data that differs needs more rounds and a lower learning rate
No leakage	No sample in two shards; validation never overlaps training	Duplicates count twice; leaked validation inflates every metric you tune on
Disk	Shard + checkpoints fit the worker disk	Set the disk minimum: Choosing hardware
VRAM	One training step fits the card	Shard size doesn’t change VRAM; batch size, sequence length and method do

Layouts by data type

“Measured” rows come from real runs; the rest are starting points to check on your data.

Image classification

job_classify, job_medical, job_imagenet_scratch. Class names come from folder names.

shard_0.zip

images/train/<class_name>/00000000.jpg
images/val/<class_name>/00000000.jpg      # the full validation set, in every shard
data.yaml                                 # nc: 101, names: [...]

Signal	Value	Basis
Samples per class per shard	≥ ~20 to fine-tune a pretrained backbone	Starting point
Shard size, fine-tune	4,750 per shard × 4 workers reached 89.42% top-1 on Food-101, above the centralized reference	Measured
Too small	Under ~1,500 per shard at batch 128 gives under ~12 steps per round. Lower the batch before shrinking shards	Starting point
From scratch	Needs many more passes per shard: job_imagenet_scratch runs 60 local epochs	Starting point

Object detection

job_yolo11, job_csgo. The shard’s data.yaml is an Ultralytics descriptor; its path is rewritten to the data root at runtime.

shard_0.zip

images/train/*.jpg     labels/train/*.txt     # cx cy w h, normalized
images/val/*.jpg       labels/val/*.txt
data.yaml                                     # nc, names, train:, val:

Signal	Value	Basis
Object instances per class per shard	≥ ~50. Balance boxes, not images: one image can hold many	Starting point
Images with no labels	Keep some in every shard, at the same ratio	Starting point

Text and LLM fine-tuning

job_llm, job_agent, job_lm_scratch. Only the LoRA adapter travels each round, so traffic stays flat at any base size.

shard_0.zip

data.jsonl     # {"instruction": ..., "input": ..., "output": ...}
               # job_agent, job_lm_scratch: {"text": "..."} per line

Budget by tokens, not rows. A row can be 100 tokens or 8,000. Equalize token mass, or return token counts as samples.
Keep each dialogue or reasoning trace in one shard. Group by conversation id, then stride.
Deduplicate before sharding. Near-duplicates spread over shards act as a hidden upweight.
Sort by length, then stride, so every shard gets the same mix of short and long rows.

Signal	Value	Basis
Rows per shard, LoRA SFT	≥ ~500 at batch 4 (~125 steps per round)	Starting point
Token mass, largest vs smallest shard	≤ ~1.5×	Starting point
Traces cut mid-chain	0	Rule

Speech recognition

job_speech (Whisper-small + LoRA). Paths in data.jsonl are relative to the shard root.

shard_0.zip

audio/00000000.wav     # 16 kHz mono PCM_16
data.jsonl             # {"audio": "audio/00000000.wav", "text": "..."}

Signal	Value	Basis
Audio per shard	≥ ~30 minutes. Balance minutes, not clip counts	Starting point
Speakers per shard	≥ ~5, overlapping across shards. A one-speaker shard drifts toward that voice	Starting point

Other example jobs

Job	Shard content
job_embed	`data.jsonl` of `{anchor, positive, negative?}` triplets
job_diffusion	`images/*.jpg` and `data.jsonl` of `{image, caption}`
job_training_sdk	`train.pt` and `val.pt` tensor dicts `{"x": …, "y": …}`

Before you upload

N zips for N workers, lowercase .zip.
Contents zipped flat, with no wrapper folder.
Every shard opens and loads with the job’s own loader on your machine.
Every label present in every shard, or the stratified splitter used.
The same validation set in every shard, with no overlap with training.
Each shard fits the worker disk; batch and method fit its VRAM.
Your fl_train_model returns a true samples.

warn

Upload the shards before you launch: the job reads the shard count at submit.

PreviousChoosing hardware Next Track jobs & download results