Image models

Visitors upload a picture of any size. Your model gets it exactly as uploaded and prepares it itself: resizing, colours and scaling all happen inside the ONNX file.

What the model receives

Image input
Data typeuint8: whole numbers from 0 to 255
Shape[1, height, width, 3]: one picture, its own height and width, RGB
SizeAny. Height and width must be flexible (named sizes like "height"), not fixed numbers
Transparent areasBecome white

PeekML recognises an input like this as a picture automatically, and shows an image upload for it. Pictures can be up to 10 MB and 4096 pixels on each side.

Export from PyTorch

Wrap your network so it does the preparation you did during training. This example is for a digit reader trained on MNIST (28 × 28 greyscale pictures):

pip install torch onnx onnxscript
import torch, torch.nn as nn, torch.nn.functional as F

class AnyPicture(nn.Module):
    def __init__(self, net):
        super().__init__()
        self.net = net

    def forward(self, image):                          # [1, H, W, 3], 0-255, any size
        x = image.permute(0, 3, 1, 2).float() / 255    # -> [1, 3, H, W], 0-1
        x = x.mean(dim=1, keepdim=True)                # greyscale
        x = F.interpolate(x, size=(28, 28), mode="bilinear", antialias=True)
        x = (x - 0.1307) / 0.3081                      # the scaling used in training
        return torch.softmax(self.net(x), dim=1)       # probabilities, not raw scores

any_size = {1: torch.export.Dim("height", min=2, max=8192),
            2: torch.export.Dim("width", min=2, max=8192)}
torch.onnx.export(AnyPicture(net).eval(), (torch.zeros(1, 64, 64, 3, dtype=torch.uint8),), "model.onnx",
                  input_names=["image"], output_names=["probabilities"],
                  dynamic_shapes={"image": any_size},   # height and width can be anything
                  opset_version=18, external_data=False)

What to put in the wrapper

  • Resize to the size your network was trained on. antialias=True keeps thin lines from disappearing when big photos are shrunk.
  • Colours: convert to greyscale if you trained on greyscale. Keep RGB order for colour models.
  • Scaling: apply the same mean and spread (or 0-1 scaling) you used in training.
  • Softmax at the end, so the model returns probabilities PeekML can show as bars.
  • Anything else your training data needed, such as flipping a dark digit on a light background to MNIST's white on black.

PeekML tests every image model with two pictures of different sizes before publishing. If the model only works at one size, publishing stops and says so.

peekml.yaml

List the input with type: string. contentMediaType: image/* is optional (PeekML adds it when it creates peekml.yaml for you) and makes the file say what the input is:

inputs:
  properties:
    image:
      type: string
      contentMediaType: image/*
      description: A picture of one digit

outputs:
  properties:
    probabilities:
      type: array
      x-classes: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]

Quote class names that look like numbers, as above. See Outputs for more on x-classes.