Building a media provider extension (or: swap S3 by activating a different one)

Here is a thing about Squilla that took me a while to fully appreciate, even though I designed it that way. The kernel keeps zero bytes on disk. Not one. Every single media operation, whether that is an upload, a fetch, a query, or a delete, gets routed out to whichever active extension declares provides:["media-provider"] in its manifest. The core itself has no concept of "where files live." It just knows how to ask.

That sounds abstract until you realize what it actually means for you as an operator. If you want to move from local disk to S3, you do not patch the kernel. You do not edit a config file with a storage driver string. You activate a different extension at a higher priority, and the kernel quietly starts routing every media call to that one instead. The old extension stays installed, ready to take over again if you deactivate the new one. That is the whole switchover.

What fills the slot today

Out of the box, the bundled media-manager extension fills the media-provider slot. It writes uploads to storage/uploads/ on the local filesystem and serves them back through the kernel's public route proxy. It is a real extension, not a special case. If you uninstalled it tomorrow and activated something called cloudfront-media instead, the kernel would not blink. It would just start asking the new plugin to handle Upload and friends.

This is the part I want people to internalize: there is no "default" storage backend baked into core. The default is whichever extension you activated first. If you activate two media providers, the one with higher priority wins. Hot swap.

What a media provider extension actually looks like

Let's walk through what a hypothetical cloudfront-media extension would contain. It is not magic. It is a gRPC plugin with a manifest that declares the right capability tag.

First, the manifest. This is the file the kernel reads on activation to figure out what the extension is and what it can do.

{
  "slug": "cloudfront-media",
  "name": "CloudFront Media Provider",
  "version": "0.1.0",
  "provides": ["media-provider"],
  "capabilities": ["files:write", "http:fetch", "settings:read"],
  "plugin": {
    "binary": "cloudfront-media",
    "protocol": "grpc"
  }
}

Two lines are doing the heavy lifting here. provides:["media-provider"] is what tells the plugin manager "this binary can fill the media slot." capabilities is what tells the CoreAPI guard which methods this extension is allowed to call back into. If your provider needs to push bytes to S3, you need files:write at minimum. If you want to read keys from the settings store, add settings:read. The kernel enforces every one of these at the boundary.

The four methods the kernel proxies

When a media-provider extension is active, the kernel's CoreAPI.Media.* calls do not run any local code. They package the request, call the plugin over gRPC, and return whatever the plugin returns. The plugin implements four methods: Upload, Get, Query, and Delete. Here is what Upload might look like for an S3 backend.

func (p *Plugin) Upload(ctx context.Context, req coreapi.MediaUploadRequest) (*coreapi.MediaFile, error) {
    key := buildKey(req.Filename, req.MimeType)

    _, err := p.s3.PutObject(ctx, &s3.PutObjectInput{
        Bucket:      aws.String(p.bucket),
        Key:         aws.String(key),
        Body:        bytes.NewReader(req.Body),
        ContentType: aws.String(req.MimeType),
    })
    if err != nil {
        return nil, fmt.Errorf("s3 put: %w", err)
    }

    return &coreapi.MediaFile{
        URL:      p.cdnBase + "/" + key,
        MimeType: req.MimeType,
        Size:     int64(len(req.Body)),
    }, nil
}

That is the whole contract for one of the four methods. Bytes in, a *MediaFile out. The kernel does not care what storage you used. It just needs the returned URL to be reachable by the public site, and the returned ID to round-trip back through Get and Delete later.

The hot-swap moment

Here is the part I find genuinely satisfying. You install the new extension, you activate it with a higher priority than media-manager, and the very next upload goes to S3. No restart. No data migration unless you want one. Old media keeps resolving because each MediaFile carries its own URL, so existing references in pages and posts point straight at the bytes without having to go back through a provider at all. New media goes to the new provider.

If you decide S3 was a mistake, you deactivate cloudfront-media and the slot falls back to media-manager. New uploads go back to local disk. The kernel never moved. It was just routing the whole time.

Why I keep harping on this

The reason this design matters is not because S3 is special. It is because the same pattern works for email providers, for search, for sitemaps, for any feature where the kernel is happier being generic than opinionated. If the core knew about S3, it would also need to know about Cloudflare R2, and Backblaze, and whatever shows up next year. Instead it knows about a slot, and the slot is filled by whoever raised their hand last.

That is the magic of keeping the kernel generic. The interesting work happens at the edges, in extensions you can swap without touching anything load-bearing.