A Two-Level Content-Moderation Pipeline: Fast L1 Gate, Async ML L2

Content moderation has an economics problem before it has a technical one. If every comment, image, and uploaded PDF has to pass through a transformer model, you're paying GPU money on content that a five-line function could have rejected — or cleared — instantly. Most user-generated content is boring: plain text with no slurs, an image that's obviously fine. You don't need a neural net to tell you that.

On ModeraStack — the moderation service inside my iamwithtrees platform — I built the pipeline in two levels precisely so the expensive part runs as rarely as possible. It's tuned to fit on a CPU-only VPS, no GPU budget at all.

Animated pipeline: content enters at /check and hits the synchronous L1 gate; most items exit instantly as block or allow, while an ambiguous item falls through to a queue, gets processed by the async L2 ML worker, and its verdict returns via a signed webhook. — Most content is settled at L1 in milliseconds; only the ambiguous fraction reaches the ML worker and comes back as a signed webhook.

L1: the synchronous gate

Everything enters through one /check endpoint. The first thing it hits is L1, an in-process engine that answers in milliseconds and never leaves the Node process. For text it does normalization (unicode folding, leetspeak, zero-width junk) and runs an Aho-Corasick blocklist — a single pass that matches thousands of banned terms at once — plus spam and pattern checks. For media it does cheap structural validation: magic-byte checks so a .jpg is really a JPEG, and perceptual hashing to catch known-bad images.

// L1 runs synchronously; most content is decided right here.
const l1 = await runL1(content); // normalize, Aho-Corasick, spam, magic-byte, phash

if (l1.decision === "block") return respond({ verdict: "block", by: "L1", reasons: l1.reasons });
if (l1.decision === "allow" && !content.needsDeepScan) return respond({ verdict: "allow", by: "L1" });

// Only the genuinely ambiguous stuff falls through to the expensive lane.
await enqueueL2({ contentId, tenantId, webhookUrl });
return respond({ verdict: "pending", by: "L1" });

That enqueueL2 on the last line is the design in one gesture. L1's job isn't to make every decision — it's to make the cheap decisions and hand only the genuinely uncertain cases to the slow lane. A hard blocklist hit is settled instantly; a clean short comment is cleared instantly; the murky middle goes to ML.

L2: the ML worker, off the request path

L2 is a separate Python worker with the heavy models — a stack of transformers for text and image classification, and faster-whisper for transcribing audio out of video so speech can be moderated too. Video gets frame- sampled with ffmpeg; PDFs get their text and images extracted. This is the part that costs CPU seconds, so it runs asynchronously, pulling jobs off a queue, completely detached from the caller's HTTP request.

Side-by-side breakdown: L1 is in-process TypeScript doing unicode folding, an Aho-Corasick blocklist, spam and pattern checks, magic-byte validation, and perceptual hashing; L2 is an async Python worker running transformer classification, faster-whisper transcription, ffmpeg frame-sampling, PDF extraction, and delivering a signed webhook. — L1 is TypeScript in-process for cheap structural checks; L2 is Python where the ML ecosystem belongs — split by each language's strength.

The client never blocks on it. L1 returns pending immediately, and when L2 finishes it delivers the verdict as a single signed webhook back to the tenant. Signed, because a moderation verdict is a security decision — the receiver has to be able to prove it came from ModeraStack and wasn't forged by whoever's trying to sneak content through.

Why the split is the whole point

I'll be blunt: a single-tier "run the model on everything" design is easier to write and wrong for anything real. The two-level split gives you three things at once:

Latency where it matters — the caller gets an answer in milliseconds for the common case, not after a model round-trip.
Cost control — the GPU-shaped workload only runs on the fraction of content that actually needs it, which is what makes a CPU VPS viable.
A clean seam — L1 is TypeScript in-process, L2 is Python with the ML ecosystem where it belongs. Neither language is fighting the other's strengths.

The interesting engineering isn't the models. It's the routing decision — how confidently L1 can clear or block something so L2 stays idle. Every rule you can push into L1 is a model invocation you don't pay for.

Multi-tenant from the first commit

I built it multi-tenant on day one: users own apps, apps carry rotating API keys, and every check is metered per tenant. That wasn't premature — retrofitting tenancy and usage metering into a moderation service after the fact means rewriting the data model under live traffic. The schema was designed so a future self-serve billing portal is purely additive, no breaking migration. Cheap insurance.

Takeaways

Put a fast synchronous L1 gate in front of your ML. It should settle the easy yes/no cases in milliseconds and forward only the ambiguous ones.
Run ML asynchronously off the request path and return the verdict via a signed webhook — a moderation result is a security decision.
Split by strength: in-process rules for cheap structural checks, a Python worker for the models.
Bake multi-tenancy and metering in early — it's brutal to retrofit.

The best moderation pipeline runs its most expensive component the least. Design for that and a CPU box goes a long way.

I build backend, ML-adjacent, and SaaS systems end-to-end. If you're working on something similar, let's talk.