How we built Okidor

The shape of the problem

An investor writes down why they own a company. Months later a filing lands. The question is not “what does this filing say”, which any language model answers well, but “does this filing bear on what that specific person wrote down”, which is a different job and a much narrower one.

Two properties fall out of that framing and both shaped the build. Company data is shared, because a 10-Q is the same document for everybody who holds the stock. Judgements are not, because they are made against one person’s reasoning. So the pipeline fans out at exactly one point: ingest once, evaluate per thesis.

Why the queue lives in the database

The obvious design is for the ingestion function to loop over the theses and create the work. We do it in a Postgres trigger instead, and the reason is durability. A serverless invocation can be killed mid-loop, and if the work items only exist in that loop, some of them are simply never created and nobody finds out. Nothing errors. A thesis just quietly stops being audited.

Inserting the source row and enqueueing the audits in the same transaction makes that impossible. Either both happened or neither did. The queue is a table, the worker is a cron job that claims rows from it, and a crash loses at most the row in flight, which is retried.

Five scheduled functions drive it, and their schedules live in migrations rather than in a dashboard, so a rebuilt database comes back with its clock intact: filing discovery every thirty minutes, the processing worker every ten, earnings transcripts every two hours, news on a rotation, and a weekly ticker map refresh.

Which models, and where

Three tiers, all on Fireworks, chosen for different constraints rather than by reputation.

• Summarisation runs once per document on a large input, so it needs context length and cheapness more than speed. Nobody is waiting.
• Audit runs once per thesis-and-document pair on a compact input. Both currently use deepseek-v4-flash.
• The thought partner runs while a person watches a spinner, so latency is the constraint. It uses a faster router, picked by measurement: on the same prompt it answered in 1.7 seconds using 197 completion tokens where the audit model took 5.6 seconds and 490, and the reply was no worse.

The discipline around them matters more than the choice of them. Temperature zero, JSON mode, a Zod schema on every response, one repair retry, and the model name, prompt version and token counts stored on every row that a model touched. Without that last part you cannot answer “why did this audit say that” six weeks later, which is the only question anybody actually asks.

One hard-won rule: verify a model is in your account’s catalogue before deploying it. An earlier default was silently removed upstream and started returning 404 on every call. The pipeline degraded into its failure fallback and looked, from the outside, exactly like a model that had got worse.

The model does not decide the verdict

This is the architectural decision the product rests on, so it is worth being precise about the mechanism rather than the intention.

The schema the audit model is allowed to return has no verdict field in it. It returns a directional reading, which assumptions the source bears on and which way, the passages it drew that from, and a per-criterion assessment. The final label is then computed from that structured output plus the deterministic checks, by an ordered set of rules in application code.

Two consequences follow, and they are the whole point. The same inputs always produce the same verdict. And a numeric criterion such as “gross margin below 40 percent” is settled by arithmetic over the company’s own reported XBRL figures before the model is asked anything, so it cannot be talked out of a breach by fluent prose.

A database constraint backs this up: a verdict may only exist on a row whose status is complete. A pipeline outage therefore cannot surface as “thesis holds”. That constraint exists because the system once produced the string “Material event filed. Thesis evaluation pending full review” and presented it to a user as a result.

Three failures worth stealing

A silent cron for two weeks. The pipeline looked like a model problem and was a scheduled job failing on a database setting nobody had applied. Nothing alerted, because a job that never runs produces no errors. If you take one thing from this page: a quiet week is indistinguishable from a broken pipeline unless you instrument the difference, and you should check the job run history before you blame the model.

An embed error swallowed as data. Adding a second foreign-key relationship between two tables made an ambiguous join, and PostgREST returned an error where rows were expected. The worker discarded the error, saw no rows and reported zero audits completed, every run, cheerfully. Discarding an error because the happy path returns an array is a very easy thing to do and a very hard thing to notice.

Nested output schemas fail more often than flat ones. A nested object dropped a required key on three consecutive attempts and was fine immediately after being flattened. If a model keeps failing validation, try removing a level of nesting before you try a better model.

The stack, briefly

Next.js on Vercel, one application deployed once and served on two hostnames: the marketing site and the product, split by a routing rule in middleware. Supabase for Postgres, auth, storage, row-level security and Deno edge functions, with pg_cron driving the schedules.

Row-level security is the data boundary rather than application checks, with one caveat learned the hard way and worth passing on: RLS decides what a caller may read, which is not the same as what a query is asking for. Two permissive policies on the same table are ORed together, so a query that did not name an owner returned the caller’s rows plus every published row in the system. Anything that means “mine” has to say so.

What this is for

All of the above exists to answer one question for one person: is the reason you bought this still true. You can see a worked audit without an account, read how the engine decides in product terms, or start with your own thesis.