Opinionated software

Jev is a different kind of LLM. You pass in text and questions, and it answers them in a standard JSON schema. There's no prose output. A couple of quick examples:

• Classify a help desk ticket into: cancellation, refund, question
• Predict from an email if a customer is happy or angry

That's the simplest possible example. It can get incredibly sophisticated. Here's an example of Jev playing Doom, being called every 100ms to decide whether to move, change directions, jump, or shoot based on the state of the game (health, ammo, location of enemies, are they shooting at you, etc):

I have an AI app called Work Coach. It's a desktop app that coaches you based on your meetings (interviews, 1:1s, sales calls, presentations, etc). It analyzes what happened, what you said, and how you said it. It gives you the feedback nobody is telling you: what went well, what to improve upon for next time, and the moment in the meeting where it happened.

That coaching is the core of the product, and it's the part I rebuilt on Jev and open-source models. Here are some high level stats of the before and after:

When you asked Work Coach to observe a meeting, it recorded it and then analyzed the transcript. The pipeline:

1. Classified the meeting type (interview, 1:1, presentation)
2. Classified your role (interviewee, manager, attendee)
3. Loaded tools, frameworks, and context to analyze that kind of a meeting
4. Called an LLM to analyze the meeting transcript with that context

The dynamic custom prompt was the primary source of coaching. An interview candidate got areas like "STAR answer quality". A manager in a one-on-one was coached on how well they coached rather than solved problems for the other person.

I had tried open weight models in the past but ultimately stuck with Sonnet. Sonnet was higher quality, especially when I used a fair amount of thinking tokens.

That all changed when I used Jev and open models together.

Here is the same meeting in the new workflow. There is power in the pattern that Jev builds on:

• Jev helps to make decision trees more deterministic and manageable
• Jev's latency and parallel design enables a much more sophisticated system
• Jev enables more targeted and well scoped tasks
• Open models are much better when given smaller and tighter scoped tasks
• There is less chance for hallucinations when you do use an LLM
• The probability score of its answers allows for code to be more considerate and handle complicated situations

Previously, I told Claude to double check what it was recommending and to ground its work in quotes from the transcript. It still hallucinated all the time. It drove me crazy.

The coaching points now write the citations that drove the coaching point, for example: "you never asked when you'd hear back." Jev is now used to fact check those points and rewrite anything that isn't grounded in the meeting. If a fact fails, Jev checks it once more against the whole meeting, in case it happened somewhere else. If it still fails, the writer gets one rewrite with the reason attached. If the rewrite fails too, the point is dropped.

Before Jev, checking every sentence like this would have added cost and another wait. Now it's the cheapest part of the whole pipeline, and I've gotten a lot fewer complaints about hallucinations.

Jev doesn't write user facing copy, so you still need an LLM for that. I've found that using Jev has closed the gap for me between the open weight models and the frontier models because the LLM has a much smaller and tighter scope of what it needs to do. The LLM no longer decides what's wrong with your meeting; it gets a chapter, the judgments that fired and a short brief, and writes a few sentences. It's faster, cheaper, and produces really high quality suggestions.

When I ran benchmarks comparing the outputs, the Jev + open weight model combination scored much better in terms of user preference as well as reducing hallucinations.

The hardest thing Work Coach grades is a case interview. A case is a business problem you work through out loud, like "Why are this client's profits down?" Interviewers score how you clarify the problem, structure it, use the data, do the math and land a recommendation.

That used to be one huge prompt. Now it's six calls in three waves, and each wave uses the answers from the wave before it. Five of the six calls are Jev. The open model has one job: it sketches the tree of ideas you talked through. Jev measures and checks everything else, including that tree.

Here is the whole process on one 45-minute job interview where you were the candidate. Round one classifies the whole transcript: what kind of meeting it is, and your role. The transcript then breaks into chapters, and round two makes one call per chapter that works out what kind of chapter it is and asks the follow-ups for every kind it could be. Round three goes deep only where round two says it should. Then the writing, and a check on every claim. Press Next or Play to step through it, and hover any box to read its question.

My ambition for the app has increased exponentially as I realize what Jev enables. I can be obsessive around evaluating different perspectives and questions, because they are virtually free. Today, a one-on-one asks Jev a few dozen questions and a case interview a few hundred. There's a lot of room to improve the coaching experience by adding more frameworks and things to evaluate. Luckily, the ceiling is really high! Drag the sliders to see.

Before, I used big prompts to decide what mattered, measure it and write about it all at once. It was impressive, the LLMs are very good at noticing things we're blind to. But there were all sorts of issues. I thought it was a magical experience before, but now that I'm using Jev it's clear that I was misusing it and missing a key piece.

Now I have a tool that costs virtually nothing, is insanely fast, and helps the LLM do a much better job. I'm able to use code to turn the answers into what gets coached. The LLM gets a chapter, the judgments that fired and a short brief, and does what it does best: write. Then I fact check it with Jev.

Each piece does the job it's best at. The app gets more predictable, faster and cheaper, and the writing gets better because the model has less to juggle.

I think this is going to lead to much, much better applications. I don't know if Typesafe will live up to its impressive valuation, but as a tech enthusiast I can't wait to see what people build with these new tools.

Thanks to Reed Shea, Buford Taylor, Anthony Roldan, Roberto Iturralde, Jeff Dwyer, Adam Fishman, and Roy Hodgman for their feedback on this post.