Building Legal Agents When You Can't See the Data — Jacob Lauritzen, CTO of Legora Why it's worth watching
Key moments The parts the AI thinks are worth your time, chosen from the creator's chapters. Click to jump.
9:18 Testing the agent in simulated worlds and a 10,000-file data room Shows how they test agents without direct trace access, with a real large-scale setup. 10:07 The evals team and Legora’s 100 legal engineers Concrete look at how they organize engineering around evals and legal domain work. 11:03 When evals didn’t seem worth it and one release took citations from 70% to 99% Useful numbers on what improved quality and why evals paid off. 26:12 Product metrics when you can’t read the traces Directly addresses the core constraint and how they measure performance anyway. Chapters From the creator's chapter list. Click to jump.
0:00 Cold open 0:40 Meet Jacob Lauritzen, CTO of Legora 1:05 Starting out as underdogs in a Stockholm basement 1:54 How Legora grew from ChatGPT with Swedish law into a legal agent 4:12 The switch from assistant to agent 5:23 Which models Legora uses and why it added a model picker 7:48 Are today’s models good enough for lawyers? 8:20 What agents couldn’t do nine months ago 9:18 Testing the agent in simulated worlds and a 10,000-file data room 10:07 The evals team and Legora’s 100 legal engineers 11:03 When evals didn’t seem worth it and one release took citations from 70% to 99% 12:18 What legal practice could look like in five to ten years 14:18 Will the law firm partnership model survive? 16:40 What happens to paralegals and junior lawyers? 18:00 The billable hour and why earlier legal tech fell short 19:39 Where the money goes now 22:44 How Legora used its own product for acquisition diligence 23:59 How Swedish and American law firms differ 26:12 Product metrics when you can’t read the traces 27:41 Paraphrasing traces and narrowing failures down to 10 or 20 types 28:42 Why data privacy led Legora to run everything on Azure 30:04 Open-weight models and model arbitrage 30:55 Transcription models and why Legora doesn’t use ElevenLabs 33:33 The agent, Tabular Review, projects, and the Word add-in 34:24 How a lawyer uses Tabular Review 35:51 Why a spreadsheet works and what you learn from the first 10 rows 37:53 How the legal intake bot works in Slack 39:11 How agents create lists and lists start agents 40:15 Building products on those same pieces and acquiring Wexler 42:37 What forward-deployed engineers do at Legora 43:52 How legal engineers and forward-deployed engineers onboard a big client 46:43 What’s next for the forward-deployed team over the coming year 47:24 Why Legora resisted building APIs 49:26 Can the agent be the API? 50:12 The 300 Spartans slide and the plan to stay at 20 engineers 51:32 The bets nobody asked Legora to make 53:05 Post-training open-weight models for legal research 54:05 Search, retrieval, and late interaction 54:51 What makes a good legal agent harness and why drafting depends on search 56:25 Which model writes the best draft? 57:03 Hallucinations and Legora’s citation system 57:41 Do lawyers check the citations? 58:19 MapReduce or one big context window for a million-token contract? 1:00:16 Getting anonymized examples and the idea of buying a law firm for its data 1:01:53 Why Jacob advises against default embeddings 1:02:44 Will lawyers work in Legora or through Claude? 1:03:26 Will the model labs cut Legora off? 1:04:24 Law schools and scholarships for computer science majors 1:05:28 Hiring in New York, Stockholm, and London 1:06:09 Why most work eventually becomes legal work About this video Legora can’t read a single trace of what its agent does for a lawyer. The data belongs to the law firm’s clients. How do you build and test an agent when you can’t see its work?
Jacob Lauritzen is the CTO of Legora, one of the fastest-growing companies in legal tech history. The team started in a Stockholm basement with ChatGPT, Swedish law, and a search tool. Today, a lawyer can use Legora to draft and edit Word documents, review 10,000 contracts in a single table, or run due diligence. The agent also coordinates work between lawyers, clients, and other agents. When Legora made its own acqui
Show more