How We Built Study Goat's Syllabus Engine: 94% Accuracy Across 200 Documents
Every professor writes a syllabus differently. One delivers a clean table of assignments and weightings, another writes twelve pages of prose with the grading breakdown buried in paragraph six, a third hands out a scan of a document last revised years ago. Study Goat needed consistent structured data from all of them. We built a GPT extraction engine with a tuned prompt layer that pulls instructor details, course specifics, participation criteria, and grading breakdowns into one fixed schema, then runs the weighting math and generates calendar events, reaching 94% field-level accuracy across a 200-syllabus test set.
Study Goat turns a course syllabus into usable academic data. The premise is sound and slightly unusual: rather than asking students to enter their courses, deadlines, and grading weights manually, take the document they were already given and read it for them.
The syllabus is the single artifact every student receives at the start of every course. It contains the instructor’s contact details, the participation rules, the full grading breakdown, and every deadline for the term. Almost nobody extracts value from it, because reading it carefully and doing the arithmetic is work that gets postponed until the moment it matters.
Study Goat came to us with the product concept and the harder half of the problem already identified. The value was obvious. Getting consistent data out of an inconsistent document was the entire build.
Every course has a syllabus. No two are built the same way.
Format variance here is the normal condition, and no main path exists that handles it later. Structured tables, unstructured prose, scanned PDFs of documents nobody has revised in a decade, and every combination in between arrive in roughly equal proportion.
So extraction that works on structured syllabi and fails on prose fails for a large share of courses, and it fails badly. A student who uploads their syllabus and gets a wrong grading weight never concludes that this particular document was difficult. They conclude the tool does not work, and they stop using it.
Rule-based extraction cannot solve that. Writing rules requires anticipating document structures, and the structures here are effectively unbounded because every instructor invents their own. Any rule set is a list of the formats someone thought of, and students will upload the ones nobody did.
A language model reading for meaning handles formats nobody anticipated, which is exactly the requirement. That moved the engineering work from writing parsers to designing how the model reads.
The framing that made it work was instructing the model to read the way a teacher reads. A teacher looking at any syllabus finds the grading breakdown wherever it sits, because they know conceptually what they are looking for. We built the prompt layer around that idea, and around nothing to do with document structure.
Schema consistency came before extraction breadth. Both downstream features depend on receiving data in a fixed shape, and richer data in an inconsistent structure is unusable by everything reading it. So the schema was locked first, and the extraction was built to fill it.
Within that schema sit four targets: instructor details, course specifics, participation criteria, and grading breakdown, each pulled into defined fields wherever they appear in the source and however they are expressed.
The grading data feeds a calculation layer that applies the weightings and computes where a student currently stands and what they need on remaining assessments.
The extracted course schedule generates .ICS events. Date and recurrence parsing had to survive the same format variance as everything else, since a schedule expressed as prose is as common as one expressed as a table.
Generated files import directly into Google Calendar, Outlook, and Apple Calendar without conversion or a platform-specific export path.
Every component served turning one unstructured document into a term's worth of structure.
We tuned the prompt layer against the syllabi most likely to fail, meaning the prose-heavy documents and the scans.
That choice determines what the accuracy figure actually means. A test set of well-formatted syllabi produces a higher headline number and a product that breaks the first time a student uploads a photograph of a whiteboard. The 94% figure is measured across 200 documents spanning the full format range, which makes it the number that predicts behavior in production. Scans and prose pull the average down, and that is the point. They are in the average because they are in the real distribution.
Schema consistency stayed the pass condition on every iteration. Neither downstream feature could be built against a moving target, so extraction had to be returning a stable shape before the grade calculation or the calendar generation existed. That sequencing cost time upfront and removed an entire class of rework later.
The failure mode we designed against was confident wrongness. A document the engine cannot read is a handleable case, because the interface can ask the student to check a field. A document the engine reads incorrectly, producing a grading weight that looks plausible and is wrong, damages trust in a way no interface recovers from. Field-level confidence was built into the extraction from the start.
One component carried disproportionate difficulty, and it was not the one we expected. A grading breakdown expressed in prose is still a set of named items with percentages. A schedule expressed in prose is a set of relative references, holiday exceptions, and recurring patterns stated in natural language, and all of it has to resolve to specific dates that import cleanly into three different calendar applications.
A student uploads their syllabus and gets back the grading breakdown, what each component is worth, where they currently stand, and a calendar file carrying every deadline into whichever calendar they already use.
The document they would have skimmed once and filed becomes the structure for the term. The arithmetic that most students do for the first time the week before a final is available from day one.
New extraction fields join the same schema and the downstream features read from it unchanged, so the engine can grow without a rebuild. That is the return on locking the schema early. Adding a field touches the extraction and nothing downstream.
"The system AppVerticals built extracts instructor details, course specifics, participation criteria, and grading breakdowns from syllabi that vary considerably in format. It applies the weighting mathematics to the grading data and generates .ICS events from the course schedule. The team handled the prompt engineering well and we would engage AppVerticals again."
We have built extraction that holds a fixed schema across prose, tables, and scans, measured on the documents most likely to break it. We know what this takes.
Start the conversation