AI is great at writing first drafts. But if someone asks, "is this faithful to the source?" — few of us can answer with confidence. Phrase Structure Grammar (PSG), a linguistic framework, lets us break sentences into structural units and mechanically compare them with the source's units. This lecture walks you through building exactly that verification prompt — step by step, with your own AI, in 10 stages. This is a hands-on lecture: don't just read, do.
What you'll learn
- How to check whether AI output is actually consistent with its source
- A minimal working use of phrase structure grammar (PSG)
- How to design a reusable "verification prompt" template
- How to apply it to real workflows: summaries, promotional materials, regulatory documents
Why phrase structure grammar?
Phrase structure grammar is a linguistic framework, codified by Noam Chomsky in the mid-20th century, that breaks sentences into structural units like "noun phrase (NP)", "verb phrase (VP)", "object phrase (NP)". By cutting a sentence into blocks, you can explicitly extract "who did what to what".
For fact-checking, reading a whole paragraph and asking "is this right?" is hard. AI struggles with it too. But if you decompose at the phrase level, you can do mechanical comparison. The sentence "Drug A's efficacy rate is 80%" decomposes to [NP Drug A's efficacy rate] [VP is 80%]. Compared with the source's same decomposition, you can match subject, value, and relationship one slot at a time.
This isn't unique to PSG — dependency grammar and semantic role labeling do the same thing. We use PSG here because it's the easiest entry point.
Why this matters in pharma
Pharma documents — package inserts, promotional materials, conference slides, formal responses — must trace to a source (primary literature). AI is convenient for drafting, but unless a human verifies consistency with the source, what you have is a regulatory risk dressed as a draft.
The prompt techniques in this lecture let an AI semi-automate the consistency check. Final judgment stays human, but 70–80% of the comparison work can be mechanized.
What you'll need: one AI of your choice (Claude / ChatGPT / Gemini / Copilot — any). Open a fresh chat. For each STEP below, copy the PROMPT block into your AI, observe the output, and proceed.
STEPs 1–10 — work through them in order
Step 01Define the source and the target document
First, set the material. We'll use this fictional clinical trial summary as the source:
Source: A fictional Phase III trial.
── Drug A was administered to 240 patients (ages 18–65) with moderate asthma. At 12 weeks, FEV1 improvement rate was 18.4% (95% CI: 15.2–21.6). Main adverse events: cough (8.3%), headache (5.0%). No serious adverse events reported.
The goal: produce a 200-character summary for healthcare professionals based on this source.
▶ Your action
Paste the following into your AI to set the context:
We will use phrase structure grammar (PSG) to build a 200-character summary that is fully consistent with a source. Memorize the following as the source:
【Source】
A fictional Phase III trial. Drug A was administered to 240 moderate-asthma patients (18-65 years). At 12 weeks, FEV1 improvement rate 18.4% (95% CI: 15.2–21.6). Main adverse events: cough 8.3%, headache 5.0%. No serious adverse events.
【Goal】
Produce a 200-character summary for healthcare professionals and verify its phrase-level consistency with the source.
Reply with just "Acknowledged." We'll proceed step by step.
Expected: AI replies briefly with "Acknowledged". If your AI launches into a long preamble, its instruction-following is weak — try a different one.
Step 02Extract the source's claims as bullet points
To check consistency, we first need a list of the source's claims.
▶ Your action
【Extract source claims】
From the source above, extract each independent claim in "subject + predicate + value (if any)" form, as bullet points.
No interpretation, no paraphrasing. Use the source's exact wording.
Output format:
- Claim 1: ...
- Claim 2: ...
- Claim 3: ...
(continue)
Expected: 5–7 independent claims as a list. E.g., "Claim 1: Drug A was administered to 240 moderate-asthma patients", "Claim 2: At 12 weeks, FEV1 improvement rate was 18.4%", etc.
Self-check: Is the number of claims roughly equal to the number of sentences in the source? If it's much fewer, the AI is summarizing.
Step 03Decompose each claim with phrase structure
Now we structure the claims via PSG. This becomes the "answer key" for comparison.
▶ Your action
【PSG decomposition of source】
Decompose each claim from Step 2 in the following format:
Format:
{
"id": "Claim 1",
"original": "(the claim)",
"NP_subject": "(subject NP)",
"VP_verb": "(verb phrase)",
"NP_object": "(object NP or complement)",
"values": ["(all numerical values, if any)"],
"modifiers": ["(relative clauses, prepositional phrases, etc.)"]
}
Return as a JSON array.
Expected: A JSON array. For Claim 2, you'd see something like {"NP_subject": "FEV1 improvement rate at 12 weeks", "VP_verb": "was", "NP_object": "18.4%", "values": ["18.4", "15.2", "21.6"], "modifiers": ["95% CI: 15.2–21.6"]}.
This is your "answer key" for the consistency check.
Step 04Have the AI draft the target document (200-char summary)
Now ask for the summary. You could just ask, but for easier verification, specify "short" and "use source wording" explicitly.
▶ Your action
【Draft creation】
Produce a 200-character summary for healthcare professionals, based on the source.
Constraints:
- Include all numerical values and conditions from the source
- Add no information that is not in the source
- Use the source's wording where possible
Output only the summary.
Expected: ~200-character summary. E.g., "In a Phase III trial, Drug A was administered to 240 moderate-asthma patients; at 12 weeks, FEV1 improvement was 18.4% (95% CI: 15.2–21.6). Main AEs: cough 8.3%, headache 5.0%. No serious AEs."
Watch out: If the AI sneaks in phrases like "demonstrated superior efficacy", they will fail later checks.
Step 05Extract the draft's claims the same way
Now make a "claim list" from the draft. Same procedure as Step 2.
▶ Your action
【Extract draft claims】
From the summary created in Step 4, extract independent claims as bullet points, in the same format as Step 2.
Output format: same as the source extraction.
Expected: A list of the draft's claims.
Self-check: Is the count of claims similar to Step 2's source list? If it's very different, the draft has dropped or added information.
Step 06PSG-decompose the draft
Same as Step 3, but for the draft.
▶ Your action
【PSG decomposition of draft】
Decompose each claim from Step 5 in the same JSON format as Step 3.
Expected: The draft's JSON array.
You now have two decomposition tables — source (Step 3) and draft (Step 6). The real work begins next.
Step 07Design the comparison prompt
We'll compare the two decompositions, phrase by phrase. Defining the judgment rule explicitly is the key.
▶ Your action
【Consistency check design】
We will now compare Step 3 (source decomposition) with Step 6 (draft decomposition).
Use these rules:
Rules:
1. For each draft claim, search the source for a "corresponding claim"
2. When found, compare four things:
- NP_subject (subject) is the same
- VP_verb (verb) means the same thing
- NP_object / values match exactly
- modifiers are not dropped or added
3. If any one fails: "Inconsistent". If all four match: "Consistent"
4. Draft claims with no corresponding source claim: "Source overreach (must fix)"
Output format:
| Draft claim ID | Matched source ID | Verdict | Mismatch detail |
|---|---|---|---|
| ... | ... | Consistent / Inconsistent / Source overreach | ... |
Reply with "Rules acknowledged." We'll run the comparison in the next step.
Expected: "Rules acknowledged." (brief).
Key insight: Explicit rules raise judgment reproducibility. This is the core of prompt engineering.
Step 08Run the consistency check
Ready. Run it.
▶ Your action
【Run consistency check】
Following the rules from Step 7, compare Step 3 (source decomposition) and Step 6 (draft decomposition), and output the result table.
Write a concise reason for each verdict.
Expected: A comparison table. If everything is "Consistent", the draft is faithful. Any "Inconsistent" or "Source overreach" rows mark items to fix.
Self-check: Glance at the AI's verdicts once yourself. AIs occasionally misjudge consistency (especially around numeric units).
Step 09Stress-test with edge cases
If the check works on the clean case, deliberately feed it a tampered draft to confirm it catches the changes.
▶ Your action
【Robustness test】
Apply the same procedure (Step 5 → 6 → 8) to the following tampered draft.
Tampered draft:
"In a Phase III trial, Drug A was administered to 240 severe-asthma patients; at 12 weeks, FEV1 improvement was 28.4% (95% CI: 15.2–21.6). Main AEs: cough 8.3%, headache 5.0%, liver injury 2.1%. No serious AEs."
Tampering (for your reference only — don't tell the AI):
- "moderate" changed to "severe" (NP_subject tampered)
- "18.4%" changed to "28.4%" (value tampered)
- "liver injury 2.1%" added (source overreach)
Expected: All three tampers are flagged as "Inconsistent" or "Source overreach".
If it misses any: add rules in Step 7 like "match numeric values character-by-character including units" and "match subject NP including modifiers". This is prompt "debugging".
Step 10Templatize for reuse
Finally, compress everything you did into a "paste-and-go template". Replace just the source and draft slots for any future task.
▶ Your action
【Template】
Compress Steps 1–9 into a single reusable mega-prompt.
Requirements:
- Accept source and draft as {{source}} {{draft}}
- Internally run PSG decomposition and comparison
- Output only the final "Consistent / Inconsistent / Source overreach" verdict table
- Hard-code Step 7's rules (for reproducibility)
Return the completed template as one code block.
Expected: A reusable mega-prompt. Save it. Going forward, drop in any source + AI draft and the check runs.
How to use next time: package insert drafts, promotional taglines, conference scripts — anything with "source + draft" structure works.
Recap — what you've learned
By experience, you now know:
- AI output "feels right" but may not be source-consistent
- PSG decomposition turns "consistency" into something mechanically checkable
- Explicit rules in the prompt sharply increase reproducibility
- Robustness testing (deliberate tampering) is how you "debug" a prompt
- Templates make the workflow portable to other tasks
Where to apply this
Try these tomorrow
- Verify promotional material claims against the package insert
- Check conference slide statements against the source paper's Abstract
- Reconcile regulatory filing drafts with internal SOPs
- Auto-check press-release numbers against the underlying data
- Compare translated material against the original (source) at PSG level
Consistency checking isn't a job to "outsource" to AI — it's a job to do "with" AI. Final judgment stays human. But mechanizing 70% of the comparison frees you to focus on the subtle interpretive differences that actually need a human.
That's what prompt engineering really means: not being used by the tool, but making the tool do correct work. Save the template from this lecture, and try it tomorrow on something real.