End-to-End Manual── One worked example, checking wording in Word documents, from idea to maintenance

The whole flow

This manual moves through the 11 stages below in order. For each stage it shows what to decide and what was decided, applied to one example: matching Word documents against an in-house style list of terms and producing a list of candidates to fix.

1. Idea
2. Problem definition
3. Decomposition
4. Data shape
5. Algorithm
6. Instructions for Copilot
7. Choosing a runtime
8. Running
9. Testing and verification
10. Operation and automation
11. Maintenance

1. Idea — find the repeated work

The idea starts from finding the part of a weekly manual task that a rule can decide. In the example, drafts of proposals and web articles are checked by eye against an in-house style list: terms to avoid, terms to rephrase and spelling variants. With many documents, things get missed and it takes time.

Four questions decide whether the task suits automation.

All four hold, so the design is: Python does the matching and people only make the decisions. Whether a term is actually inappropriate in its context is not left to the machine.

Reading and fixing a single document can be done with Copilot in Word (Word in the practical edition). Python is used here to check many documents against the same list with the same criteria every time, and to keep the result as a table.

2. Problem definition — write down input, output and what counts as a pass

Before building anything, write on one page what goes in, what comes out, what counts as a pass and what is out of scope. Asking Copilot while this is vague leads to more rereading and rework.

3. Decomposition — split into small steps and name them

Split the task into small steps that can each be checked on their own, and give each a function name. Named steps line up the instructions to Copilot with the units you test.

A good split is one where each step can be checked alone with a small input. Steps 4 (normalize) and 5 (find_hits) can be tested without opening any document, so they are the core of the tests in stage 9.

4. Data shape — decide what is passed between steps

Decide the shape of the values passed between steps. In the example, one row of the term list is a Term and one finding is a Hit. A Python dataclass fixes the names and number of fields, which makes typos easier to catch. According to the Python documentation, setting frozen to true makes assigning to fields raise an exception after the object is created.

Text taken from a document is paired with its place (paragraph 5, table 1 row 2 col 2 and so on), so that whoever reads the list can find the spot to fix in the original document.

According to the python-docx documentation, Document.paragraphs returns the paragraphs in document order, and Document.tables returns only tables at the top level of the document; a table nested inside a table cell is not included. The example uses these two, so nested tables are out of scope (see Word × Python for reading nested tables).

5. Algorithm — decide the order and the edge cases

1. Read the term list and turn each term into its normalized form (key)
2. Open the .docx files in the folder in name order, skipping files that start with "~$", which Word creates while a document is open
3. Take the paragraphs in order, then the table cells in order
4. Split into sentences at sentence-ending marks (。！？!?)
5. Normalize each sentence and check whether it contains each term's key
6. Collect what is found as Hit values and write them to CSV

Spelling variants are evened out with Unicode normalization form NFKC. The Python unicodedata documentation describes NFKC as one of the normal forms based on compatibility equivalence. Full-width "Ｎｏ．１" becomes "No.1" under NFKC (checked by a test in stage 9). Upper and lower case are evened out with casefold.

The regular expression (?<=[。！？!?]) splits right after a sentence-ending mark, using a lookbehind. The Python re documentation says the pattern inside a lookbehind must only match strings of some fixed length; a set of single characters meets that condition.

Edge case: "No.10" contains "No.1", so it becomes a candidate. That is an allowed extra under stage 2, so it is reported and a person removes it. Adding conditions on the surrounding characters would cut extras but could create misses, so check any such change against the pass condition first. If the term list grows a lot, measure the run time before changing the method.

6. Instructions for Copilot — hand over what you decided

Write the input, output, pass condition and function split decided so far straight into the instructions. The same wording works in Copilot Chat in Microsoft 365 and in GitHub Copilot.

Check the code you receive in this order: (1) do the functions and arguments it uses exist in the official documentation (compare with the python-docx API docs); (2) does anything write to the input files; (3) run it on a few samples and compare with a count done by hand (stage 9). Do not use it on work documents until these checks are done. Instruction patterns are covered in more detail on From Instructions to Code.

7. Choosing a runtime — decide where it runs

For Python in Excel, Microsoft's documentation states that the Python code has no access to your computer, devices or account, and no network access, so it does not suit this example, which reads Word files (see Choosing Where Code Runs for the comparison).

Locally, create a virtual environment per project and install libraries into it. According to the Python documentation, a virtual environment is created by running the venv module and activated in PowerShell with Scripts\Activate.ps1, which may require setting the execution policy. Record the installed versions in requirements.txt with pip freeze.

Run in PowerShell from the project folder

python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install python-docx==1.2.0
python -m pip freeze > requirements.txt

8. Running — the finished program and its output

This is the finished program. The output from running it on three sample documents (doc_a.docx, doc_b.docx and e2e_doc_c.docx) with the term list is shown under Output below.

check_terms.py

"""Check .docx files in a folder against a style list and write the hits to a CSV file.

usage: python check_terms.py DOCS_FOLDER TERMS_CSV OUT_CSV
"""
import csv
import re
import sys
import unicodedata
from collections import Counter
from dataclasses import dataclass
from pathlib import Path

from docx import Document

SENTENCE_END = re.compile(r"(?<=[。！？!?])")


@dataclass(frozen=True)
class Term:
    term: str
    category: str
    action: str
    key: str


@dataclass(frozen=True)
class Hit:
    file: str
    where: str
    term: str
    action: str
    sentence: str


def normalize(text):
    # NFKC turns full-width letters and digits into their usual forms; casefold ignores case
    return unicodedata.normalize("NFKC", text).casefold()


def load_terms(path):
    with open(path, encoding="utf-8-sig", newline="") as f:
        return [Term(r["term"], r["category"], r["action"], normalize(r["term"]))
                for r in csv.DictReader(f) if r["term"].strip()]


def iter_blocks(doc):
    for i, p in enumerate(doc.paragraphs, 1):
        yield "paragraph %d" % i, p.text
    for ti, table in enumerate(doc.tables, 1):
        for ri, row in enumerate(table.rows, 1):
            for ci, cell in enumerate(row.cells, 1):
                yield "table %d row %d col %d" % (ti, ri, ci), cell.text


def split_sentences(text):
    return [s.strip() for s in SENTENCE_END.split(text) if s.strip()]


def find_hits(name, where, sentence, terms):
    key = normalize(sentence)
    return [Hit(name, where, t.term, t.action, sentence) for t in terms if t.key in key]


def check_folder(folder, terms):
    hits = []
    for path in sorted(Path(folder).glob("*.docx")):
        if path.name.startswith("~$"):
            continue  # lock files that Word leaves while a document is open
        for where, text in iter_blocks(Document(path)):
            for sentence in split_sentences(text):
                hits += find_hits(path.name, where, sentence, terms)
    return hits


def write_csv(hits, out):
    with open(out, "w", encoding="utf-8-sig", newline="") as f:
        w = csv.writer(f)
        w.writerow(["file", "where", "term", "action", "sentence"])
        w.writerows([h.file, h.where, h.term, h.action, h.sentence] for h in hits)


def main(folder, terms_csv, out_csv):
    hits = check_folder(folder, load_terms(terms_csv))
    write_csv(hits, out_csv)
    print("hits:", len(hits))
    for name, n in sorted(Counter(h.file for h in hits).items()):
        print(" ", name, n)
    for h in hits[:8]:
        print("  %s | %s | %s -> %s" % (h.file, h.where, h.term, h.action))


if __name__ == "__main__":
    main(*(sys.argv[1:] or ["samples", "samples/style_terms.csv", "out_hits.csv"]))

hits: 11
  doc_a.docx 7
  e2e_doc_c.docx 4
  doc_a.docx | paragraph 5 | 最高 -> 根拠を確認する
  doc_a.docx | paragraph 5 | 業界初 -> 根拠を確認する
  doc_a.docx | paragraph 5 | 絶対 -> 言い換える
  doc_a.docx | paragraph 6 | No.1 -> 調査の出典・時点・範囲を併記する
  doc_a.docx | paragraph 7 | 下さい -> 「ください」に統一
  doc_a.docx | paragraph 7 | 問合せ -> 「問い合わせ」に統一
  doc_a.docx | table 1 row 2 col 2 | 完全無料 -> 条件を併記する
  e2e_doc_c.docx | paragraph 2 | No.1 -> 調査の出典・時点・範囲を併記する

Reading the output: there are 11 candidates in all. The 7 in doc_a.docx match a count done by hand (stage 9). doc_b.docx has none, so it does not appear in the per-file counts. The 4 in e2e_doc_c.docx include the full-width "Ｎｏ．１", the allowed extra "No.10", "唯一" inside a table cell, and the variant spelling "出来る".

9. Testing and verification — check that it runs and that it is right, separately

Running without errors and being right (reporting what should be reported, and nothing that should not) are checked separately, in three ways.

1. Unit tests: try normalize, split_sentences and find_hits on small inputs without opening documents
2. Known-answer test: compare the program's count with a count done by hand; doc_a.docx has 7 by hand
3. Edge-case test: a sample made on purpose (e2e_doc_c.docx) with full-width characters, a term inside a table cell, and "No.10" as an allowed extra

With Python's unittest, tests are methods whose names start with test, in a class that subclasses unittest.TestCase (per the Python documentation).

test_check_terms.py (placed next to check_terms.py)

"""Tests for check_terms.py (run: python test_check_terms.py)."""
import sys
import unittest

from check_terms import Term, check_folder, find_hits, load_terms, normalize, split_sentences

TERMS = [Term("No.1", "順位の表現", "出典を併記する", normalize("No.1")),
         Term("下さい", "表記ゆれ", "「ください」に統一", normalize("下さい"))]


class CheckTermsTest(unittest.TestCase):
    def test_full_width_matches(self):
        hits = find_hits("a.docx", "paragraph 1", "満足度Ｎｏ．１です。", TERMS)
        self.assertEqual([h.term for h in hits], ["No.1"])

    def test_rewritten_sentence_has_no_hit(self):
        self.assertEqual(find_hits("a.docx", "paragraph 1", "ご連絡ください。", TERMS), [])

    def test_sentences_are_split_after_the_mark(self):
        self.assertEqual(split_sentences("一文目。二文目！三文目"), ["一文目。", "二文目！", "三文目"])

    def test_known_false_positive_stays_visible(self):
        # "No.10" contains "No.1". It is reported on purpose; a person removes it from the list.
        self.assertEqual(len(find_hits("a.docx", "paragraph 1", "No.10 を参照。", TERMS)), 1)

    def test_known_answer_on_sample_document(self):
        hits = check_folder("samples", load_terms("samples/style_terms.csv"))
        self.assertEqual(sum(1 for h in hits if h.file == "doc_a.docx"), 7)

    def test_table_cell_is_checked(self):
        hits = check_folder("samples", load_terms("samples/style_terms.csv"))
        self.assertTrue(any(h.file == "e2e_doc_c.docx" and h.where.startswith("table") and h.term == "唯一"
                            for h in hits))


if __name__ == "__main__":
    suite = unittest.defaultTestLoader.loadTestsFromTestCase(CheckTermsTest)
    result = unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)
    sys.exit(0 if result.wasSuccessful() else 1)

test_full_width_matches (__main__.CheckTermsTest.test_full_width_matches) ... ok
test_known_answer_on_sample_document (__main__.CheckTermsTest.test_known_answer_on_sample_document) ... ok
test_known_false_positive_stays_visible (__main__.CheckTermsTest.test_known_false_positive_stays_visible) ... ok
test_rewritten_sentence_has_no_hit (__main__.CheckTermsTest.test_rewritten_sentence_has_no_hit) ... ok
test_sentences_are_split_after_the_mark (__main__.CheckTermsTest.test_sentences_are_split_after_the_mark) ... ok
test_table_cell_is_checked (__main__.CheckTermsTest.test_table_cell_is_checked) ... ok

----------------------------------------------------------------------
Ran 6 tests in 0.114s

OK

Run the tests every time the term list or the program changes, and whenever a library version is raised.

Remaining limits: text in headers, footers and text boxes is not read (out of scope). According to the python-docx documentation, paragraphs inside revision marks (tracked insertions and deletions) do not appear in Document.paragraphs, so check documents after the changes have been accepted.

10. Operation and automation — run on a schedule and keep a record

Once local runs give stable results, run it on a schedule. On a local Windows PC, register it with Task Scheduler. According to the schtasks create documentation, a weekly schedule requires /sc weekly, days of the week can be set with /d, the start time /st uses the 24-hour HH:mm format, and /tr names the program or command to run.

Run run.cmd (which calls check_terms.py with the virtual environment's python) every Monday at 08:00

schtasks /create /sc weekly /d MON /st 08:00 /tn "TermCheck" /tr "C:\tools\term-check\run.cmd"

To share it with a team, or when the documents are in SharePoint, run it with an Azure Functions timer and fetch documents with Microsoft Graph. According to the Azure Functions documentation, the schedule is an NCRONTAB expression ({second} {minute} {hour} {day} {month} {day-of-week}) and the default time zone is UTC; its example "0 30 9 * * 1-5" means 9:30 AM every weekday.

function_app.py (skeleton started by the timer)

"""Run the term check at 9:30 every weekday (Azure Functions, Python v2 model, timer trigger)."""
import logging
import os

import azure.functions as func

app = func.FunctionApp()


# {second} {minute} {hour} {day} {month} {day-of-week}. The default time zone is UTC;
# the Azure Functions docs describe the WEBSITE_TIME_ZONE app setting for other time zones.
@app.timer_trigger(schedule="0 30 9 * * 1-5", arg_name="timer", run_on_startup=False)
def weekday_term_check(timer: func.TimerRequest) -> None:
    if timer.past_due:
        logging.warning("The timer is past due")
    folder = os.environ.get("TERM_CHECK_FOLDER", "")
    logging.info("term check started (folder setting: %s)", folder or "not set")
    # Fetch the changed documents with Microsoft Graph and call check_folder() here
    # (see "Check only documents that changed" in the workflow recipes).

Each run should record when it ran, how many documents it read and how many candidates it found. The logging.basicConfig(filename=..., level=logging.INFO) form from the Python logging documentation writes this to a file. Keep document text and personal information out of the record; file names and counts are enough. For sending the result, see Teams × Python.

11. Maintenance — watch what changes and rerun the tests

This section's automatic collection watches python-docx releases on GitHub every day. When a release changes how the library is used, a tip is added to the Word × Python page.

Set a review date in advance (for example every six months). At review, look at how many candidates people removed from the list and adjust the term list and the rules accordingly.