← Back to project catalogue
GP-DA-146UVEHData and AIReady

Multimodal document consistency auditor

An offline document analysis project that checks whether text, tables, chart labels, captions, totals, ranges, and trends agree inside a technical PDF report.

Multimodal document consistency auditor project visual
GP-DA-146UVEH · Data and AI
  • Python 3.14
  • PyMuPDF 1.28
  • scikit-learn 1.9
  • ReportLab
  • Matplotlib

Project definition

Problem statement

Technical reports often repeat the same result in narrative text, tables, charts, and captions. Copying or editing one part can leave another part with the wrong value, unit, trend, total, or figure number.

The engineering problem is to find these disagreements in a PDF and show the page and source evidence for every warning.

Project objectives

  • Extract selectable text and table information from technical PDF reports.
  • Convert values and units into comparable claim records.
  • Detect scalar, unit, total, range, trend, and figure-number inconsistencies.
  • Keep the page number and original evidence for every finding.
  • Evaluate the rules on clean reports and reports with controlled faults.

Project structure

Project components

01

PDF extraction

Reads pages, text, and table metadata from born-digital PDF files.

02

Claim matching

Matches similar metric labels using transparent text similarity.

03

Unit handling

Normalises aliases and converts compatible mass, pressure, flow, power, ratio, and temperature units.

04

Consistency rules

Checks values, dimensions, totals, ranges, trends, and figure numbers.

05

Benchmark

Generates 36 labelled technical reports with clean controls and six fault categories.

06

Evidence output

Writes page-level findings, CSV results, JSON summaries, and labelled figures.

Methodology

Project workflow

  1. 01
    Read the PDF

    Open the report locally and extract selectable evidence from each page.

  2. 02
    Build claims

    Store the metric, value, unit, source type, page, and original text.

  3. 03
    Compare evidence

    Match related claims and run the six consistency checks.

  4. 04
    Review findings

    Inspect the page, message, and evidence recorded for each warning.

  5. 05
    Reproduce results

    Generate and audit the complete benchmark with one command.

Demonstration scenario

A technical PDF repeats outlet flow in narrative text and a table. The table value is changed while the remaining report stays consistent. The auditor matches the two claims, converts their units, flags the value conflict, and records both source statements with the page number.

Engineering

Tools and method

Runtime
Python 3.11 or later with an installable command-line package.
Document processing
PyMuPDF page text and native table extraction.
Text matching
Character TF-IDF cosine similarity and sequence similarity.
Rules
Deterministic unit conversion, arithmetic, interval, trend, and identifier checks.
Experiments
ReportLab benchmark generation and Matplotlib evidence figures.
Quality controls
Fifty-seven tests, Ruff, dependency audit, delivery checks, and document accessibility audits.

Testing

Evaluation

Evaluation measures

  • Precision, recall, and F1 across planted faults
  • Exact report-level classification accuracy
  • False positives across six clean controls
  • Performance for each of the six fault categories
  • Automated tests, statement coverage, and dependency audit

Project boundaries

  • The retained benchmark uses born-digital PDFs with an explicit evidence grammar.
  • Scanned pages and raster-only charts require OCR or chart digitisation, which are not included.
  • The benchmark result does not measure performance on arbitrary real-world reports.
  • The tool assists review and does not certify scientific correctness or replace subject experts.

Included

  1. 01Complete Python source code
  2. 0236-report PDF benchmark with ground-truth labels
  3. 03CSV and JSON experiment results
  4. 04Eight labelled result figures
  5. 0557 automated tests with 99 percent statement coverage
  6. 06Complete source code in a private GitHub repository
  7. 0787-page project documentation in PDF and editable Word formats
  8. 0815-page setup and usage guide in PDF and editable Word formats
  9. 0944 annotated references

Project record

No information is collected on this page.

Permanent project ID
GP-DA-146UVEH
Catalogued
21 Aug 2026
Completed
24 Aug 2026
Verified
24 Aug 2026
Demonstration
Included in repository

Handover

After purchase

  1. 01
    Payment is confirmed

    The project is marked unavailable and cannot be purchased again.

  2. 02
    Repository access is granted

    The buyer's submitted GitHub account receives access to the private repository.

  3. 03
    The purchase record is delivered

    The certification sheet is prepared from the reviewed buyer details and sent privately by email.