Deepfake authenticity evaluation studio
A reproducible media-forensics study comparing compact deepfake image detectors under compression, resizing, blur, brightness changes, and mixed reposting.
Project definition
Problem statement
A detector that performs well on original images can become less reliable after compression, resizing, blur, brightness changes, or a mixed reposting pipeline. Raw confidence scores may also be poorly calibrated.
The project measures these effects under fixed, repeatable conditions and uses an uncertain decision region so that weak predictions are not presented as certain conclusions.
Project objectives
- Acquire and verify a balanced public research dataset through a licence-gated workflow.
- Compare metadata-only, forensic logistic, and forensic histogram-gradient detectors on the same fixed splits.
- Measure accuracy, AUROC, calibration error, selective coverage, and accepted-case accuracy.
- Quantify performance changes across seven original and reposting conditions.
- Retain prediction-level audit records and analyse the most important failure patterns.
Project structure
Project components
Dataset acquisition
Downloads the pinned archive only after explicit licence acceptance and verifies its size and SHA-256 digest.
Feature extraction
Calculates 130 colour, gradient, residual, block, and radial-frequency features from each image.
Detector training
Fits three compact classifiers on a deterministic training split and stores reproducible model artifacts.
Calibration
Uses a separate held-out split for temperature scaling and threshold selection.
Robustness evaluation
Applies six controlled transformations in addition to the original condition and records every prediction.
Reporting
Builds metrics, tables, figures, error analyses, and the complete project documentation from retained outputs.
Methodology
Project workflow
- 01Acquire data
Accept the documented non-commercial dataset licence, download the pinned archive, and verify its integrity.
- 02Create splits
Build balanced deterministic training, calibration, and evaluation manifests without retaining source filenames in the committed audit data.
- 03Train models
Extract forensic features and fit the three defined detectors using fixed seeds and configurations.
- 04Calibrate decisions
Fit temperature scaling and define the uncertain region using only the held-out calibration split.
- 05Evaluate conditions
Run all models across the original images and six controlled reposting transformations.
- 06Review findings
Compare metrics, inspect failure groups, and rebuild the figures and documentation from the retained results.
Demonstration scenario
The student runs the three detectors on the held-out evaluation split, compares the original-image results, and then studies the same models after six controlled transformations. The strongest detector reaches an original AUROC of 0.9930, while the report explains the largest observed degradation and the role of calibrated uncertainty.
Engineering
Tools and method
- Experiment runner
- Python command-line workflow with fixed configuration, deterministic splits, structured logs, and cached features.
- Forensic representation
- Engineered colour, texture, residual, gradient, block, and frequency-domain measurements suitable for a resource-conscious engineering study.
- Model layer
- scikit-learn logistic and histogram-gradient classifiers with saved preprocessing and temperature-calibration parameters.
- Evidence layer
- CSV and JSON audit outputs, model artifacts, figures, integrity records, and machine-readable experiment summaries.
- Reproducibility
- Pinned dependencies, Docker support, automated tests, branch coverage, dependency audit, and repository acceptance checks.
Testing
Evaluation
Evaluation measures
- Accuracy, balanced accuracy, AUROC, precision, recall, and F1 score
- Expected calibration error and Brier score before and after temperature scaling
- Selective coverage and accepted-case accuracy at the defined uncertainty threshold
- Metric changes under JPEG compression, resizing, blur, brightness reduction, and mixed reposting
- False-positive and false-negative patterns by model and media condition
- Repeatability of retained results under local and Linux container validation
Project boundaries
- The supplied dataset workflow is restricted to the upstream CC BY-NC 4.0 research licence and requires explicit acceptance.
- The retained mirror does not expose manipulation-family or source-pair metadata, so the study does not claim family-wise or identity-disjoint generalisation.
- The models are research baselines for controlled image experiments, not forensic proof or a production authenticity service.
- The project does not identify people, infer intent, or make legal or disciplinary decisions.
Included
- 01Complete source code in a private GitHub repository
- 02Project documentation in a 74-page PDF and editable Word document
- 03A 16-page setup and usage guide in PDF and editable Word formats
- 04Three trained model artifacts and a reproducible evaluation pipeline
- 05Twelve project figures and 40,488 retained prediction records
- 06Dataset acquisition, integrity checking, feature extraction, calibration, and robustness experiments
- 07Automated tests, container validation, model card, data card, and retained results
Project record
No information is collected on this page.
- Permanent project ID
- GP-CY-0W0GOCJ
- Catalogued
- 21 Aug 2026
- Completed
- 26 Aug 2026
- Verified
- 26 Aug 2026
- Demonstration
- Included in repository
Handover
After purchase
- 01Payment is confirmed
The project is marked unavailable and cannot be purchased again.
- 02Repository access is granted
The buyer's submitted GitHub account receives access to the private repository.
- 03The purchase record is delivered
The certification sheet is prepared from the reviewed buyer details and sent privately by email.