Kannada-English cyberbullying detection
A privacy-conscious Kannada-English offensive-language classification study with transparent baselines, calibration, selective review, script analysis, and robustness testing.
Project definition
Problem statement
Kannada-English comments can combine Kannada script, Roman transliteration, English, spelling variants, and limited context. An overall accuracy value can hide severe errors on rare offensive categories.
The engineering problem is to compare transparent text models on an official split, measure probability quality and language-form errors, and expose uncertain cases for review without retaining raw comments in project evidence.
Project objectives
- Verify and reproduce the official DravidianCodeMix train, development, and test split.
- Replace common direct identifiers before text enters the model pipeline.
- Compare three transparent classical text-classification candidates.
- Fit calibration and select a review threshold using development data only.
- Measure per-class, script-form, length, calibration, and robustness behaviour.
Project structure
Project components
Dataset loader
Downloads the official Zenodo archive, verifies its MD5, parses the three published splits, and rejects unexpected labels.
Privacy processor
Replaces URLs, email addresses, username mentions, and Indian telephone patterns before modelling and stores only hashed prediction records.
Model comparison
Compares character TF-IDF logistic, hybrid character and word logistic, and character Complement Naive Bayes pipelines.
Calibration module
Fits temperature scaling and a confidence threshold using held-out development probabilities.
Evaluation module
Calculates overall, per-class, calibration, selective-review, script-form, length, and collapsed binary diagnostics.
Robustness module
Measures sensitivity to punctuation removal, reduced character elongation, and Roman-vowel deletion.
Methodology
Project workflow
- 01Verify the archive
The source ZIP is matched against the checksum published for the retained experiment.
- 02Redact and parse
Direct identifier patterns are replaced in memory and official labels and split boundaries are preserved.
- 03Select the model
All candidates train on the same data and the winner is chosen by development macro F1.
- 04Calibrate and set review
Development probabilities fit the temperature and selective-review threshold.
- 05Evaluate once
The frozen pipeline produces test metrics, hashed audit rows, slices, perturbation results, and report figures.
Demonstration scenario
The experiment verifies and parses the official Kannada-English archive, redacts direct identifier patterns, compares three models on the development split, calibrates the selected character model, and applies the frozen pipeline to 777 test comments. The retained figures show the confusion matrix, reliability, class recall, script-form errors, robustness, and the review tradeoff without displaying raw comments.
Engineering
Tools and method
- Runtime
- Python 3.12 with an installable command-line package and exact dependency versions.
- Text representation
- Character and word TF-IDF pipelines designed for noisy code-mixed spelling and script variation.
- Models
- Class-weighted logistic regression and Complement Naive Bayes through scikit-learn.
- Probability analysis
- Multiclass log loss, Brier score, expected calibration error, temperature scaling, and risk-coverage curves.
- Evidence
- CSV and JSON results plus ten figures in PNG and SVG formats without raw comment text.
- Quality controls
- Fifty-three tests, Ruff, dependency audit, repository validation, Docker verification, and document accessibility audits.
Testing
Evaluation
Evaluation measures
- Accuracy, macro F1, weighted F1, and per-class precision and recall
- Log loss, multiclass Brier score, and expected calibration error
- Automatic coverage, accepted accuracy, and review workload
- Performance across native-script, Roman-only, mixed-script, and length slices
- Prediction stability under three controlled text transformations
- Automated tests, statement coverage, dependency audit, and Docker verification
Project boundaries
- The source task identifies offensive-language labels in individual comments and does not establish complete cyberbullying behaviour.
- Pattern redaction reduces direct-identifier exposure but cannot guarantee removal of every indirect identifier.
- The retained six-class macro F1 is 49.00 percent, so the project is not ready for autonomous moderation.
- The benchmark does not establish intent, repetition, identity, legal violation, or platform-wide fairness.
Included
- 01Installable Python package and command-line dataset fetcher
- 02Official 6,217-row training, 776-row development, and 777-row test split
- 03Character, hybrid, and Complement Naive Bayes model comparison
- 04Held-out temperature scaling and selective-review analysis
- 05Script-form, length, per-class, and controlled robustness evidence
- 06Hashed prediction audit without retained raw comment text
- 0753 automated tests with 97 percent statement coverage
- 08Complete source code in a private GitHub repository
- 09103-page project report in PDF and editable Word formats
- 1020-page setup and usage guide in PDF and editable Word formats
- 1155 annotated references and three attributed literature images
Project record
No information is collected on this page.
- Permanent project ID
- GP-DA-15A5VFS
- Catalogued
- 21 Aug 2026
- Completed
- 27 Aug 2026
- Verified
- 29 Aug 2026
- Demonstration
- Included in repository
Handover
After purchase
- 01Payment is confirmed
The project is marked unavailable and cannot be purchased again.
- 02Repository access is granted
The buyer's submitted GitHub account receives access to the private repository.
- 03The purchase record is delivered
The certification sheet is prepared from the reviewed buyer details and sent privately by email.