From Paper Ballots to Digital Results: AI-Powered Election Vote Counting

The Problem: Manual Vote Counting Doesn't Scale

Election day ends. The real work begins.

Thousands of physical ballot papers—each with front and back sides, handwritten marks, QR codes, and official signatures—need to be processed, validated, and counted. In a preferential voting system, each ballot carries first, second, and third preference votes, multiplying the complexity.

Manual counting faces critical challenges:

• Speed: Counting thousands of ballots by hand takes days
• Accuracy: Human fatigue leads to miscounts and missed marks
• Fraud prevention: Duplicate ballots and missing signatures are hard to catch at scale
• Audit trail: Paper-based tracking makes recounts and verification painful
• Geographic complexity: Results need to be tracked across regions, provinces, districts, wards, and individual ballot boxes

What if you could photograph each ballot and let AI handle the rest—counting, validation, and reporting?

What We Built: AI-Powered Ballot Processing System

We built a complete election digitization platform for Papua New Guinea's preferential voting system. Photograph ballot papers, and the system automatically reads QR codes, identifies voter selections, verifies signatures, and tallies results—with a full audit trail for every ballot processed.

The Processing Pipeline

How It Works: From Paper to Data

Smart Ballot Pairing

The most clever piece: election ballots have two sides, but operators photograph them in random order. The system:

1. Reads QR codes from every uploaded image
2. Identifies front sides (A-prefix QR) and back sides (B-prefix QR)
3. Automatically pairs matching fronts and backs by serial number
4. Detects and flags duplicates to prevent double-counting

This means operators can photograph a stack of ballots in any order—fronts, backs, mixed—and the system sorts it out.

Multi-Scale QR Detection

Ballot QR codes are often small, smudged, or photographed at angles. Our detection system tries multiple approaches:

• Standard scan at original resolution
• Auto-rotation at 90°, 180°, 270°
• Multi-scale scanning at different zoom levels
• Inversion attempts for low-contrast codes

This cascade approach achieves reliable detection even from poor-quality phone photographs.

AI Vision for Mark Recognition

Once paired, Gemini 2.0 Flash Vision AI analyzes each ballot to identify:

• Checkmarks in candidate boxes
• Filled circles or boxes indicating selection
• Crosses and ticks in various handwriting styles
• Written-in votes that need special handling
• Preference ordering (1st, 2nd, 3rd choice)

The system shows operators a side-by-side view of each extracted field with cropped regions, allowing quick verification before confirmation.

The Technical Edge

Signature Verification Pipeline

Election integrity requires verifying that authorized officials signed each ballot. Our three-stage pipeline:

1. Detection: Roboflow AI locates signature regions on the ballot image
2. Extraction: Sharp image processing crops and normalizes the signature
3. Comparison: MobileNetV2 generates embeddings, and cosine similarity scores match against registered official signatures

This catches unsigned ballots and flags potential forgeries for manual review.

Hierarchical Electoral Geography

Papua New Guinea has a complex electoral structure. The system models the full hierarchy:

Region → Province → District → LLG → Ward → Polling Location

Results roll up through every level, enabling analysis at any granularity—from individual ballot box to national totals. The geography data is imported via CSV and protected as read-only to prevent manipulation.

Complete Audit Trail

Every ballot gets a permanent log entry:

• Original image (front and back)
• QR code data extracted
• AI-detected voter selections
• Validation status and any flags
• Operator who processed it
• Timestamp and ballot box location

This makes recounts straightforward and provides evidence for any disputes.

Results & Reporting

Real-Time Dashboard

As ballots are processed, results update live:

Session Recording

The system supports recording election counting sessions with:

• Audio/video capture of the counting process
• AI-generated transcripts of proceedings
• Text-to-speech narration of results (via ElevenLabs)
• Exportable video summaries with charts and data overlays

Export Capabilities

• CSV export of all vote data for external analysis
• JSON export for integration with national election systems
• PDF reports for official record-keeping
• Chart visualizations for public result announcements

Security & Access Control

Impact

Who This Is For

This solution works for:

• Electoral commissions modernizing vote counting processes
• Government agencies digitizing paper-based workflows with audit requirements
• NGOs and observers monitoring election integrity
• Organizations running internal elections (unions, associations, boards)
• Any entity processing high-volume documents with verification requirements

If you're counting anything on paper and need accuracy, speed, and an audit trail, the same pipeline applies.