Expense Module

Reading Handwritten Grocery Slips Into a Household Ledger

Solo Engineer · Next.js + Vision LLM · May – Oct 2026

Overview

Household expense tracking built as a second sub-app inside the same private family application as the tenant manager. Expenses are logged against a catalogue of items and categories, then rolled up into monthly summaries with category breakdowns and analytics.

The feature worth talking about is Scan slip. Household spending in this family is written on paper in a running list — Hinglish item names, day-first dates, several days per page. Rather than asking anyone to retype that, you photograph the page and a vision model reads it into draft rows you correct before saving.

Nothing from the photograph reaches the database directly. Every line lands on a review screen first, uncertain readings are flagged, and the slip's own written total is checked against the sum of the parsed lines.

The public demo has two years of invented shopping, bills, and wages across categories like Sabzi and Kirana, always ending today, and slip reading works there too: photograph any handwritten list and review the rows it drafts.

Next.js 16 React 19 TypeScript Claude API Gemini API Supabase Zod PWA

Problem & Solution

The Problem

Spending recorded on paper in handwritten running lists
Item names written in Hinglish, spelled inconsistently
Day-first dates a model will misread as month-first
One page often spans the end of a month
Typing a week of entries by hand is why nobody does it

The Solution

Photograph the page; a vision model drafts the rows
The catalogue is handed to the model so names match
The date convention is stated explicitly in the prompt
Each line keeps its own date and lands in its own month
A review screen stands between the photo and the database

Key Highlights

Catalogue Grounding
The reader is handed the household's existing item catalogue, so it answers in the family's own Hinglish spellings and links lines back to existing items instead of starting parallel spending histories under new names

Date Convention
The prompt states the day-first Indian convention explicitly, so a slip reading 3.7.26 is parsed as the third of July rather than the seventh of March

Per-Line Dates
A photographed page is usually several days of spending, so each line keeps its own date and the sheet groups by day — a page spanning a month boundary lands correctly in both months rather than being flattened into one

Verification Layer
Uncertain readings are flagged for attention and the slip's handwritten total is reconciled against the parsed lines, turning a silent misread into a visible discrepancy

Basket Entry
Manual logging is basket-shaped — one date and payment method covering many lines — so typing a shop visit stays fast for anyone who would rather not use the camera at all

Camera Deep Link
Installed as a progressive web app with its own manifest, and a dedicated scan route can be saved as its own home-screen icon that opens straight to the camera

Both Sides of the Page
A slip written on front and back is photographed as up to four pictures and read together as one document, so a date written on the front carries onto the back

Actionable Insights
Insights notice a missing bill, a lapsed regular purchase, or an item logged in two units, and each suggestion carries the button that resolves it on the same row

Technical Approach

Supported two vision providers behind one interface, preferring Claude when its key is present because it reads the handwriting more accurately and does not train on inputs, with Gemini as the free fallback
Documented the privacy trade-off honestly in the setup instructions: the free Gemini tier may use submitted images to improve their products, and every scan is a photograph of household spending
Made the scan feature entirely optional — with no key configured it returns a clear error and the rest of the application is unaffected, because the manual entry sheet was built to be fast on its own
Treated model output as a draft rather than a result, writing nothing to the database until a human confirms it on a review screen
Metered slip reading on the public demo, where the PINs are public and stop nobody: a few reads per visitor an hour, a few more a day, and a shared daily ceiling sized to the free tier, with visitors identified by a keyed hash so no IP address is stored
Gated the scan endpoint behind a hard server-side authentication check, since each call both reads back household spending and spends money at a paid API
Kept every derivation — category classification, summary math, percentage breakdowns, analytics — in pure server-side library modules with their own unit tests, rather than computing in components
Split the logic into focused, individually tested modules for expenses, items, categories, summaries, and analytics, so a change to reporting cannot quietly break entry
Gave the module its own web app manifest separate from the parent hub, so it installs as a standalone icon on a phone
Applied the same two-tier PIN model as the rest of the app: locked viewers see real monthly totals and category breakdowns but an empty line-item list

What This Demonstrates

Practical AI Integration: Used a vision model where it genuinely removes drudgery, and designed the surrounding system on the assumption that it will sometimes be wrong
Prompt Engineering as Design: Grounded the model in the household's own catalogue and stated the date convention explicitly, fixing two whole classes of error in the prompt rather than in post-processing
Human-in-the-Loop by Default: Put a mandatory review step between model output and persisted data, with confidence flags and a total reconciliation to direct attention
Honest Privacy Trade-offs: Wrote down what a free tier does with submitted images so the person enabling it can decide, instead of burying it
Graceful Optionality: Built the AI path as an accelerator on top of a manual path that stands alone, so the feature degrades rather than breaks
Closing the Loop: Made every insight end in an action — a pre-filled entry that still needs a second tap to save, because a wrong save cannot be undone without the admin PIN
Domain Modelling: Recognised that a photographed page is a multi-day document, not a single receipt, and modelled it that way from the start