Built an Open Source LLM Agent for Personal Finance
Built an Open Source LLM Agent for Personal Finance
Bank transaction descriptions are a mess. "POS DEBIT 03/15 AMZN MKTP US" could mean groceries, office supplies, or a birthday gift. Categorizing hundreds of these manually is tedious. A local LLM with structured outputs handles it surprisingly well.
Structured Outputs Are the Key
The breakthrough was not using a smarter model - it was constraining the output format. Instead of asking the model to "categorize this transaction," you give it a JSON schema with specific categories and force it to return valid JSON.
The model receives the transaction description and amount, and returns a structured object with category, subcategory, confidence score, and a flag for whether it needs human review. Because the output is constrained, parsing is trivial and errors are rare.
Privacy Through Local Processing
Financial data is exactly the kind of information you do not want sending to cloud APIs. Every transaction, every merchant, every amount - that is a complete picture of your life. Running the categorization model locally means none of this data leaves your machine.
A 7B model running on Ollama handles transaction categorization with about 90% accuracy. The remaining 10% gets flagged for manual review. Over time, the model's prompts improve as you correct its mistakes and add examples.
The Workflow
Export transactions as CSV from your bank. The agent reads the file, categorizes each transaction, and generates a spending report grouped by category. It can also compare against previous months, flag unusual charges, and create budget summaries.
The whole thing runs as a desktop automation - the agent opens your bank's website, downloads the latest statement, processes it, and saves the report. No manual steps after the initial setup.
Open sourcing the categorization prompts and schemas lets others adapt it for their banks and spending patterns without starting from scratch.
Fazm is an open source macOS AI agent. Open source on GitHub.