# Firefly III Analytics - Architecture & Data Flow ## System Architecture Diagram ``` ┌─────────────────────────────────────────────────────────────────┐ │ FIREFLY III INSTANCE │ │ (https://firefly.scsimedia.duckdns.org) │ │ │ │ - Transactions │ │ - Categories │ │ - Accounts │ └────────────────────────┬────────────────────────────────────────┘ │ API Requests (Token-based Auth) ↓ ┌─────────────────────────────────────────────────────────────────┐ │ BACKEND (FastAPI) │ │ Running on port 8000 │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ Firefly III API Client (firefly_client.py) │ │ │ │ - Async HTTP requests to Firefly III │ │ │ │ - Token-based authentication │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ │ │ ↓ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ Background Sync Service (sync_service.py) │ │ │ │ - Runs every 30 minutes (APScheduler) │ │ │ │ - Fetches transactions, categories, accounts │ │ │ │ - Stores in SQLite database │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ │ │ ↓ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ SQLite Database (data/app.db) │ │ │ │ - Cached transactions │ │ │ │ - Cached categories │ │ │ │ - Cached accounts │ │ │ │ - Sync logs │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ │ │ ↓ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ API Routers │ │ │ │ ├── /api/transactions (list, filter) │ │ │ │ ├── /api/categories (list) │ │ │ │ ├── /api/accounts (list) │ │ │ │ ├── /api/reports/* (analytics) │ │ │ │ │ ├── /spending-by-category │ │ │ │ │ ├── /income-vs-expenses │ │ │ │ │ └── /trends │ │ │ │ └── /api/summary (dashboard metrics) │ │ │ │ ├── /docs (interactive API docs) │ │ │ │ └── /health (status check) │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────┬────────────────────────────────────────┘ │ JSON REST API (CORS enabled) ↓ ┌─────────────────────────────────────────────────────────────────┐ │ FRONTEND (React) │ │ Running on port 3000 │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ API Service (services/api.ts) │ │ │ │ - Axios client for HTTP requests │ │ │ │ - Manages API calls to backend │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ │ │ ↓ │ │ ┌──────────────────────────────────────────────────────────┐ │ │ │ React Components & Pages │ │ │ │ │ │ │ │ Dashboard (pages/Dashboard.tsx) │ │ │ │ ├── Total Assets Card │ │ │ │ ├── Recent Transactions Card │ │ │ │ └── 30-Day Income/Expenses Card │ │ │ │ │ │ │ │ Reports (pages/ReportsPage.tsx) │ │ │ │ ├── Date Range Picker (components/DateRangePicker.tsx) │ │ │ │ ├── Report Selector │ │ │ │ ├── Spending Chart (components/SpendingChart.tsx) │ │ │ │ │ └── Pie Chart (Recharts) │ │ │ │ └── Trend Chart (components/TrendChart.tsx) │ │ │ │ └── Line Chart (Recharts) │ │ │ └──────────────────────────────────────────────────────────┘ │ │ │ └────────────────────┬───────────────────────────────────────────┘ │ ↓ ┌──────────────┐ │ Web Browser │ │ localhost:3 │ │ 000 │ └──────────────┘ ``` --- ## Data Flow Sequence ``` ┌─────────────────────────────────────────────────────────────────────┐ │ USER INTERACTION │ │ │ │ 1. User navigates to http://localhost:3000 (React app loads) │ │ │ │ 2. Dashboard component mounts │ │ └─→ Calls summaryService.getSummary() │ │ └─→ HTTP GET /api/summary │ │ └─→ Backend calculates metrics from SQLite │ │ └─→ Response: JSON with totals, trends │ │ └─→ Frontend renders cards with metrics │ │ │ │ 3. User clicks on "Reports" tab │ │ └─→ ReportsPage component mounts │ │ │ │ 4. User selects date range (default: last 30 days) │ │ └─→ onPreset() or manual date selection │ │ └─→ Triggers reportService call │ │ └─→ HTTP GET /api/reports/spending-by-category │ │ ?start_date=2024-11-25&end_date=2024-12-25 │ │ │ │ 5. Backend processes report request │ │ └─→ Queries SQLite transactions table │ │ └─→ Filters by date range │ │ └─→ Groups by category │ │ └─→ Sums amounts per category │ │ └─→ Returns JSON: [{category, amount}, ...] │ │ │ │ 6. Frontend receives report data │ │ └─→ Passes to SpendingChart component │ │ └─→ Recharts renders pie chart │ │ └─→ User sees visual breakdown │ │ │ │ 7. Meanwhile, background sync runs every 30 minutes │ │ └─→ sync_service.sync_job() scheduled by APScheduler │ │ └─→ Calls firefly_client.get_transactions() │ │ └─→ Calls firefly_client.get_categories() │ │ └─→ Calls firefly_client.get_accounts() │ │ └─→ Updates SQLite database │ │ └─→ Logs sync status to sync_logs table │ │ │ │ 8. Next API request uses updated cached data │ │ └─→ User might see new/updated transactions │ │ in reports and dashboard │ │ │ └─────────────────────────────────────────────────────────────────────┘ ``` --- ## Component Hierarchy ``` App (src/App.tsx) │ ├── Navigation Bar │ ├── Dashboard Button │ └── Reports Button │ ├── Dashboard (pages/Dashboard.tsx) [Conditional Render] │ ├── <-- calls summaryService.getSummary() │ │ │ └── Metrics Grid │ ├── Total Assets Card │ ├── Recent Transactions Card │ └── 30-Day Summary Card │ └── Reports Page (pages/ReportsPage.tsx) [Conditional Render] │ ├── <-- calls reportService methods │ ├── DateRangePicker (components/DateRangePicker.tsx) │ ├── Quick Presets (30/90/365 days) │ └── Custom Date Inputs │ ├── Report Type Selector │ ├── Spending by Category │ ├── Income vs Expenses │ └── Trends │ └── Chart Display [Conditional based on report type] ├── SpendingChart (components/SpendingChart.tsx) │ └── Recharts PieChart ├── Comparison Display │ └── Text/Numbers (Income vs Expenses) └── TrendChart (components/TrendChart.tsx) └── Recharts LineChart ``` --- ## Database Schema Relationship Diagram ``` ┌─────────────────────┐ ┌──────────────────────┐ │ categories │ │ accounts │ ├─────────────────────┤ ├──────────────────────┤ │ id (PK) │ │ id (PK) │ │ firefly_id (U) │ │ firefly_id (U) │ │ name (U) │ │ name (U) │ │ type │ │ type │ │ created_at │ │ balance │ │ updated_at │ │ currency_code │ │ │ │ active │ └──────────┬──────────┘ │ created_at │ │ │ updated_at │ │ 1:N └──────────────────────┘ │ │ ┌─────────────────────────────┐ │ │ transactions │ │ ├─────────────────────────────┤ ├──────→│ id (PK) │ │ firefly_id (U) │ │ date (INDEX) │ │ amount │ │ description │ │ type (withdrawal/deposit) │ │ category_id (FK)──→categories.id │ from_account_id (FK)─→accounts.id │ to_account_id (FK)─→accounts.id │ notes │ │ created_at │ │ updated_at │ └─────────────────────────────┘ ┌─────────────────────┐ │ sync_logs │ ├─────────────────────┤ │ id (PK) │ │ entity_type │ │ last_sync │ │ status │ │ message │ │ created_at │ └─────────────────────┘ ``` --- ## Request/Response Examples ### Get Spending by Category Report **Request:** ``` GET /api/reports/spending-by-category?start_date=2024-11-25&end_date=2024-12-25 ``` **Response:** ```json { "period": { "start": "2024-11-25", "end": "2024-12-25" }, "data": [ { "category": "Groceries", "amount": 425.50 }, { "category": "Dining Out", "amount": 189.75 }, { "category": "Gas", "amount": 120.00 }, { "category": "Entertainment", "amount": 95.25 } ] } ``` ### Get Dashboard Summary **Request:** ``` GET /api/summary ``` **Response:** ```json { "total_assets": 15234.50, "recent_transactions": 47, "last_30_days": { "income": 3500.00, "expenses": 2145.75, "net": 1354.25 } } ``` ### Get Income vs Expenses **Request:** ``` GET /api/reports/income-vs-expenses?start_date=2024-11-25&end_date=2024-12-25 ``` **Response:** ```json { "period": { "start": "2024-11-25", "end": "2024-12-25" }, "income": 3500.00, "expenses": 2145.75, "net": 1354.25 } ``` --- ## Technology Stack Summary | Layer | Technology | Purpose | |-------|-----------|---------| | **Frontend** | React 18 | User interface | | | TypeScript | Type safety | | | Recharts | Interactive charts | | | Axios | HTTP client | | **Backend** | FastAPI | Web framework | | | Python 3.11 | Runtime | | | SQLAlchemy | ORM | | | httpx | Async HTTP client | | | APScheduler | Task scheduling | | **Database** | SQLite | Data storage | | **Deployment** | Docker | Containerization | | | Docker Compose | Orchestration | | | Nginx | Reverse proxy | --- ## Performance Considerations 1. **Caching**: SQLite holds cached data; minimizes API calls to Firefly III 2. **Async**: Backend uses async/await for I/O operations 3. **Indexing**: Transactions table indexed on date for fast filtering 4. **Sync Interval**: Default 30 minutes balances freshness vs API load 5. **Lazy Loading**: Charts only render when report type selected --- ## Security Architecture ``` ┌──────────────────┐ │ Firefly III │ │ API Token │ │ (in .env) │ └────────┬─────────┘ │ ↓ ┌──────────────────────────────────────────┐ │ Backend (Python) │ │ - Reads token from environment │ │ - Includes token in API headers │ │ - Validates requests │ │ - No sensitive data in logs │ └────────┬─────────────────────────────────┘ │ ↓ (CORS Enabled for localhost:3000) ┌──────────────────────────────────────────┐ │ Frontend (React) │ │ - No token storage in browser │ │ - Token never exposed to client │ │ - Communicates via CORS │ │ - All requests go through backend │ └──────────────────────────────────────────┘ ``` --- This architecture provides a clean, scalable foundation for financial analytics with: - ✅ Separation of concerns (frontend/backend) - ✅ Efficient data caching - ✅ Background data sync - ✅ RESTful API design - ✅ Type safety (TypeScript + Python type hints) - ✅ Container-ready deployment