377 lines
20 KiB
Markdown
377 lines
20 KiB
Markdown
# Firefly III Analytics - Architecture & Data Flow
|
|
|
|
## System Architecture Diagram
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ FIREFLY III INSTANCE │
|
|
│ (https://firefly.scsimedia.duckdns.org) │
|
|
│ │
|
|
│ - Transactions │
|
|
│ - Categories │
|
|
│ - Accounts │
|
|
└────────────────────────┬────────────────────────────────────────┘
|
|
│ API Requests (Token-based Auth)
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ BACKEND (FastAPI) │
|
|
│ Running on port 8000 │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ Firefly III API Client (firefly_client.py) │ │
|
|
│ │ - Async HTTP requests to Firefly III │ │
|
|
│ │ - Token-based authentication │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ↓ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ Background Sync Service (sync_service.py) │ │
|
|
│ │ - Runs every 30 minutes (APScheduler) │ │
|
|
│ │ - Fetches transactions, categories, accounts │ │
|
|
│ │ - Stores in SQLite database │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ↓ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ SQLite Database (data/app.db) │ │
|
|
│ │ - Cached transactions │ │
|
|
│ │ - Cached categories │ │
|
|
│ │ - Cached accounts │ │
|
|
│ │ - Sync logs │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ↓ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ API Routers │ │
|
|
│ │ ├── /api/transactions (list, filter) │ │
|
|
│ │ ├── /api/categories (list) │ │
|
|
│ │ ├── /api/accounts (list) │ │
|
|
│ │ ├── /api/reports/* (analytics) │ │
|
|
│ │ │ ├── /spending-by-category │ │
|
|
│ │ │ ├── /income-vs-expenses │ │
|
|
│ │ │ └── /trends │ │
|
|
│ │ └── /api/summary (dashboard metrics) │ │
|
|
│ │ ├── /docs (interactive API docs) │ │
|
|
│ │ └── /health (status check) │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└─────────────────────────┬────────────────────────────────────────┘
|
|
│ JSON REST API (CORS enabled)
|
|
↓
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ FRONTEND (React) │
|
|
│ Running on port 3000 │
|
|
│ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ API Service (services/api.ts) │ │
|
|
│ │ - Axios client for HTTP requests │ │
|
|
│ │ - Manages API calls to backend │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ↓ │
|
|
│ ┌──────────────────────────────────────────────────────────┐ │
|
|
│ │ React Components & Pages │ │
|
|
│ │ │ │
|
|
│ │ Dashboard (pages/Dashboard.tsx) │ │
|
|
│ │ ├── Total Assets Card │ │
|
|
│ │ ├── Recent Transactions Card │ │
|
|
│ │ └── 30-Day Income/Expenses Card │ │
|
|
│ │ │ │
|
|
│ │ Reports (pages/ReportsPage.tsx) │ │
|
|
│ │ ├── Date Range Picker (components/DateRangePicker.tsx) │ │
|
|
│ │ ├── Report Selector │ │
|
|
│ │ ├── Spending Chart (components/SpendingChart.tsx) │ │
|
|
│ │ │ └── Pie Chart (Recharts) │ │
|
|
│ │ └── Trend Chart (components/TrendChart.tsx) │ │
|
|
│ │ └── Line Chart (Recharts) │ │
|
|
│ └──────────────────────────────────────────────────────────┘ │
|
|
│ │
|
|
└────────────────────┬───────────────────────────────────────────┘
|
|
│
|
|
↓
|
|
┌──────────────┐
|
|
│ Web Browser │
|
|
│ localhost:3 │
|
|
│ 000 │
|
|
└──────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Data Flow Sequence
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────┐
|
|
│ USER INTERACTION │
|
|
│ │
|
|
│ 1. User navigates to http://localhost:3000 (React app loads) │
|
|
│ │
|
|
│ 2. Dashboard component mounts │
|
|
│ └─→ Calls summaryService.getSummary() │
|
|
│ └─→ HTTP GET /api/summary │
|
|
│ └─→ Backend calculates metrics from SQLite │
|
|
│ └─→ Response: JSON with totals, trends │
|
|
│ └─→ Frontend renders cards with metrics │
|
|
│ │
|
|
│ 3. User clicks on "Reports" tab │
|
|
│ └─→ ReportsPage component mounts │
|
|
│ │
|
|
│ 4. User selects date range (default: last 30 days) │
|
|
│ └─→ onPreset() or manual date selection │
|
|
│ └─→ Triggers reportService call │
|
|
│ └─→ HTTP GET /api/reports/spending-by-category │
|
|
│ ?start_date=2024-11-25&end_date=2024-12-25 │
|
|
│ │
|
|
│ 5. Backend processes report request │
|
|
│ └─→ Queries SQLite transactions table │
|
|
│ └─→ Filters by date range │
|
|
│ └─→ Groups by category │
|
|
│ └─→ Sums amounts per category │
|
|
│ └─→ Returns JSON: [{category, amount}, ...] │
|
|
│ │
|
|
│ 6. Frontend receives report data │
|
|
│ └─→ Passes to SpendingChart component │
|
|
│ └─→ Recharts renders pie chart │
|
|
│ └─→ User sees visual breakdown │
|
|
│ │
|
|
│ 7. Meanwhile, background sync runs every 30 minutes │
|
|
│ └─→ sync_service.sync_job() scheduled by APScheduler │
|
|
│ └─→ Calls firefly_client.get_transactions() │
|
|
│ └─→ Calls firefly_client.get_categories() │
|
|
│ └─→ Calls firefly_client.get_accounts() │
|
|
│ └─→ Updates SQLite database │
|
|
│ └─→ Logs sync status to sync_logs table │
|
|
│ │
|
|
│ 8. Next API request uses updated cached data │
|
|
│ └─→ User might see new/updated transactions │
|
|
│ in reports and dashboard │
|
|
│ │
|
|
└─────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Component Hierarchy
|
|
|
|
```
|
|
App (src/App.tsx)
|
|
│
|
|
├── Navigation Bar
|
|
│ ├── Dashboard Button
|
|
│ └── Reports Button
|
|
│
|
|
├── Dashboard (pages/Dashboard.tsx) [Conditional Render]
|
|
│ ├── <-- calls summaryService.getSummary()
|
|
│ │
|
|
│ └── Metrics Grid
|
|
│ ├── Total Assets Card
|
|
│ ├── Recent Transactions Card
|
|
│ └── 30-Day Summary Card
|
|
│
|
|
└── Reports Page (pages/ReportsPage.tsx) [Conditional Render]
|
|
│
|
|
├── <-- calls reportService methods
|
|
│
|
|
├── DateRangePicker (components/DateRangePicker.tsx)
|
|
│ ├── Quick Presets (30/90/365 days)
|
|
│ └── Custom Date Inputs
|
|
│
|
|
├── Report Type Selector
|
|
│ ├── Spending by Category
|
|
│ ├── Income vs Expenses
|
|
│ └── Trends
|
|
│
|
|
└── Chart Display [Conditional based on report type]
|
|
├── SpendingChart (components/SpendingChart.tsx)
|
|
│ └── Recharts PieChart
|
|
├── Comparison Display
|
|
│ └── Text/Numbers (Income vs Expenses)
|
|
└── TrendChart (components/TrendChart.tsx)
|
|
└── Recharts LineChart
|
|
```
|
|
|
|
---
|
|
|
|
## Database Schema Relationship Diagram
|
|
|
|
```
|
|
┌─────────────────────┐ ┌──────────────────────┐
|
|
│ categories │ │ accounts │
|
|
├─────────────────────┤ ├──────────────────────┤
|
|
│ id (PK) │ │ id (PK) │
|
|
│ firefly_id (U) │ │ firefly_id (U) │
|
|
│ name (U) │ │ name (U) │
|
|
│ type │ │ type │
|
|
│ created_at │ │ balance │
|
|
│ updated_at │ │ currency_code │
|
|
│ │ │ active │
|
|
└──────────┬──────────┘ │ created_at │
|
|
│ │ updated_at │
|
|
│ 1:N └──────────────────────┘
|
|
│
|
|
│ ┌─────────────────────────────┐
|
|
│ │ transactions │
|
|
│ ├─────────────────────────────┤
|
|
├──────→│ id (PK) │
|
|
│ firefly_id (U) │
|
|
│ date (INDEX) │
|
|
│ amount │
|
|
│ description │
|
|
│ type (withdrawal/deposit) │
|
|
│ category_id (FK)──→categories.id
|
|
│ from_account_id (FK)─→accounts.id
|
|
│ to_account_id (FK)─→accounts.id
|
|
│ notes │
|
|
│ created_at │
|
|
│ updated_at │
|
|
└─────────────────────────────┘
|
|
|
|
┌─────────────────────┐
|
|
│ sync_logs │
|
|
├─────────────────────┤
|
|
│ id (PK) │
|
|
│ entity_type │
|
|
│ last_sync │
|
|
│ status │
|
|
│ message │
|
|
│ created_at │
|
|
└─────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## Request/Response Examples
|
|
|
|
### Get Spending by Category Report
|
|
|
|
**Request:**
|
|
```
|
|
GET /api/reports/spending-by-category?start_date=2024-11-25&end_date=2024-12-25
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"period": {
|
|
"start": "2024-11-25",
|
|
"end": "2024-12-25"
|
|
},
|
|
"data": [
|
|
{ "category": "Groceries", "amount": 425.50 },
|
|
{ "category": "Dining Out", "amount": 189.75 },
|
|
{ "category": "Gas", "amount": 120.00 },
|
|
{ "category": "Entertainment", "amount": 95.25 }
|
|
]
|
|
}
|
|
```
|
|
|
|
### Get Dashboard Summary
|
|
|
|
**Request:**
|
|
```
|
|
GET /api/summary
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"total_assets": 15234.50,
|
|
"recent_transactions": 47,
|
|
"last_30_days": {
|
|
"income": 3500.00,
|
|
"expenses": 2145.75,
|
|
"net": 1354.25
|
|
}
|
|
}
|
|
```
|
|
|
|
### Get Income vs Expenses
|
|
|
|
**Request:**
|
|
```
|
|
GET /api/reports/income-vs-expenses?start_date=2024-11-25&end_date=2024-12-25
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"period": {
|
|
"start": "2024-11-25",
|
|
"end": "2024-12-25"
|
|
},
|
|
"income": 3500.00,
|
|
"expenses": 2145.75,
|
|
"net": 1354.25
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Technology Stack Summary
|
|
|
|
| Layer | Technology | Purpose |
|
|
|-------|-----------|---------|
|
|
| **Frontend** | React 18 | User interface |
|
|
| | TypeScript | Type safety |
|
|
| | Recharts | Interactive charts |
|
|
| | Axios | HTTP client |
|
|
| **Backend** | FastAPI | Web framework |
|
|
| | Python 3.11 | Runtime |
|
|
| | SQLAlchemy | ORM |
|
|
| | httpx | Async HTTP client |
|
|
| | APScheduler | Task scheduling |
|
|
| **Database** | SQLite | Data storage |
|
|
| **Deployment** | Docker | Containerization |
|
|
| | Docker Compose | Orchestration |
|
|
| | Nginx | Reverse proxy |
|
|
|
|
---
|
|
|
|
## Performance Considerations
|
|
|
|
1. **Caching**: SQLite holds cached data; minimizes API calls to Firefly III
|
|
2. **Async**: Backend uses async/await for I/O operations
|
|
3. **Indexing**: Transactions table indexed on date for fast filtering
|
|
4. **Sync Interval**: Default 30 minutes balances freshness vs API load
|
|
5. **Lazy Loading**: Charts only render when report type selected
|
|
|
|
---
|
|
|
|
## Security Architecture
|
|
|
|
```
|
|
┌──────────────────┐
|
|
│ Firefly III │
|
|
│ API Token │
|
|
│ (in .env) │
|
|
└────────┬─────────┘
|
|
│
|
|
↓
|
|
┌──────────────────────────────────────────┐
|
|
│ Backend (Python) │
|
|
│ - Reads token from environment │
|
|
│ - Includes token in API headers │
|
|
│ - Validates requests │
|
|
│ - No sensitive data in logs │
|
|
└────────┬─────────────────────────────────┘
|
|
│
|
|
↓ (CORS Enabled for localhost:3000)
|
|
┌──────────────────────────────────────────┐
|
|
│ Frontend (React) │
|
|
│ - No token storage in browser │
|
|
│ - Token never exposed to client │
|
|
│ - Communicates via CORS │
|
|
│ - All requests go through backend │
|
|
└──────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
This architecture provides a clean, scalable foundation for financial analytics with:
|
|
- ✅ Separation of concerns (frontend/backend)
|
|
- ✅ Efficient data caching
|
|
- ✅ Background data sync
|
|
- ✅ RESTful API design
|
|
- ✅ Type safety (TypeScript + Python type hints)
|
|
- ✅ Container-ready deployment
|