🛠️ Tech Stack

🚀 Features

• LLM-powered edge classification — Every Wikipedia hop is classified into one of 5 relationship categories using GPT-3.5 Turbo with JSON mode, exponential backoff, and a normalization map that handles model drift (e.g., "historical" → causal)

• Interactive force-directed graph — D3.js visualization with adaptive force parameters that scale with graph size, category-coded edge colors, node sizing by visit frequency, hover highlighting that dims unconnected nodes to 15% opacity, and drag-to-pin positioning

• Community detection + cluster naming — Greedy modularity algorithm identifies thematic clusters, then the LLM generates creative 3-5 word names (e.g., "Roman Philosophical Heritage"). Click a cluster card to spotlight its members on the graph

• Surprise connection finder — Discovers the longest shortest paths across cluster boundaries, surfacing unexpected connections like Surfing → Fluid Dynamics → Aqueducts → Roman Empire → Marcus Aurelius → Stoicism → CBT → Aaron T. Beck

• Write-through dual persistence — Every graph mutation writes to both the in-memory NetworkX DiGraph and SQLite simultaneously, so no data is lost on crash. On startup, the full graph reconstructs from the database

• Session management + filtering — Multiple browsing sessions tracked independently. Filter the graph to a single session or view all sessions overlaid. Form chaining auto-fills the next "From" field after each visit

• Zero-config demo mode — seed_demo.py --mock populates 3 sessions with 22 pre-labeled edges using hard-coded data, no API key required. Reviewers can clone and see a working graph in under 60 seconds

• 50-edge evaluation suite — Manually labeled dataset with also_acceptable fields for genuinely ambiguous edges. Reports strict accuracy, lenient accuracy, per-category precision/recall/F1, and a 5x5 confusion matrix

🧗‍♂️ Challenges

• Dual-write consistency — The GraphManager maintains both a NetworkX graph (for fast traversals) and SQLite (for durability). Every add_node and add_edge writes to both stores atomically. The tricky part was ensuring foreign key constraints in SQLite were satisfied during write-through — the in-memory graph doesn't enforce referential integrity, but the database does. This surfaced in testing when edges referenced sessions that hadn't been created in the test DB yet

• LLM output reliability at scale — GPT-3.5 Turbo occasionally returns categories outside the defined taxonomy ("historical", "conceptual", "related"). Built a normalization map to correct common drift, defaulting to "tangential" for unmapped responses and "unknown" when all retries are exhausted. The retry strategy uses exponential backoff [1s, 3s, 9s] with doubled delays for rate limit errors specifically

• D3 force layout scaling — Fixed force parameters that look great with 15 nodes produce an unreadable hairball at 80+. Implemented adaptive parameters: charge strength weakens from -200 to -100, link distance stretches from 80px to 120px, and labels are hidden for low-degree nodes above 30 total — all keyed to current node count

• Framework swap mid-build — The spec was written for Flask, but we switched to FastAPI after Phase 1. Because the data layer was framework-agnostic from day one (no flask.g, no request-scoped connections), the swap only required rewriting main.py, the Dockerfile, and one test. A clean architecture boundary turned a risky pivot into a 30-minute task

• Transitive dependency conflict — FastAPI's Starlette pinned httpx <0.28, but the OpenAI SDK pulled in httpx 0.28+, which removed the app= kwarg the test client depends on. Diagnosed by tracing the TypeError through starlette's TestClient constructor and resolved by pinning httpx >=0.23,<0.28 as a dev dependency

🧠 Learnings

• Architecture decisions compound — Debugging cycles decreased with each phase: Phase 1 had 4, Phase 2 had 2, Phase 3 had 1, Phases 4-6 had 0. Early investments in test isolation, service boundaries, and framework-agnostic data layers prevented entire classes of bugs later. The Phase 1 decision to enable PRAGMA foreign_keys = ON caught a data integrity issue in Phase 3 that would have silently corrupted production data

• Evaluate subjective AI output honestly — The eval suite includes an also_acceptable field because "Marcus Aurelius → Roman Empire" is legitimately both biographical and geographic. Forcing single labels inflates error counts and makes evals less useful. Reporting both strict and lenient accuracy gives a more honest picture of model performance

• Domain knowledge beats algorithmic brute force — The surprise finder's key optimization isn't sampling (though it does that above 200 nodes) — it's skipping intra-community pairs. Paths within a cluster aren't surprising by definition. This community-aware filtering produces better results while also reducing computation by ~33%

• Portfolio projects need a zero-config demo path — If seeing the project requires an API key and 10 minutes of setup, most reviewers bounce. The --mock seed mode bridges the gap between git clone and a populated, interactive graph. This was the single most impactful feature for portfolio presentation

• Build the frontend last against a stable API — Phases 1-4 built and fully tested the backend (93 tests, all passing). Phase 5 was pure greenfield frontend with zero debugging cycles, because every API contract was already proven. No ambiguity about whether a rendering bug was a frontend or backend problem