PermitParser scrapes, parses, and normalizes building permits and zoning filings from fractured municipal portals: PDFs, HTML tables, scanned agendas. All of it lands in one queryable, spatial record. Built to survive fifteen towns' worth of inconsistent government web software.
A PDF agenda and a JavaScript permit portal fail differently, so the pipeline is built to expect that: a schema-drift landing zone, a content-hash ledger, and a normalization boundary between messy source text and a queryable record.
Polls a town's listing surface (a table, an agenda index) for documents published since the last run. Never downloads a full body just to check if it's new.
SHA-256 of the raw bytes before anything touches disk. The same PDF fetched twice, whether from a re-run or an unchanged listing, costs one indexed lookup and never a duplicate row.
pdfplumber for a real text layer, PyMuPDF + Tesseract OCR when
there isn't one. Pure functions with no I/O, validated into a strict
Pydantic PermitRecord.
PostGIS-indexed, queryable by category, valuation, date range, or radius. A rescrape updates a record in place, never a duplicate for the same real-world filing.
Every town in South Central Connecticut runs its own GIS, its own permit vendor, its own PDF export format. Each one gets checked and built for real before it's marked live. No placeholder rows.
Planning & Zoning Commission agendas, CivicPlus AgendaCenter. PDF, statically scrapable.
Portal vendor not yet confirmed against a live scraper.
Portal vendor not yet confirmed against a live scraper.
Portal vendor not yet confirmed against a live scraper.
Every figure on this page comes from the same public, unauthenticated API. Filter by municipality, category, valuation, date range, or a straight-line radius search.
// live response...
Every design decision, the raw/normalized split, the config-driven scrapers, the honest "not built yet" coverage table, exists to make the next town an addition, not a rewrite.