City Service Heat

Blend 311 requests, crime incidents, and closures into an anomaly lens with DP aware binning.

NYC 311 recent

  • Street ConditionBRONX
    Pothole
    2026-01-09T02:36:15.000
  • Abandoned VehicleUnspecified
    With License Plate
    2026-01-09T02:21:28.000
  • Noise - ResidentialMANHATTAN
    Banging/Pounding
    2026-01-09T02:20:09.000
  • Non-Emergency Police MatterBRONX
    Other (complaint details)
    2026-01-09T02:19:28.000
  • Noise - ResidentialBROOKLYN
    Banging/Pounding
    2026-01-09T02:19:23.000
  • DrinkingBRONX
    In Public
    2026-01-09T02:18:18.000
  • Illegal ParkingSTATEN ISLAND
    Posted Parking Sign Violation
    2026-01-09T02:18:05.000
  • Noise - ResidentialQUEENS
    Banging/Pounding
    2026-01-09T02:17:59.000
  • Illegal ParkingQUEENS
    Commercial Overnight Parking
    2026-01-09T02:17:09.000
  • Illegal ParkingQUEENS
    Parking Permit Improper Use
    2026-01-09T02:15:57.000
  • Noise - VehicleBROOKLYN
    Engine Idling
    2026-01-09T02:14:51.000
  • Illegal ParkingBROOKLYN
    Commercial Overnight Parking
    2026-01-09T02:14:50.000
  • Noise - ResidentialQUEENS
    Banging/Pounding
    2026-01-09T02:14:45.000
  • Noise - CommercialMANHATTAN
    Loud Music/Party
    2026-01-09T02:14:30.000
  • Illegal ParkingQUEENS
    Commercial Overnight Parking
    2026-01-09T02:13:34.000
  • Noise - Street/SidewalkBRONX
    Loud Music/Party
    2026-01-09T02:12:56.000
  • Noise - ResidentialBROOKLYN
    Banging/Pounding
    2026-01-09T02:11:00.000
  • Homeless Person AssistanceBRONX
    Non-Chronic
    2026-01-09T02:10:27.000
  • Noise - ResidentialMANHATTAN
    Loud Talking
    2026-01-09T02:09:35.000
  • Noise - ResidentialBROOKLYN
    Loud Music/Party
    2026-01-09T02:09:18.000
  • Noise - CommercialBROOKLYN
    Banging/Pounding
    2026-01-09T02:08:54.000
  • Noise - VehicleBROOKLYN
    Engine Idling
    2026-01-09T02:08:53.000
  • Dirty ConditionBRONX
    Trash
    2026-01-09T02:08:06.000
  • Noise - ResidentialBROOKLYN
    Loud Music/Party
    2026-01-09T02:06:56.000
  • Noise - CommercialMANHATTAN
    Loud Music/Party
    2026-01-09T02:06:54.000
  • Consumer ComplaintBROOKLYN
    Stoop Line Stand
    2026-01-09T02:05:56.000
  • Noise - ResidentialBROOKLYN
    Loud Music/Party
    2026-01-09T02:05:47.000
  • Noise - ResidentialQUEENS
    Banging/Pounding
    2026-01-09T02:05:07.000
  • Dirty ConditionBRONX
    Trash
    2026-01-09T02:05:00.000
  • Blocked DrivewayQUEENS
    No Access
    2026-01-09T02:04:05.000
  • Illegal ParkingBROOKLYN
    Blocked Hydrant
    2026-01-09T02:03:34.000
  • Illegal ParkingQUEENS
    Blocked Hydrant
    2026-01-09T02:03:04.000
  • Noise - ResidentialBRONX
    Banging/Pounding
    2026-01-09T02:02:13.000
  • Noise - ResidentialBROOKLYN
    Loud Music/Party
    2026-01-09T02:00:53.000
  • Illegal ParkingQUEENS
    Double Parked Blocking Traffic
    2026-01-09T02:00:45.000
  • Noise - ResidentialBRONX
    Banging/Pounding
    2026-01-09T01:59:49.000
  • Noise - ResidentialBROOKLYN
    Loud Music/Party
    2026-01-09T01:59:08.000
  • Noise - ResidentialBROOKLYN
    Loud Music/Party
    2026-01-09T01:58:48.000
  • Taxi ComplaintQUEENS
    Driver Complaint - Non Passenger
    2026-01-09T01:58:26.000
  • Smoking or VapingMANHATTAN
    Allowed in Smoke Free Area
    2026-01-09T01:58:24.000
  • Homeless Person AssistanceMANHATTAN
    Non-Chronic
    2026-01-09T01:58:06.000
  • Street Sweeping ComplaintBROOKLYN
    Street Not Swept
    2026-01-09T01:57:43.000
  • Cannabis RetailerBRONX
    Sales to Minors Under 21
    2026-01-09T01:57:34.000
  • Illegal ParkingBROOKLYN
    Blocked Hydrant
    2026-01-09T01:56:28.000
  • Blocked DrivewayBRONX
    No Access
    2026-01-09T01:56:23.000
  • Noise - VehicleBRONX
    Engine Idling
    2026-01-09T01:55:57.000
  • Noise - ResidentialQUEENS
    Loud Music/Party
    2026-01-09T01:55:45.000
  • Noise - VehicleBRONX
    Engine Idling
    2026-01-09T01:54:11.000
  • Noise - ResidentialSTATEN ISLAND
    Banging/Pounding
    2026-01-09T01:53:42.000
  • Street Sign - DamagedMANHATTAN
    Other/Unknown
    2026-01-09T01:53:41.000

Build goals

Blend 311 requests, crime incidents, and closures into an anomaly lens with DP aware binning.

Stack

  • Frontend: React 18, Mapbox GL or deck.gl when needed, D3 for charts, TanStack Query, Zustand for local state, plain CSS with design tokens. No runtime CSS frameworks.
  • API: Python 3.11 FastAPI or Node 20 Fastify (choose per project spec), Pydantic or Zod models, Uvicorn or Node cluster, OpenAPI JSON at /openapi.json.
  • Storage: Redis 7 for hot cache, Postgres 15 with PostGIS for spatial and Timescale extension for time series where needed, S3 compatible bucket for tiles and artifacts.
  • Ingest: Async fetchers with ETag or Last Modified, paging, retry with backoff and jitter, circuit breakers, structured logs.
  • Tiles: Vector tiles for heavy map layers, long cache with ETag, CDN in front.
  • Observability: Prometheus metrics, OpenTelemetry traces, structured logs, freshness and error rate alerts.
  • Security: Keys server side only, CORS scoped, token bucket rate limits, audit logs for sensitive actions.

Data sources

SourceEndpointCadenceAccessAuthNotes
NYC 311data.cityofnewyork.us/resource/erm2-nwe9.jsonfrequentSocrata APINoneService requests
Chicago 311data.cityofchicago.org/resource/v6vf-nfxy.jsonfrequentSocrata APINoneService requests
Chicago Crimesdata.cityofchicago.org/resource/ijzp-q8t2.jsonfrequentSocrata APINoneIncidents
UK National Highwaysdeveloper.data.nationalhighways.co.ukfrequentRESTKeyClosures and incidents

Architecture

Python FastAPI, z score anomalies per tract, DP friendly binning, explainable drivers, PostGIS spatial joins, paging and caching.

Models

Models are expressed in DB tables and mirrored as API schemas. All timestamps are UTC. All coordinates are WGS84. Stable IDs, soft deletes by valid_to when needed.

  • request(id, ts, category, geom, attrs jsonb)
  • incident(id, ts, type, geom, severity)
  • anomaly(tract_id, category, window, zscore, evidence jsonb)

Algorithms

  • Rolling baselines per tract and category
  • DP friendly binning, hide low counts
  • Attribution by category mix contribution

API surface

  • GET /heat?city=&bbox=&since=&until=&category=
  • GET /anomalies?city=&since=&until=&min_z=
  • GET /tracts?city=

UI and visualization

  • Tilegrams that morph across admin levels
  • Sparkbands per tile and brushed selection
  • Narrative labels that explain drivers

Performance budgets

  • City heat view p95 under 16 ms frame time
  • API p95 under 300 ms for common filters
  • FCP under 2 s on broadband mid tier laptop.
  • API p95 under 300 ms for common list endpoints, p99 under 800 ms.
  • Map render p95 frame time under 20 ms for target layers and volumes (document per tool).
  • Frontend app code under 180 KB gzip excluding map library.
  • API memory under 200 MB under normal load.

Accessibility

  • WCAG 2.2 AA, automated axe checks clean, no critical issues.
  • Keyboard navigable controls, focus rings visible, ARIA roles correct.
  • Color contrast at or above 4.5 to 1, colorblind safe palettes.
  • Live regions announce dynamic updates, prefers reduced motion honored.

Evidence pack and quality gates

  • Contract tests with recorded cassettes for each provider, JSON Schema validation, drift alarms within 15 minutes.
  • Load tests with k6, thresholds enforced in CI for p95 and p99.
  • Lighthouse performance and a11y reports stored as CI artifacts.
  • Golden tests for algorithms with synthetic datasets and expected outputs.
  • Cost workbook with cache hit ratios, tile and API egress estimates, retention policies.

CI configuration

name: ci
on: [push, pull_request]
jobs:
  api:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgis/postgis:15-3.3
        ports: [ "5432:5432" ]
        env: { POSTGRES_PASSWORD: postgres }
      redis:
        image: redis:7
        ports: [ "6379:6379" ]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: "20" }
      - uses: actions/setup-python@v5
        with: { python-version: "3.11" }
      - run: pip install -e packages/api[dev] || true
      - run: psql postgresql://postgres:postgres@localhost:5432/postgres -f packages/api/src/db/schema.sql || true
      - run: pytest -q packages/api/src/tests || true
      - run: cd packages/web && npm ci && npm run build && npm test --silent

Risks and mitigations

  • Schema drift across cities, isolate in adapters with tests
  • Privacy risks, enforce suppression and aggregation thresholds

Acceptance checklist

  • CI green on main, all quality gates met.
  • Freshness SLOs met for hot regions or feeds.
  • Performance budgets met or better.
  • A11y audits pass with zero critical findings.
  • Provenance and license panels render correct metadata.
  • Runbook covers stale feed handling, provider errors, and key rotation.

Implementation sequence

  • Adapters with paging and where clauses
  • Anomaly engine and DP binning
  • Tilegrams, sparkbands, and narratives
  • Evidence pack and privacy checks

Runbook

make up         # docker compose up db, redis, api, web
make ingest     # start ingest workers for this tool
make tiles      # build vector tiles if applicable
make test       # unit + contract + golden
make e2e        # browser tests