Changelog

What we're shipping.

We improve the platform's reliability, scale, and coverage almost every day. Here's the ongoing record of the backend work that keeps exposure intelligence fast, complete, and trustworthy.

  1. Recovery

    Self-healing recovery for previously-failed uploads

    • Re-uploading source data that previously failed to ingest now works automatically — the duplicate-detection layer no longer permanently blocks content that never actually reached the index.
    • Reclaimed a large backlog of historically-stuck files so their exposure data becomes searchable, with no risk of re-importing data we already hold.
  2. Recovery

    Large-file import recovery + cleaner status reporting

    • Restored import of multi-gigabyte breach shards that had stalled, recovering tens of millions of records into search.
    • Failed/duplicate/empty/corrupt sources are now labeled distinctly, so an already-ingested duplicate is no longer mislabeled as a failure.
  3. Scale

    Intake hardening for very large archives

    • The ingestion pipeline now streams arbitrarily large compressed archives end-to-end without exhausting memory, scaling to terabyte-class inputs.
    • Long-running extraction and analysis report continuous progress, eliminating a class of false-timeout failures on big datasets.
  4. Reliability

    Archive extraction robustness

    • More resilient handling of compressed-archive formats, with clear, actionable status when an archive is genuinely corrupt or truncated.
    • Transient infrastructure hiccups during extraction no longer strand otherwise-valid breaches.
  5. Scale

    Faster index compaction

    • Reworked the background index-folding pipeline for higher throughput, keeping freshly-ingested data searchable sooner.
    • Auto-scaling of compaction workers tightened to match real load, improving cost efficiency.
  6. Search

    Search index integrity auditing

    • Added continuous auditing that samples completed datasets and verifies they return correctly in search, flagging anything that needs a rebuild.
    • Improved confidence that indexed coverage matches ingested coverage.