The tradeoffs behind the systems work.
Field notes on intelligence systems, evidence engineering, automation that survives reality, and the operational discipline of shipping systems — not demos.
Why Most OSINT Platforms Collapse at Scale
OSINT platforms fail because they optimize for demos, not operations. Discussing the silent accumulation of technical and human debt.
Read the essay →Start here.
Metadata Filtering in Vector Search: Pre-Filter, Post-Filter, and the Recall Cliff
How metadata filtering actually behaves in vector search: why post-filtering breaks tenant isolation, when selective filters collapse ANN recall, and what to do.
Embedding Model Selection for Retrieval: How to Choose Without Trusting a Leaderboard
How I pick an embedding model for retrieval: the constraints that decide it before quality does, a bake-off you can run, and the re-index nobody prices.
Worker Fleet Architecture at Scale: Pools, Scaling Signals, and Draining
Scaling a worker fleet isn't adding workers to one queue. Here's how I partition pools, pick the scaling signal, and drain workers without losing jobs.
Metadata Filtering in Vector Search: Pre-Filter, Post-Filter, and the Recall Cliff
How metadata filtering actually behaves in vector search: why post-filtering breaks tenant isolation, when selective filters collapse ANN recall, and what to do.
Embedding Model Selection for Retrieval: How to Choose Without Trusting a Leaderboard
How I pick an embedding model for retrieval: the constraints that decide it before quality does, a bake-off you can run, and the re-index nobody prices.
Worker Fleet Architecture at Scale: Pools, Scaling Signals, and Draining
Scaling a worker fleet isn't adding workers to one queue. Here's how I partition pools, pick the scaling signal, and drain workers without losing jobs.
How to Evaluate RAG Retrieval: Eval Sets, Metrics, and Ship Gates
Retrieval eval is 20% metrics and 80% eval set. How I build one that survives re-indexing, which number to read at which k, and how to gate a change.
RAG Chunking Strategy: How to Split Documents So Retrieval Works
Chunking decides what your retriever can find. Here's how I split real documents — structure-first boundaries, parent-child units, and how to prove it worked.
Why Is My RAG Retrieval Bad? A Diagnostic Order of Operations
Bad RAG retrieval is four or five distinct failures wearing one costume. Here's how I localize which one you have before changing anything.
Retry with Exponential Backoff and Jitter (and the Retry Budget Nobody Sets)
Exponential backoff caps how fast a client retries; jitter stops every client retrying together. Here's the retry logic I actually ship, and the two limits most teams miss.
Reranking in RAG: How a Cross-Encoder Fixes Retrieval Quality
A cross-encoder reranker re-scores your top candidates by reading query and document together. Here's how I add one, size it, and prove it worked.
Dead Letter Queue Design Patterns (Routing, Envelopes, and Redrive)
A DLQ is the giving-up mechanism, and most teams build it wrong. The routing, envelope, isolation, and redrive patterns I use to make failed messages recoverable.
How to Build Hybrid Search with pgvector and BM25 in Postgres
Build hybrid search in Postgres with pgvector, tsvector, and RRF in one SQL query — the schema, index tuning, and when you actually need real BM25.
Hybrid Search vs Vector Search: Why RAG Retrieval Needs Both
Vector search understands meaning but fumbles exact identifiers; keyword search is the opposite. Here's how I build hybrid retrieval that does both — with fusion, reranking, and a pgvector setup.
How to Design an Idempotent Job Queue (Retries, Backoff, and Dead Letters)
At-least-once delivery makes idempotency mandatory, not optional. Here's how I design job queues that retry safely, back off with jitter, and dead-letter poison messages — with code.
From Analyst-Heavy to System-Heavy: Scaling Without Burning Humans
Analysts should supervise systems, not compensate for them. How to build sustainable feedback loops between engineering and analysis.
Why Most OSINT Platforms Collapse at Scale
OSINT platforms fail because they optimize for demos, not operations. Discussing the silent accumulation of technical and human debt.
Sovrint: Temporal Propagation of Coordinated Narratives
Narratives propagate like systems, not stories. A strategic deep dive into signal clustering, sentiment drift, and the velocity of coordinated information operations.
Web Forensics: Reconstructing Digital Traces After the Fact
The web leaves scars if you know where to look. A technical deep dive into session reconstruction, browser artifacts, and digital evidence decay.
Browser Telemetry Evasion: The Silent Arms Race
Detection happens at layers most engineers ignore. A technical deep dive into TLS fingerprinting, Canvas poisoning, and managing behavioral jitter in high-scale automation.
Deterministic Scrapers in a Non-Deterministic Web
Web scraping is no longer about CSS selectors; it is about adaptive systems. A technical exploration of LLM-based element recovery, visual anchors, and resilient web orchestration.
TaskEngine: Android Automation Without Root or Instrumentation
Human-grade mobile automation is possible without invasive hooks. A technical breakdown of the TaskEngine runtime, Accessibility Services, and UI drift management.
Automation That Survives Reality
Automation must expect and embrace entropy. A philosophical and technical deep dive into building resilient systems that handle drift, decay, and adversarial environments.
Entity Resolution Without Illusions
Identity is probabilistic, not deterministic. Confronting the instability of digital identity in open-source intelligence.
Screenshots as Evidence: Designing for Trust, Not Just Storage
Evidence must survive scrutiny, not just exist. A deep dive into Evidence Engineering, immutability, and the chain of custody for digital artifacts.
Hybrid Search in Practice: Tuning Relevance Without Lying to Yourself
Relevance tuning is an operational discipline, not a one-time configuration. A deep dive into evaluation metrics, bias suppression, and feedback loops for intelligence systems.
The Hybrid Search Engine: Combining Lexical and Semantic Ranks
OSINT relevance is multi-modal. A technical exploration of why keywords fail and how to fuse BM25 with Vector Embeddings for operator-grade retrieval.
Usage-Based Intelligence: Building Scalable Billing Infrastructures
Billing is a distributed systems problem in disguise. Integrating real-time usage tracking with high-stakes intelligence signals.
Probabilistic Entity Resolution: Correlating Signals in the Noise
Identity in the digital wild is never certain—it is a score. A technical deep dive into probabilistic linkage, signal stacking, and confidence models for intelligence systems.
Monitoring Is Not Alerting
Alerting is an interruption budget, not a metric. Designing high-signal, low-fatigue observability systems.
Designing for Disruption: Fault-Tolerance in Worker Fleets
Systems must degrade gracefully, not heroically. How to survive proxy pool collapses and API disruptions.
Worker Fleets in Practice: Retries, Idempotency, and Failure Taxonomies
Failures are classes, not surprises. Designing resilient worker fleets for complex, non-deterministic environments.
Scaling the Ingest: Architectural Lessons from TraxinteL
Ingestion is a state machine, not a scraper. Lessons learned from building high-scale distributed collection pipelines.
What I Mean When I Say “Shipping Systems”
Shipping systems means shipping behavior under load, over time. A philosophical anchor for the operator-grade engineer.
The Intelligence Core: Designing Systems That Turn Noise Into Signal
Intelligence is not a feature—it is a pipeline with failure modes. A deep dive into the canonical architecture of high-scale intelligence systems.
The OSINT platform teardown
Drop your email and I'll send it over — plus new essays as they land. No noise.
- → The failure modes that quietly sink platforms at scale.
- → A checklist for collection, resolution, scoring, and evidence.
- → What to build first when reliability and trust both matter.