Spots 
🆕 TL;DR: My CrewAI competitive intelligence pipeline forgot everything between runs. I added a persistent memory layer (Hindsight plus typed events), and it now spots patterns across weeks instead of summarising one week's news.
I built a multi-agent pipeline that writes a cited competitive intelligence briefing. You give it a topic and some competitors, and CrewAI agents research the web, analyse what they find, and write a report. Every claim has to carry a citation that resolves to a real source.
It worked, but only for a single run. Every run started blind. Ask it about a competitor in week 8 and it had no idea weeks 1 to 7 existed. It would summarise that week's news and call it intelligence. That's a lookup with good formatting. A real analyst sees five moves in a quarter and says "that's a pattern," and that conclusion needs history.
So I built a persistent memory layer and put it in the middle of the pipeline. This post covers how it works, what changed, and what I got wrong. Before: four stateless agents
The original crew was four agents in sequence: Discovery, Research, Analyst, Writer. Each passed its output to the next, and everything was thrown away at the end. The governance layer already worked, with a citation guard and a prompt-injection guard. The gap was time. After: a Memory Agent between Research and Analyst The pipeline now has seven agents: Discovery, Research, Memory, Analyst, Strategy Evolution, Prediction, and Writer.
I use Hindsight as the persistent memory layer, with a class I wrote called HindsightStore in memory/hindsight_store.py acting as the application-level wrapper around it.
My application still maintains structured event and profile data locally, while Hindsight provides the persistent memory and retrieval layer. The local structured data is used for deterministic profile calculations, strategies, and predictions.Each finding is stored as a typed event using a Pydantic schema, CompetitorEvent. It holds a competitor, an event type (feature launch, pricing change, hiring, acquisition, funding, partnership, market signal), a date, a title, a description, an impact score, a confidence value, and evidence URLs.
I expose the memory functionality to my CrewAI agents through a HindsightStoreTool. The tool provides six operations: store_event, get_history, get_profile, search_memory, get_strategy, and get_predictions.
The Memory Agent stores new findings through the memory layer and retrieves relevant historical context before the Analyst runs. It can use get_history and get_profile to combine persistent memory with the structured competitor profile, then passes the memory-enriched context to the Analyst. Strategy and Prediction agents can also retrieve relevant historical information before producing their outputs. Every write updates a derived profile
Each store_event call also recomputes a CompetitorMemoryProfile for that competitor, with no batch job. Here's the core of it, from hindsight_store.py:
🆕 TL;DR: My CrewAI competitive intelligence pipeline forgot everything between runs.