Case 03AI and sales intelligence
LeadSpector
From 300,000 companies to the right pitch, with AI citing its sources.
A prospecting platform that combines Brazil’s federal company registry with Reclame Aqui complaints and Glassdoor reviews to show where each company hurts and what to offer it. Sellers filter 300,000 companies in seconds, open a diagnosis with an opportunity score and leave with the outreach email ready.
- Client
- B2B software house · in-house sales team
- Period
- 2025 — 2026
- Team
- 6-person squad
- Role
- Full stack engineer · frontend, RAG, insights and search
- Stack
- Go · Rod · Apache Spark · PostgreSQL · Neo4j · Elasticsearch
- Confidentiality
- Client data kept private
Results in numbers
- 300kcompanies searchable in secondsfederal registry data kept in sync with Elasticsearch
- 15 minto prepare outreach for one accountdown from about 2 hours of manual research
- 13 filterscombinable in lead searchregion, size, industry code, capital, years in business, score and status
- 4 factorsin the 0–100 opportunity scorepain, offer breadth, size fit and signal confidence
The business context
- Client
- B2B software house · in-house sales team
- Period
- 2025 — 2026
The software house sells system modernization, integration, data and AI to mid-sized and large companies. The sales team’s bottleneck was not closing deals: it was reaching the right account with the right argument. Each SDR built lists by hand, researched the company, read customer complaints and employee reviews looking for legacy systems, manual processes or instability, and only then wrote the first email. That meant hours per account, little market coverage and generic outreach.
The challenge
The signals exist, but they live in three places that do not talk to each other: the official company registry, in raw dumps with millions of rows; Reclame Aqui, with the end customer’s pain; and Glassdoor, with the pain of the people running the systems from the inside. Everything had to converge on the company ID, become searchable across hundreds of thousands of companies in seconds and turn loose text into sales opportunity, without letting the AI invent a problem the company does not have.
Three sources, three formats
Registry dumps in batches, Reclame Aqui in HTML with JSON-LD and Glassdoor in paginated GraphQL. Each source got its own collector, its own pace and per-company collection status.
Sales search across 300,000 companies
Sellers combine region, size, industry code, share capital, years in business and AI score in one query, and the list must arrive in seconds, already without other SDRs’ leads.
AI that cannot make things up
A wrong diagnosis burns the account. Every piece of evidence the model cites must exist in the text retrieved for that company; anything without an anchor is dropped before it reaches the screen.
What I delivered
I split the platform into collection, preparation, retrieval and use. A Spark ETL cleans the registry data, a daemon keeps Elasticsearch in sync with PostgreSQL and Go crawlers bring complaints and reviews into a Neo4j graph with local embeddings. The RAG chat and the insights engine, which scores each company from 0 to 100, run on top of that graph. Above it all, a React workspace brings together search, profile, pipeline, scheduling and emails.
Search across 300,000 companies
An Elasticsearch index with 13 combinable filters, a four-level industry taxonomy and per-user saved filters.
Diagnosis with an opportunity score
Three analysis fronts per company, audited evidence and a 0–100 score that orders the team’s queue.
RAG chat that cites its sources
Free-form questions about a company or an industry, answered with retrieved passages and an evidence table.
Prospecting workspace
A five-stage pipeline, scheduling, cadences, AI-generated emails with SMTP delivery and prompt control.
How it works
Follow the work through the system step by step, or click any component on the board to see what it does.
drag to see the flow
Legend
- Synchronous
- Asynchronous
- Secure channel
- External integration
Text description of the flow
- From Glassdoor crawler to Context graph — Reviews (asynchronous).
- From Reclame Aqui crawler to Local embeddings — Complaints (synchronous).
- From Federal registry to Summary worker — Registry data (synchronous).
- From Federal registry to Search index — Sync every 5 min (asynchronous).
- From Local embeddings to Context graph — Chunks + vectors (asynchronous).
- From Summary worker to Context graph — Profile per company (asynchronous).
- From Context graph to RAG chat — Context (synchronous).
- From Context graph to Insights engine — Pain signals (synchronous).
- From Search index to Lead search — Results (synchronous).
- From RAG chat to Sales workspace — Sourced answers (synchronous).
- From Insights engine to Sales workspace — Diagnosis (synchronous).
- From Insights engine to Prospecting and emails — Outreach draft (asynchronous).
- From Lead search to Sales workspace — Filtered leads (synchronous).
- From Sales workspace to Prospecting and emails — Lead in prospecting (synchronous).
Walk through the flow
←→change step
6 steps
From start to finish
Pick a step or press Play to follow the whole flow. The board highlights where the work goes as you move along.
System components
Glassdoor crawler
- What it solves
- Bring in the view of the people running the systems from the inside: employee reviews tied to the company ID.
- How it works
- A goroutine pool, three workers by default, reads active companies from PostgreSQL. For each one, the resolver finds the Glassdoor employer by matching name and website domain, which rules out namesakes. The interceptor captures the paginated GraphQL responses, up to 100 reviews per company, with a 30 s page timeout and retry with exponential backoff. Each company’s status lives in PostgreSQL and the reviews go to Neo4j.
Go · Rod · PostgreSQL · Neo4j
Reputation collection
- What it solves
- Bring the end customer’s pain into a searchable corpus, with no manual reading.
- How it works
- The Reclame Aqui crawler, written in Go with a headless browser, finds the company page, extracts complaints from JSON-LD and keeps up to 10 per company, with 15 s between companies and 3 s between complaints. A daemon moves the collected material into the graph in batches of 50: reduced pace during business hours, full speed outside them.
Go · Rod · PostgreSQL · Neo4j
Registry and index
- What it solves
- Make the full base of active companies searchable with sales filters.
- How it works
- An Apache Spark job reads the registry dumps, deduplicates by company ID, keeps only active companies and writes them to PostgreSQL. A sync daemon runs every 5 minutes: it indexes what changed in bulk batches and, on every cycle, reconciles the full set of company IDs between PostgreSQL and Elasticsearch so no company goes missing from search.
Java · Apache Spark · PostgreSQL · Elasticsearch
Chunking and embeddings
- What it solves
- Turn collected text into retrievable passages without sending raw data outside.
- How it works
- Text is segmented into Portuguese sentences and grouped into chunks of up to 1,000 characters, with one sentence of overlap and a content hash to prevent duplicates. Vectors come from nomic-embed-text running on Ollama, inside our infrastructure, and live on the Neo4j node itself, searched by cosine similarity.
TypeScript · Ollama · Neo4j
Company profile summaries
- What it solves
- Give sellers a first paragraph on who the company is before any question.
- How it works
- A Go worker finds companies without a summary in the graph, anchors industry and website in PostgreSQL, adds registry data and generates the profile with an open-weight LLM via Groq. Fixed pacing between companies, longer backoff on 429 errors and no company marked as enriched if the graph write fails.
Go · Groq · PostgreSQL · Neo4j
RAG chat
- What it solves
- Answer sales questions with the company’s own passages and show where every claim came from.
- How it works
- Greetings are answered without calling the model. A classifier separates questions about one company from broad searches. In the first case, retrieval brings chunks by similarity scoped to the 8-digit company ID, plus the profile summary and PostgreSQL firmographics. In the second, industry and state are recognized deterministically and become the same filters as the lead screen. Citations without an anchor in the context are removed and a second model checks the answer is supported. Capped at 30 questions per minute with a 75 s timeout.
TypeScript · Fastify · LangChain · Neo4j · Elasticsearch
Insights engine and score
- What it solves
- Tell sellers where the company hurts, with what evidence, and how much it is worth approaching.
- How it works
- Three analysis fronts (Architecture and Scalability, AI and Automation, Modernization and Engineering) run in parallel. Each retrieves up to 8 complaints and 6 employee reviews above the similarity threshold, and the model returns criticality, evidence and recommended services. Evidence without an anchor is dropped; a front without evidence does not light up. The score adds pain (40%), offer breadth (25%), size fit (20%) and signal confidence (15%). A PostgreSQL lease with a fencing token guarantees a single generation per company across the individual route, batches of up to 200 and the background job.
TypeScript · LangChain · Neo4j · PostgreSQL
Sales workspace
- What it solves
- Take sellers from the list to the sent email without switching tools.
- How it works
- React 19 with TanStack Router and Query. Tabbed filter search with the industry taxonomy; company profile with partners, contacts, summary, insights and chat; a five-stage pipeline with Kanban; SDR scheduling and a manager view; multichannel cadences; emails generated from the lit fronts, with SMTP delivery and IMAP reply tracking. Chat and per-front prompts are editable on screen, with required fields validated.
React · TypeScript · TanStack Router · Fastify
Decisions that hold the system up
Graph for context, index for search
Neo4j connects company, chunks, summary and reviews by company ID, with the vector on the same node: retrieval finds the right passages for the right company. Elasticsearch answers 13 filters over 300,000 documents. Each does what it does best.
Auditing before the screen
Every piece of evidence and every citation is checked against the context given to the model, by literal passage or word overlap. Anything that does not match is removed. Sellers only see claims they can trace back to the source.
Local embeddings, prompts in the team’s hands
Vectors generated on Ollama keep collected data inside the infrastructure and remove per-document cost. Prompts live in the database, editable on screen, with required placeholders validated on save: the team tunes tone and focus without breaking generation.
One generation per company, even under concurrency
Insights come from three paths: a click on the profile, a batch from the list and a background job with 10 workers and a persistent checkpoint. A PostgreSQL lease with a fencing token prevents duplicate generations and emails, and the job resumes where it stopped after a deploy.
Business results
Sellers build a qualified list in seconds, such as mid-sized manufacturers in the South with over ten years in business and a high score, instead of digging through spreadsheets.
Every account arrives with the diagnosis ready: where the company hurts, the evidence behind it and which of the firm’s services solves it.
Preparing outreach dropped from about two hours of research to 15 minutes of reading.
Research, pipeline, scheduling and email in one place: managers see each SDR’s portfolio and no context gets lost between tools.
My role
- Role
- Full stack engineer · frontend, RAG, insights and search
- Team
- 6-person squad
I built the Glassdoor crawler and the entire LeadSpector frontend. In the backend, leadspector-rag, I designed and implemented the RAG chat, the insights engine with its opportunity score, lead listing on Elasticsearch, prospecting management, email generation from the opportunities found and the prompt control that lets the sales team tune every generation without waiting for a deploy.
What I owned
- Wrote the Glassdoor crawler in Go: worker pool, employer resolution by name and domain, paginated GraphQL capture and retry with exponential backoff.
- Built the React 19 frontend with TanStack Router and Query: smart-filter search, company profile, pipeline, scheduling, emails and settings.
- Implemented the RAG chat: intent classification, retrieval scoped to the company ID, industry and state search, citation auditing and answer verification by a second model.
- Created the insights engine: three analysis fronts, anti-hallucination auditing, a weighted score and a PostgreSQL lease so the same company is never generated twice.
- Moved lead listing onto Elasticsearch, with 13 filters, score ordering and visibility by user role.
- Developed email generation and delivery, reply tracking over IMAP and prompt control with required fields validated on save.
Stack by layer
Collection
- Go
- Rod
- Java
- Apache Spark
Data and search
- PostgreSQL
- Neo4j
- Elasticsearch
- Redis
AI
- RAG
- LangChain
- Ollama
- nomic-embed-text
- Groq
Product
- TypeScript
- Fastify
- React 19
- TanStack Router
- TanStack Query
- Tailwind CSS
Infra
- Docker
- AWS EC2
- Caddy
- GitHub Actions
- Prometheus
- Grafana
What I take from this project
Sales AI earns the team’s trust when it shows the proof. The evidence table with each passage’s source did more for adoption than any model switch.
Company scoping needs a double lock: a filter in the query and sanitization after retrieval. Mixing two companies in one diagnosis is the worst possible mistake in prospecting.
Putting prompt control in the sales team’s hands cut tuning requests to engineering and let each analysis front speak the language of the people selling.