Detecting Company Expansion Signals with WTTJ, LinkedIn, and n8n¶
The problem I was trying to solve wasn't technical at first. It was a timing problem.
In B2B, the moment you reach out to a company changes almost everything. Same company, same profile, same pitch: if you show up at the right moment, it's a conversation. Otherwise, it's an ignored follow-up.
Traditional databases describe companies well. They say almost nothing about their timing.
That's what this project set out to fix: building a pipeline capable of automatically detecting that a company is probably in an expansion phase, using public sources.
The Starting Problem¶
A sales team typically has access to precise data about its targets: size, industry, tech stack, estimated revenue. That profile tells you what a company is. It doesn't tell you what it's doing right now.
Yet expansion signals are public. Scattered, noisy, hard to aggregate, but public.
A company that opens 15 positions in six weeks, publishes LinkedIn posts about a geographic expansion, and announces a new partnership on its blog is not in the same state as an apparently similar company that has been static for 18 months.
The real question
The question wasn't "how to scrape data". It was "how to turn messy public data into commercially actionable signals, without mostly generating false positives".
Why Existing Tools Didn't Work¶
| Problem | What it produces |
|---|---|
| Static data | Little context about timing |
| Generic signals | High volume, poor signal-to-noise ratio |
| Opaque scoring | Impossible to understand the reasoning |
| Cost at scale | Hard to experiment without a significant budget |
Most sales intelligence tools never show the reasoning behind the signal. I wanted to see the source, the content, the logic, the timing. Almost like an investigation notebook.
Sources Used¶
The pipeline relies on three types of complementary sources:
What it reveals: active job listings, contract types, locations, volume of open positions.
The approach: no HTML scraping. WTTJ exposes its Algolia index in the frontend: a single structured POST request is enough.
What it doesn't tell you alone: whether the positions are real or have been automatically reposted for six months.
What it reveals: company posts, launch announcements, statements about growth, featured job openings.
The approach: via Apify to avoid maintaining a fragile homegrown scraper. Same logic as in the article on multi-source competitive monitoring.
What it doesn't tell you: the majority of posts are generic employer branding content. The signal-to-noise ratio is the highest of the three sources.
What it reveals: funding rounds, partnerships, product launches, market expansions, intentional announcements.
The approach: HTTP crawl of /blog, /actualites, /news paths for each company, same heuristic logic as in the Scrapy technology detection project.
What it doesn't tell you: some blogs have been abandoned for two years. The crawler detects the absence of recent content and moves on.
Overall Architecture¶
flowchart TD
A[(Supabase\nentreprises)] --> B[n8n Scheduler]
B --> C{Loop Over Items}
C --> D[HTTP Algolia\nWTTJ]
C --> E[Apify\nLinkedIn]
C --> F[HTTP\nPages actualités]
D --> G[Code\nParse Jobs]
E --> H[Code\nParse Posts]
F --> I[Code\nParse Contenu]
G --> J[Assemblage\ncontexte]
H --> J
I --> J
J --> K[AI Agent\nOpenRouter]
K --> L[Parsing\nJSON]
L --> M[(Supabase\nsignaux)]Redis comes in upstream of the loop: a semaphore limits the number of parallel executions to avoid overwhelming external APIs when processing a large list of companies.
The Real Discovery: WTTJ Uses Algolia¶
At first, I was planning to do classic HTML scraping of Welcome to the Jungle. Then I opened the DevTools network tab.
WTTJ loads its job listings via Algolia requests, the search engine they use internally. And their search key is present in the frontend, accessible to anyone who inspects the network requests.
This isn't an exploit. It's Algolia's normal behavior: Search-Only API Keys are designed to be exposed client-side. They allow reading data, not modifying it. WTTJ is fully aware of this.
It completely changes the approach: instead of parsing fragile HTML, you make a structured POST request to their index.
POST https://<APP_ID>-dsn.algolia.net/1/indexes/wttj_jobs_production_fr/query
X-Algolia-Application-Id: <APP_ID>
X-Algolia-API-Key: <search-only key visible dans les DevTools WTTJ>
Content-Type: application/json
The response is clean, structured, and unambiguous. No CSS selectors to maintain, no JavaScript rendering to handle.
The WTTJ slug
The organization.slug field is not always identical to the domain or company name. It needs to be stored explicitly in the database. In this pipeline, I added a wttj_slug_computed field to the companies table in Supabase, filled in manually or via a prior enrichment step.
This is a real friction point: on the first runs, around 15% of companies had no slug filled in. The API returned 0 hits, technically correct but silent.
Parsing Job Listings¶
After the Algolia request, a Code node extracts the useful fields from the hits:
const resp = $input.first().json;
const domain = ($('Loop Over Items').item.json.domain || '').trim();
const slug = ($('Loop Over Items').item.json.wttj_slug_computed || '').trim();
const hits = Array.isArray(resp?.hits) ? resp.hits : [];
const nbJobs = resp?.nbHits ?? hits.length;
const jobs = hits.slice(0, 20).map(j => ({
title: j.name || null,
contract: j.contract_type || null,
remote: j.remote || null,
location: j.offices?.[0]?.city || null,
published_at: j.published_at || null,
url: j.organization?.slug && j.slug
? `https://www.welcometothejungle.com/fr/companies/${j.organization.slug}/jobs/${j.slug}`
: null,
}));
return [{ json: { domain, wttj_slug: slug, total_jobs: nbJobs, jobs } }];
What this produces for an active company:
{
"domain": "example.fr",
"total_jobs": 23,
"jobs": [
{
"title": "Senior Backend Engineer",
"contract": "CDI",
"remote": "hybrid",
"location": "Paris",
"published_at": "2026-05-12T08:00:00Z",
"url": "https://www.welcometothejungle.com/fr/companies/example/jobs/senior-backend-engineer"
}
]
}
The Analysis Prompt¶
Once job listings are parsed and assembled with data from the other sources, an AI Agent analyzes the context.
The choice of persona in the system prompt is intentional. "Analyze these job listings" produces well-written summaries that are useless in a pipeline. "You are a senior B2B account manager looking for upsell opportunities" steers the model toward what you actually want.
Tu es un account manager B2B senior spécialisé en expansion de revenu (upsell et cross-sell).
Ton rôle : analyser les offres d'emploi d'un client sur Welcome to the Jungle pour identifier
des opportunités commerciales.
Chaque insight doit être basé sur un signal précis (rôle recruté, volume, type de contrat).
Tu identifies uniquement des opportunités exploitables, pas des résumés.
Tu réponds uniquement en JSON valide, sans markdown, sans texte hors JSON.
The strict JSON constraint is essential. Without it, the model inserts text between code blocks and downstream parsing breaks.
The model: openai/gpt-4o-mini via OpenRouter. Good cost-to-quality ratio for a classification and structured extraction task at volume.
Redis: Bounding Concurrency¶
With a reasonably sized company database, n8n naturally launches several branches in parallel inside the loop. Without control, this quickly becomes dozens of simultaneous requests to Algolia, Apify, and external APIs.
I use Redis as a simple semaphore: before entering processing, each iteration acquires a slot via INCR on a counter key. If the limit is reached, it waits. After processing, DECR.
No external library, no Celery, no full queue. A native Redis node in n8n is enough for this.
What Broke¶
Reposted listings¶
The most frequent problem on WTTJ: a company reposts the same listing every few weeks. published_at refreshes, the total climbs, but nothing particularly new is actually happening.
The fix: hash on (title, contract, location) per company. If the same triplet already exists in Supabase with a recent date, ignore it. The raw total_jobs is still tracked separately to detect genuine volume variations over time.
The silent missing slug¶
An Algolia response with 0 hits can mean two very different things: the company genuinely has no open positions, or the slug is missing from the database. Treated the same way initially, both cases were masking coverage gaps.
Added an explicit log in the Code Parse node to distinguish the two situations. Simple, but not done from the start.
Prompts that were too open-ended¶
First version of the system prompt: "identify expansion signals". Result: well-written text, no structure, impossible to parse downstream. The model extrapolates and generalizes when not constrained.
Strict JSON combined with an explicit taxonomy of expected outputs changed everything.
The classic trap
The model doesn't "understand" the company. It completes patterns. If the prompt is vague, the output will be plausible but not reliable. Constraining the output format and the persona isn't a detail: it's the main tuning variable.
What I Learned¶
- Inspect the network before scraping. WTTJ would have taken several days of development with HTML parsing. Algolia reduced it to under an hour. Opening DevTools first has become a systematic reflex.
- Redis for throttling, not for caching. Very targeted use: bounding concurrency, nothing more. Don't over-engineer.
- The deduplication hash is essential. Without it, reposted listings pollute signals indefinitely. Implement it from the start, not as a patch later.
- Log "0 results" differently from errors. An empty response can be correct or symptomatic. Treating them the same hides bugs.
- A single signal almost never means anything. 20 WTTJ listings can indicate strong growth or massive turnover. It's the combination with other sources that produces context.
- Raw total_jobs is a red herring. Variation over time is more useful than instantaneous volume. What you actually want is a rolling 4-week history.
- The persona in the prompt changes output quality. "Analyze" produces summaries. "You're looking for concrete upsell opportunities" produces actions.
- Store slugs from the start. Any enrichment not done upfront is expensive to catch up on at scale.
- Modularize by source. Each source (WTTJ, LinkedIn, news pages) in an independent sub-workflow, testable without rerunning the whole thing.
- Plan for contradictory signals. A company can be hiring aggressively while losing customers. Aggregating without cross-referencing produces noise.
Current Limitations¶
| Topic | Status |
|---|---|
| wttj_slug | Needs manual enrichment or a dedicated step |
| Deduplication | Basic hash, doesn't cover title reformulations |
| Apify dependency, fragile if the actor changes | |
| Trend history | Not yet implemented for WTTJ volume |
| Scoring | Still empirical, no real-world benchmark |
| Algolia index | Configured for wttj_jobs_production_fr only |
Conclusion¶
The project started as a scraping system. What it became is mostly a system for reading organizations.
The Algolia discovery illustrates the general approach well: before writing code, look at what already exists. The best technical solution is often the one that invents nothing.
The scraping itself was the simplest part. The real difficulty was elsewhere: distinguishing a genuine signal from a listing that's been sitting around, avoiding false positives on companies that communicate without moving, making the output usable by someone who doesn't read raw data.
Interesting signals are rarely spectacular. They are often repetitions, changes in rhythm, weak correlations, and gradual movements.
What remains to be built: temporal weighting of signals, a trend score based on the evolution of WTTJ volume over a rolling 4-week window, and correlation (the real test) with actual sales outcomes.
What I want to explore next
- Trend score based on the evolution of
total_jobsover 4 weeks - Cycle detection: hiring rhythms, communication variations
- Correlation with sales conversion rates: do companies detected as "expanding" convert better?
- Temporal weighting: a signal three months old doesn't carry the same weight as a listing published yesterday