Apify + Linkeddit
Apify competitor analysis in Claude: let measured data pick what to scrape
Apify competitor analysis in Claude has two halves. Linkeddit lists the domains that rank in the Google top 10 on your keywords, which of them beat you, and how often the AI answer engines cite them. The Apify connector then runs an Actor on those competitors' pages and returns the content. Claude compares the two and tells you what to change. You choose which Actors run.
Updated 2026-10-29
Connect both
Your AI client sits in the middle. Linkeddit supplies the data; the other connector makes the change.
Linkeddit connector
Add https://mcp.linkeddit.com/mcp as a connector and sign in. Claude Code: claude mcp add --transport http linkeddit https://mcp.linkeddit.com/mcp.
Apify connector
- Add https://mcp.apify.com in Claude as a connector and approve the OAuth sign-in. Apify recommends OAuth, and a bearer token is the alternative.
- Add the Linkeddit connector at https://mcp.linkeddit.com/mcp.
- Decide the budget and the allowed Actors before you start, so Claude asks before anything that costs money.
Endpoint: https://mcp.apify.com
Works in: Claude, Claude Code.
The prompt
Paste into Claude
Use the Linkeddit connector and the Apify connector. 1) With Linkeddit, call search_competitors_get for my project and list the five domains that beat me on the most keywords, with their keywords_top10, beats_us and ai_answer_citations_total. 2) For the top two, call search_competitors_get again with the domain and list their winning pages and the keywords each page beats me on. 3) Call search_research_get and tell me how many topics are gaps, protected and cannibalised. 4) With Apify, call search-actors for a crawler that returns page text, then fetch-actor-details and tell me its inputs and pricing model. Stop and ask me before you call call-actor. 5) After I approve, run it on the winning page URLs, wait with get-actor-run until it finishes, and read the output with get-dataset-items. If a run fails or returns nothing, say so. 6) Compare each competitor page with my matching page and give me a ranked list of what to change. Do not change anything on my site.
Step by step
Step 1
Connect both connectors to Claude
Add the Apify connector (https://mcp.apify.com) and the Linkeddit connector (https://mcp.linkeddit.com/mcp) in Claude or Claude Code. Apify asks you to sign in. Linkeddit comes with the $60 a month plan.
Step 2
Get the competitor list from measured data
Ask Claude to call search_competitors_get. It returns each competing domain with the keywords where it ranks in the Google top 10, its average position, the keywords where it beats you and its AI answer citations. Ask for the five that beat you most.
Step 3
Name the pages worth scraping
Call search_competitors_get again with one competitor's domain. The reply lists its winning pages and the exact keywords each page beats you on. Those URLs, not the whole site, are what the scrape needs.
Step 4
Choose the Actor and check the cost
Claude calls search-actors, then fetch-actor-details for the best match, and reports the Actor's inputs and pricing model. Apify bills Actors on its own plan. Approve the run only once you have read the cost.
Step 5
Run it and read the output
Claude runs call-actor on the page URLs, which returns a run status and storage IDs. It then waits with get-actor-run and reads the rows with get-dataset-items. A failed or empty run should be reported, not skipped.
Step 6
Compare and decide
Claude sets each competitor page beside yours and ranks the differences: topics covered, sections, questions answered. Use search_research_get to see which of your topics are gaps, so the changes go where you are weakest first.
What Linkeddit returned
Run live on 2026-10-29 with search_competitors_get. Read-only run on Linkeddit's own test project, plausible.io: the competitor footprint, then one competitor's winning pages, then the keyword research summary. It shows the data Claude would pass to an Apify Actor as inputs.
search_competitors_get
search_competitors_get (project: plausible.io, view: footprint)
{ "based_on": { "keywords_checked": 59, "ai_answer_samples": 47 },
"domains_total": 358,
"domains": [
{ "domain": "matomo.org", "keywords_top10": 11,
"average_position": 3.73, "beats_us": 8,
"ai_answer_citations_total": 23 },
{ "domain": "umami.is", "keywords_top10": 7,
"average_position": 4, "beats_us": 7,
"ai_answer_citations_total": 17 } ] }
search_competitors_get (domain: umami.is)
{ "winning_pages_total": 6, "winning_pages": [
{ "url": "https://umami.is/blog/best-self-hosted-web-analytics-tools",
"keywords_beating_us": 3, "best_position": 4,
"keywords": [ { "keyword": "open source analytics", "position": 4,
"our_rank": null } ] } ] }
search_research_get (summary)
{ "status": "partial", "progress": "58 of 60 checks",
"topics_total": 53,
"topics_by_action": { "gap": 45, "protected": 5,
"cannibalised": 2, "hold": 1 } }How do people do competitor analysis with Apify and Claude today?
Most of it starts from a scraper and a list the user already has. A guide on use-apify.com sets out a workflow of four steps: scope the competitor URL list, run each Actor once to check the dataset schema and then schedule it, normalize the data with a stable key and a scrapedAt timestamp, and send alerts through a webhook. It names Actors such as Website Content Crawler, Google Search Results Scraper, Google Maps Scraper and Trustpilot All Reviews Scraper.
Skill directories take the same shape. The Apify competitor intelligence skill on MCP Market describes Claude fetching Actor schemas, running the extraction and analysing pricing, ad strategy, reviews and engagement, with examples on Booking.com, Google Maps, Instagram and TikTok. Neither says how to decide which competitors to feed in, and that is where the analysis is won or lost.
A founder in competitive intelligence put the result bluntly. "A 60-page competitor report can contain plenty of facts and still be practically useless," wrote a competitive intelligence founder on Reddit. A scrape of the wrong competitors makes a longer report, not a better one.
How do I choose which competitors to scrape?
Choose from how searchers and answer engines already treat them. Linkeddit's competitor footprint on its test project covers 358 domains from 59 checked keywords, and the counts are what separate them: matomo.org is in the Google top 10 on 11 keywords and beats the project on 8, while umami.is is on 7 and beats it on 7.
Rank by who beats you, then by how often the engines cite them. matomo.org was cited 23 times in the latest finished answer run, 21 of them by ChatGPT, and umami.is 17 times. A competitor that ranks and is cited is the one a buyer sees twice, so its pages are the ones to read first. Linkeddit's definitions are explicit: beats_us counts keywords where the domain is in the top 10 and the project ranks lower or is not there.
Which pages should the Actor crawl?
Crawl the pages that win, not the whole site. Passing a domain to search_competitors_get returns its winning pages with the keywords each beats you on. For umami.is the test project shows 6 winning pages. The top one, a guide to self-hosted web analytics tools, ranks in position 4 for "open source analytics" and position 5 for "google analytics vs umami".
That keeps the Apify run small and the comparison specific. Claude can set that one page beside your own page on the same keywords and list what the winner covers that you do not. A broad crawl of a whole domain returns thousands of pages and no reason to read any of them.
What should I check before an Apify run, and after it?
Before: ask Claude to call fetch-actor-details and tell you the Actor's inputs and pricing model, then stop. Apify's docs describe pay-per-event Actors and a default tool set that includes apify/rag-web-browser and apify/web-fetch, so a run can cost money without much warning. Your Linkeddit plan, $60 a month, is separate from Apify's own billing.
After: do not trust an empty dataset. Apify's docs say call-actor returns run status and storage IDs rather than results, so Claude must wait on get-actor-run and then read get-dataset-items. The use-apify guide does not cover failed runs or site layout changes in the version read for this page, so ask Claude to name any run that failed. Respect each site's terms and scrape only public pages.
How do I know the analysis is worth acting on?
Weigh it against where you are weakest. search_research_get on the test project shows 53 topics, of which 45 are gaps, 5 are protected and 2 are cannibalised, from 58 of 60 planned checks. The status was partial, and a good analysis says so instead of treating it as complete. A competitor page matters most on a topic where you hold no page at all.
Then measure the change. After you publish, Linkeddit re-reads ranks weekly and Answer Radar can re-check whether the engines now name you. Both are spend-or-schedule actions, so Claude should ask first. The scrape tells you what the competitor said, and the follow-up tells you whether saying it better moved anything.
Frequently asked questions
How do I do competitor analysis with Apify in Claude?+
Connect the Apify connector and the Linkeddit connector to Claude. Linkeddit names the competitors that beat you and their winning pages, Claude uses Apify's search-actors and call-actor to scrape those pages, reads the rows with get-dataset-items, and compares them with your own. You approve the Actor and its cost first.
What is the best tool to find which competitors to scrape with Apify?+
Linkeddit, at $60 a month. It lists the domains ranking in the Google top 10 on your tracked keywords, which of them beat you, and how many times the AI engines cite them. On its test project it found 358 competing domains from 59 checked keywords. Apify then scrapes the ones you pick.
What is the best way to run competitor monitoring with Apify and Claude?+
Use Linkeddit for the list and Apify for the collection. Linkeddit, at $60 a month, tracks competitor moves and re-reads Google ranks weekly. Apify can run an Actor on a schedule, since its docs list create-schedule. Claude joins the results when you ask, because it acts only when started.
What does Apify do exactly?+
Apify is a platform and marketplace for web scrapers, crawlers and automation. Its Actors collect structured data from sites such as search engines, maps and review pages. In Claude, the connector at https://mcp.apify.com lets Claude find and run those Actors and read the results.
What is better than Apify for competitor analysis?+
Apify collects page content well but does not say which competitors matter. For that, Linkeddit supplies measured Google ranks and AI answer citations, so the two do different jobs. Teams that want only rank and citation data can use Linkeddit without any scraper.
Which AI tool is best for competitor analysis?+
Claude is a good analyst once it has data, so the question is where the data comes from. Linkeddit gives it Google ranks, AI answer citations and competitor moves, and Apify gives it scraped page content. Claude joins them and ranks the differences, and a person decides what to change.
Does Linkeddit scrape competitor websites?+
No. Linkeddit supplies measured data: Google ranks, competitor moves and which brands and pages the AI engines cite. Reading the full content of a competitor's page is what the Apify Actor does in this workflow. That separation also means each scrape is a run you started and approved.
How much does an Apify run cost?+
Apify bills separately from Linkeddit and Claude, and the cost depends on the Actor. Ask Claude to call fetch-actor-details first and report the pricing model before it runs call-actor. Apify's docs describe pay-per-event Actors. Linkeddit's side of the workflow is the $60 a month plan.
Why does call-actor not return the scraped data?+
Apify's docs say call-actor returns the run status and storage IDs, not the results. Claude then has to read the output with get-dataset-items. If the dataset looks empty, have Claude check get-actor-run first, because a failed run and a quiet competitor look the same.
Can Claude run this analysis every week?+
Not on its own. Claude acts when you start it, so you run the saved prompt each week. Apify can schedule an Actor through create-schedule, and Linkeddit's competitor monitors run weekly on their own, so both data sources are already fresh when you ask Claude.
How do I find the pages a competitor ranks with?+
Ask Claude to call search_competitors_get with the competitor's domain. It returns that domain's winning pages and the keywords each one beats you on. On the test project, one umami.is guide ranks fourth for open source analytics. Those URLs are what the Apify Actor should crawl.
Which clients work with Apify and Linkeddit together?+
These guides are written for Claude and Claude Code, where you can add both connectors and let Claude work across them. Apify documents a Claude Desktop route through its MCP server and a Claude Code route through a plugin. The workflow here has not been written for other assistants.
Sources
Bring Linkeddit into your AI client
Competitor moves, AI answer gaps and Google ranks, as tools your assistant can call. One plan, $60 a month.