SERPTool · Docs

How SERPTool gathers data

A keyword tool is only as good as the signals underneath. This page is a transparent walkthrough of every data source SERPTool uses, where they come from, and what's cached.

The pipeline

For each keyword you submit, SERPTool runs roughly this sequence:

  1. Live SERP retrieval — top-10 organic results + every SERP feature present (People Also Ask, featured snippet, local pack, shopping, etc.), pulled fresh at the moment of the request.
  2. Keyword metrics lookup — monthly search volume, CPC, competition index, and the aggregate Keyword Difficulty score.
  3. Per-result metrics — for each of the 10 results returned, pull domain-level authority, backlink count, referring domains, spam score.
  4. Per-result page metrics — last-published date, page-level authority, word count, title/meta/H1 detection.
  5. Weakness detection — every result is scored against 17 weakness signals derived from the metrics above.
  6. Opportunity scoring — per-keyword composite score that blends weakness density, top-10 authority, difficulty, and search volume into a 0–100 rank-ability score.

Steps 1–4 live behind external APIs. Steps 5–6 are pure computation.

Where the external data comes from

DataForSEO is SERPTool's primary data provider. It's a wholesale API used by hundreds of SEO tools behind the scenes (including some well-known ones). We use three of their product families:

  • SERP API — live Google SERPs for any keyword in any of ~200 locations. Returns ranks 1–10 organic plus SERP features.
  • Labs API — keyword metrics: monthly volume, CPC, competition, keyword difficulty index. Also powers the DOMAIN and COMPETITOR modes.
  • Keywords API — used for the "AI Mentions" feature that measures how often a keyword is cited inside ChatGPT and Google AI Overviews.

DataForSEO's own data is stitched together from Google's public surfaces plus long-running crawls. It's the same under-the-hood source many bigger tools charge you a premium to access.

Google PageSpeed Insights (when configured) provides the LCP (Largest Contentful Paint) values used for the SLOW_PS weakness code. Free API, 25k requests per day.

No third party in the middle — SERPTool calls DataForSEO directly from our VPS. Your queries and results aren't shared with any analytics platform.

What's cached, and for how long

To keep costs manageable we cache a few things aggressively:

  • Domain metrics (DS, backlinks, referring domains, spam score) are cached per-domain for 7 days. If the same domain appears in 20 keywords we pay for that domain once.
  • SERP results are never cached — every keyword you analyze gets a fresh SERP snapshot, because SERPs churn daily and stale data is worse than no data.
  • Keyword metrics (volume, CPC, KD) are not cached within an analysis, but are stored on the Keyword row after the fact so the dashboard can show them without a re-query.

Why this approach vs a bigger suite

Big SEO suites run their own crawlers. That gets them richer historical graphs ("keyword X has been trending up for 6 months") but makes them expensive. SERPTool deliberately doesn't try to be an everything-tool: it's focused on the single workflow of surfacing rankable keywords from a list, using data that's accurate the moment you look at it. The tradeoff is no historical time-series; the upside is you pay for what you use, typically 20% less than the nearest branded alternative.

Next steps