Page Density Checker
Python SEO toolkit: site crawler, keyword density checker, and duplicate-content detector.
Page Density Checker is an SEO toolkit built during my Master’s degree. The aim was simple: a budget-friendly, code-first alternative to the parts of paid SEO suites that are mostly arithmetic (crawling, word density, similarity), the kind of tools that are easy enough to write and unreasonable to license by the month.
Web crawler
A crawler walks a site and pulls per-page metadata for analysis: titles, headings, word counts, and the structural information the other three tools build on. It’s the foundation the rest of the toolkit runs on top of, rather than a standalone feature.
Page and search density checkers
The page density checker reports word density on any URL, useful for checking whether a target keyword is actually present at a reasonable frequency, or missing entirely. The search density checker goes a step further: run a query, fetch the top 10 Google results, and get their density profiles back side by side. That turns “what does a page that ranks for this term actually look like” from a manual, page-by-page exercise into one report covering the whole first page of results at once.
Duplicate content detection via cosine similarity
The similar page checker runs cosine similarity across a site’s pages to flag near-duplicates and cannibalization risk: two pages competing for the same query because they’re similar enough that Google can’t confidently pick one to rank. That’s a problem that’s invisible from inside a single page and only shows up when you compare documents directly, which is exactly what the similarity scoring is for.
Why build vs. buy
The individual pieces here (crawl a site, count words, compare two documents) are each a few hundred lines of Python once you know what to build. Paid SEO suites bundle them behind a subscription because the value they’re selling is the workflow around the numbers, not the arithmetic itself. Building the toolkit from scratch was as much about understanding what those suites actually compute under the hood as it was about avoiding the subscription.
Requests and BeautifulSoup handle the scraping, Pandas shapes the resulting data, and a simple NLP layer covers keyword extraction and similarity scoring, all in Python, no paid APIs anywhere in the pipeline.
Related projects
- AI-Powered Content Engine turns identified content gaps into published pages.
- IsVisible.ai tests whether search and AI crawlers can access those pages.