Duplicate Links: Why One URL Shows Up Many Times

## Why it matters

A raw link count lies. A page might report "450 links" but if 200 of them all point to your homepage (because the footer template links home), the real number of distinct destinations is 250. That's a huge difference.

The distinct URL count is what matters for auditing. How many different pages does this page actually link to? That's what reflects your internal link topology and crawl surface. The raw count inflates because templates repeat.

Deduping reveals the truth. It also helps you spot accidents. A page that links the homepage 15 times by mistake jumps out when you compare unique to raw counts. A real link map (50 distinct destinations) looks very different from a spammed one (200 raw links to 50 destinations means 150 are redundant).

## How it works

Deduplication normalises URLs and removes exact duplicates:

1. Lowercase the domain and scheme. https://Example.com/page and HTTPS://example.com/page are the same site.

2. Keep the path case-sensitive. Paths in URLs are case-sensitive by spec. /Page and /page are technically different. In practice on most servers they're the same (except Unix). The deduper treats them as the same because that's the pragmatic choice for an audit.

3. Ignore trailing slashes. example.com/about and example.com/about/ typically serve the same content. Most servers redirect one to the other. The deduper treats them as identical because they resolve to the same page.

4. Keep query strings and fragments. ?utm_source=newsletter and no query string are different links (one is tracked, one is not). A deduper can't drop them without losing information. But a toggle lets you treat ?utm_*=* parameters as noise and ignore them if you want.

5. Remove duplicates. Keep the first occurrence, drop the rest. The raw count is still shown so you can see how many links were redundant.

## What does not matter

Duplicates don't harm SEO. Google's crawler dedupes on its end. It sees a link repeated 15 times and counts it once. You're not penalised for internal redundancy. The issue is practical: a raw count that overstates your link topology by 3x makes auditing harder.

Canonical URLs are different. Deduping is local (one page) and is a report tool ("show me the distinct destinations"). Canonical is global (across pages) and is a directive ("when I publish this article at three URLs, index this one"). You need both, but they solve different problems.

Pagination links shouldn't be deduped. "Page 1," "Page 2," "Page 3" links to the same domain repeat on purpose. Deduping would hide them. A good tool lets you choose: include pagination or exclude it.

## Code example

A page with duplicate links:

<!-- Header nav -->
<nav>
  <a href="https://example.com/">Home</a>
  <a href="https://example.com/about">About</a>
  <a href="https://example.com/contact">Contact</a>
</nav>

<!-- Body -->
<article>
  <p>Learn more about us on our <a href="https://example.com/about">about page</a>.</p>
  <p>Visit our <a href="https://example.com/">homepage</a> for updates.</p>
</article>

<!-- Footer nav (same as header) -->
<footer>
  <a href="https://example.com/">Home</a>
  <a href="https://example.com/about">About</a>
  <a href="https://example.com/contact">Contact</a>
</footer>
Copy

Raw count: 9 links Unique count: 3 links (home, about, contact)

The deduper shows you both numbers. The jump from 9 to 3 tells you this page has heavy template redundancy.

A case normalisation example:

https://example.com/about
HTTPS://EXAMPLE.COM/ABOUT
https://Example.com/About
Copy

All three normalise to https://example.com/about. Counted as 1 unique link.

A trailing slash example:

https://example.com/products
https://example.com/products/
Copy

Treated as the same (1 unique link).

A query parameter example:

https://example.com/newsletter
https://example.com/newsletter?ref=header
https://example.com/newsletter?utm_source=social
Copy

Without filtering: 3 unique links. With utm_* ignored: 2 unique links (the two non-utm links collapse).

## How Scalpel Links shows it

The link list shows both counts at the top: "Raw: 450 | Unique: 180" (or whatever your page has). A toggle switches between raw and deduped view. In deduped mode, redundant links are hidden; you see only the first occurrence of each distinct URL.

The "Unique URLs" scope filters the link list to only show one copy of each destination. Use this mode to audit the real link topology without noise. Toggle back to raw to see where the redundancy is.

A badge on the list header shows the count. Hover it to see the difference ("177 duplicates removed").