Web Operations Field Guide

Pagination That Crawlers Can Follow: Links, Not Infinite Scroll Alone

Published

Long lists of articles or products are usually split into pages. Visitors can click through or scroll, but crawlers need plain links to move from one page to the next. If later pages cannot be reached by links, the items listed only on those pages may never be discovered through the list.

Unique URLs and plain links

Give each page in the series its own URL, commonly with a page parameter or a path segment. Link from each page to the next one, and ideally to the first page and nearby pages, with ordinary anchor elements. Fragments after a hash sign do not create separate pages for crawlers.

Google no longer uses rel=next and rel=prev as indexing signals, although the markup does no harm. The visible links matter more.

<nav aria-label="Pagination">
  <a href="/guides/?page=1">1</a>
  <a href="/guides/?page=2">2</a>
  <a href="/guides/?page=3">Next</a>
</nav>

Canonical tags across a series

Each page should usually declare itself as canonical. Pointing page two and beyond to the first page tells search engines the pages are duplicates, which they are not, and may lead engines to ignore links that appear only on later pages.

A view-all page can be a reasonable canonical if it loads quickly and contains every item.

Infinite scroll and load-more buttons

Infinite scroll is convenient for people but usually depends on scrolling events crawlers do not trigger. Pair it with paginated URLs: as new items load, update the address to the matching page URL, and provide links to those URLs in the HTML.

Check that a crawler can reach the last page of a long series by following links alone.