How Search Engines Work

Before a page can rank, it has to be found. Search engines work in three basic steps: crawling, indexing and ranking.

Crawling is how bots discover pages by following links from one page to another. Indexing is how the page’s content gets stored and analyzed so it can be retrieved later. Ranking is what happens when someone searches: the engine decides which indexed pages best match the query, and in what order.

If a page isn’t crawled, it can’t be indexed. If it isn’t indexed, it can never rank, no matter how good the content is. This is why basics like sitemaps and robots.txt matter: they’re how you make sure search engines can actually find and read your pages in the first place.