Google recently published a page called How Search Works, which gives insight into their content policies and the techniques they use to crawl and index content. Google also published their Search Quality Rating Guidelines –an abridged version of the manual, used by raters to manually check the quality of Google’s results. It’s not the full version, but it provides us with insight into the process Google uses to determine the quality of search results. And for the sake of making this technical information more digestible, I’ve compiled the most pertinent information into a few short paragraphs that define search quality rating, explains the process and outlines current guidelines.
What is Search Quality Rating?
Search Quality Rating is a process that involves raters all over the world. The raters review search queries and URLs. Then, they provide feedback about the quality of the search results. Their feedback does not directly affect the index, but the information is harvested for consideration in future algorithm updates.
How It Works
First, the raters receive a query and a URL. The query includes keyword(s), and the language and country of the searcher. For example: [basketball] English (US). The rater reviews the query to establish the user’s interpretation and intent. Using the example above, the user may want to see a basketball game or research the stats of a certain player. Then, the rater visits the URL and rates how relevant it is to the query, based on the guidelines below. According to Google, “User intent and page utility are the most important factors to consider when rating.”
Some queries may have multiple interpretations. The interpretations are rated as dominant, common or minor. Dominant is what most people are seeking. For example, search for [windows] English (US) typically refers to the Microsoft operating system. Common is for queries that do not have a dominant interpretation. For example, [mercury] English (US) can be the car brand, the planet, or the element. Minor has only a few users in mind, such as a local, small business with the keyword in its name. Minor interpretations are assigned a lower rating.
The rater also assesses the user’s intent. Does the user want to take action, find information, or navigate somewhere? These criteria are also known as do-know-go. Action is for users who want to do something, such as download a file or watch a video. Information is for users that want to know about a certain topic. Navigation is for users who want to go to a certain webpage. For example: [youtube] English (US) to go the YouTube homepage. Some users may have multiple intentions. For example, a user may want to purchase a smartphone (do), research it (know), and visit the manufacturer’s website (go).
The next step is ranking the usefulness of the page to the user: vital, useful, relevant, slightly relevant, off-topic/useless, or unratable. Vital pages are the most-authoritative source, such as the official page of a celebrity, but not necessarily useful. Off-topic/useless is for pages with information that is irrelevant to the user. Unratable is used to classify pages that don’t load or use a language foreign to the user. The remaining categories are assigned based on the usefulness of the page.
Raters also flag pages for certain kinds of content: spam, porn, and malicious. Spam refers to pages that are deceptive and violate Webspam Guidelines. Porn refers to any pornographic images, content, pop-ups, etc., relative to the standards of the query location. Malicious