Much has been written recently about the peculiarities of Google’s Quick Answer Boxes. Many site owners are concerned that content is being scrapped and showcased on Google itself, while SEOs are puzzling over why some boxes come with a citation, while many do not. Then there’s the pesky question of whether or not Quick Answer Boxes are even part of the proper Knowledge Graph. After all, how is scrapping a page particularly intelligent?
I recently devoted time to poking and prodding this feature. What I found suggests that Quick Answer Boxes are less than ideal for certain types of queries, and are sometimes featured when organic results would better serve the user.
Example 1: Don’t Know Much about History
When it comes to search, Google is only as good as its sources. Unfortunately, plenty of people post things on the internet that just aren’t true (I know, Grandma – it shocked me, too!). Malicious lies aside, let’s consider an example regarding what many accept as common knowledge. Below is the answer box I was served for the query, “when was the declaration of independence signed”:
Google is confident in the July 4, 1776 answer, and has displayed text scrapped from History.com to back-up this assertion. However, those who take the time to read the paragraph will realize they’ve been misled. History.com is actually reporting that the document “wasn’t signed on July 4, 1776.” Many other sites also call attention to this misconception, so why has Google accepted the “common knowledge” answer?
My suspicion is that there are two culprits at play: bias and verb confusion. When I search for the exact statement, “The Declaration of Independence was signed on July 4, 1776”, Google returns 37,100 results. Meanwhile, when I run the same search with the date historians believe is correct, (August 2, 1776) I only get 12,200 results.
If Google is simply crawling pages for an answer, July 4th would seem to be the safe bet. Most users are probably satisfied with this result, as it is the nationally celebrated birthday of our great nation. Due to this misconception, there are probably few behavioral signals to indicate to Google that the result is, in fact, wrong.
I repeated the original query several days later. While the same date was displayed, the accompanying text and source changed:
Here we have Archives.gov writing that the document was “adopted” on July 4, 1776. The delegates “began signing it” on August 2, 1776. At least in this case, “adopted” and “signed” seem to be close enough to keep the erroneous answer front and center. Google failed to understand the less straightforward line with the correct answer: “on August 2, 1776, delegates began signing it.”
I wanted to test the verb confusion hypothesis with another date, so I searched for “when was the magna carta signed”. The following answer box is courtesy of Wikipedia:
While the date isn’t served up top, it is bolded. What I find interesting is that there’s no version of the verb “signed” in the text. Instead, it was “sealed under oath” on 15 June 1215. I have no idea if anyone actually signed the Magna Carta, but Google has made the assumption that this is a satisfactory answer.
This is a case where the Quick Answer Box does a disservice to the user by jumping to conclusions. Simply presenting the organic results would’ve allowed users to consider the controversy, while bolding July 4, 1776 misleads them.
If the Knowledge Graph is ever to reach its full potential as an answer engine, it will need to be sensitive to conflicting results that require a human tie breaker. Sometimes “I don’t know” is the right answer.
Example 2: Welcome to Jurassic Park?
Misconceptions aren’t the only things Google has to be concerned with. Humanity has created fictional people, places, and things which have become integral parts of our culture. L. Frank Baum wrote dozens of books about the magical world of Oz, and, if you take films at face value, Indiana Jones was a thorn in Hitler’s side.
While our brains are powerful enough to consider contextual clues, a skill which makes distinguishing fact from fiction a cinch, Google’s not quite there yet.
To further prove this point, I tried my luck using Google to find a fictional location. Like many of my 30-something peers, I’m a big Jurassic Park fan. So I asked, “where is jurassic park”. Much to my surprise, I got a straightforward answer:
The culprit is actually WikiTravel.org, a seemingly upstanding site that decided to create a page for Jurassic Park on April Fools’ Day. Unfortunately, Google didn’t get the joke. The Jurassic Park page does contain a disclaimer at the bottom. I suppose that the content was so well-suited to my query, Google returned it without seeking corroborating evidence from other travel sites.
When I tried the same query on Bing, I was initially served information about the film. However, Bing offered a search suggestion for “Where is Jurassic Park located?” When I clicked on that, I received a suitable answer:
Bing managed to avoid the WikiTravel trap and determined that, unfortunately, I cannot book a trip to Jurassic Park after all (sorry for getting your hopes up, teenage me).
Regarding Google, this seems to be yet another case where the search engine was eager to please me with a Quick Answer Box. As a result, it assigned undeserved weight to a single source. While the ultimate goal of the Knowledge Graph – and advanced search in general – is to classify entities according to their relations, any search tool needs the ability to understand when an answer is ambiguous or contested. In fact, the examples above suggest that good, old-fashioned organic results are sometimes the best way to go.
As engineers build out the Knowledge Graph and Quick Answer Box, I’d remind them that it’s OK to ask for help. Humans create fiction and misconceptions, so bringing us back into the loop is often the best way to shake-out the truth.
In the end, this is not an indictment of the Knowledge Graph or search algorithms. Instead, we’re brought to the realization that human knowledge is strangely ambiguous, and until computers can integrate this into their structured knowledge bases, there will always be room for people to weigh in.