Voice recognition: Siri v Evi v Google

Published 21 July 2012

The Dream
Wouldn’t it be great if you could press a button, ask a question, and get the information back? This is what Apple’s Siri and other similar systems offered to do. So I thought I’d put them to the test.

Truth is, if you’re competent at typing into your smartphone’s browser, you’re probably better off ignoring the speech recognition feature. In my tests, I had to forgive a few misinterpretations: one of my questions was interpreted as “What is Tom Quimby house laser”, “What is truck can be an ounce laser”, “What is troponin beer and else laser” and “What is Hawkinsville mouth Malaysia”. Obviously, the real question was “What is Trockenbeerenauslese?”.

So, my quick answer is, if you can type, do so.

But, if you really insist on being lazy, what should you do? I tested:

• Apple’s Siri
• True Knowledge’s Evi
• Google’s speech recognition

Methodology
All tests were conducted using my iPhone 4s. I asked 20 questions to each platform:

• 9 general information (such as “What grapes are used in Champagne?”)
• 8 location-based (such as “Where is Walla Walla?”)
• 2 retail (such as “What is the cost of a bottle of Mumm Champagne in Waitrose?”)
• 1 calculation (regarding units of alcohol in a 250ml glass of wine),

I gave 10 points to the fastest/most accurate answer; 5 points for coming second; 0 points for third place or when it failed to provide an answer.

The Results
In short, use Google.

Overall scores: the split of points from the test shows Google to be a clear winner

Open Safari, go to the Google search bar (top right), then click the microphone button (left of the space bar), ask your question, press done, then click ‘Search’. This is a much longer process than using Siri and Evi, but the voice recognition works better and it returns better results. So much so, that it won 57% of the time (Evi won 19%; Siri won 10%; 14% none returned the correct result).

The successes (clockwise from top left): Google calculates units; Siri gives a detailed description of Gamay; Evi tells me the grapes in Champagne; Evi tells me how to store wine; Evi calculates the distance to Bordeaux; Google tells us where Walla Walla is

Evi was pretty good though. Although it was frequently beaten by Google, it was often a close contender. Both Google and Evi both answered the questions most often (they only failed 4 and 7 times, respectively).

In contrast, Siri failed to return an answer 12 times (that’s 60%!). This was partly because Siri only works in the US for location-based services, but it was still a dismal performance.

The fails: (Left) Siri had a (sometimes questionable) US-focus; (right) an example of Evi misunderstanding the question.