Where Online Shopping Search Has Gone Wrong

While online shopping has exploded over the last decade and innovation around e-commerce has skyrocketed, there has been very little development around quality improvements in product search, navigation or discovery, notwithstanding the success of generic and open source search solutions, such as Elastic. That is ironic, as great search can make all the difference for an etailer’s bottom line.

Here’s the problem: Define exactly what “great search” is.

For starters, great search should be judged not only by how accurate it is, but also by how helpful it is. Did it deliver what the user needed? Did the user find it with relative ease in a short time frame?

What Users Want

Frequently, the problem with e-commerce search boils down to understanding a user’s intent. Search queries are often ambiguous and rarely have all the information necessary to capture a user’s full intent. Does the user want to buy, or just to learn more about a particular product or category? Is the user a novice or a professional?

Herein lies the challenge — for too long, consumers have been treated as a monolithic entity, and so have the technologies and software meant to serve them. Companies still haven’t delivered the personalization meant to capture and retain their audiences.

When people search for a router, for example, do they mean a WiFi networking device or a woodworking power tool? Google gives preference to the former in all of its ads and on the first-page of its organic results, only giving lip service to the latter by defining it in its top insert:

The conventional solution to extracting user intent from ambiguous search queries is to go with what “most users” intend. This is exactly what Google did in the above example, as far more people are interested in the networking kind of routers than the woodworking kind.

To accomplish this, search engines extract as much information as they can and then learn from it. They observe not only what people do when they’re searching, but also what they do after they search. Are they viewing items, adding them to their cart and purchasing? Are they scrolling without ever clicking on something? Or are they abandoning items in their cart?

With this information, search companies can better guide others who perform the same (or similar) queries. If a significant population of users is looking for a specific item without purchasing it, companies should be able to analyze this to identify the reason for abandonment, and appropriately address the issue.

Understanding this process solves many problems and is the main reason today’s search engines are vastly better than they were 15 years ago. They have more data and more facilities to analyze it.

Nevertheless, trying to understand what a “specific user” needs by understanding what “most users” need creates a whole new set of problems and friction for users.

So, for example, when you’ve got a piece of art and need an uncommon-sized picture mat to frame it, Amazon’s first 15 search results are irrelevant:

You have to scroll down to the 16th result, a virtual death zone in search terms, to see the first relevant result.

The 11×18 custom mat probably doesn’t sell enough to warrant a high position on the page. This is an example of a conscious decision that Amazon and many other search providers make — putting their desire to sell (what most people buy) ahead of their desire to satisfy (what you want to buy).

Amazon is counting on the user to somehow punctuate their needs if they want something off the beaten track — in this case adding double quotes (“11 x 18 picture mat”) will give you good results. However, this is simply too much to ask of users.

Or, maybe you want to know how to design a kitchen intended for a catering business. A Google search for “kitchens for catering” will show you primarily kitchen appliances and catering businesses — not what you’re looking for.

The query is ambiguous. The choice made by Google is to show you what “most users” are looking for but it is not what you want. To get what you need, you have to be more specific. (“Kitchens that are designed for catering” will get you what you are looking for.) This approach is not necessarily intuitive and may be just enough of a nuisance to cause the user to walk away from the query altogether.

Indeed, a lot of companies have been investing in search personalization according to a user’s profile to better understand user intent. However, it may not be as effective a strategy as it’s made out to be. Many computer scientists enjoy woodworking, and most professional woodworkers need WiFi. It turns out that a user’s intent is much more dependent on what they need now than anything about them from the past.

Customer Trust in the Balance

In an era of artificial intelligence and machine learning, the real challenge for search is to understand a user’s present and immediate need, despite ambiguity or incomplete information. That is the holy grail of search.

E-commerce search is somewhat easier than general search, but not by much. The purpose of e-commerce sites is to sell. Showing exactly what a consumer is looking for is one of the surefire ways to turn a profit. Online shopping was created for its ease, and making consumers scroll through page after page of products defeats the point entirely.

The intent of people who visit online retail sites is mostly to buy or to research buying. That would seem to make things simpler. If the goal of e-commerce is to sell, then the e-commerce search that brings the most sales, or the most profits, is the best one, isn’t it? If retailers present products that consumers are most likely to buy, then their purposes are served, or so the argument goes.

That’s not necessarily true, though. Improving sales in the short term actually may have a detrimental impact on sales in the long term.

Case in point: “Do you want fries with that?” is one of the most common and successful sales routines in business. It leads to more sales of fries, no question about it. If you try that line on customers who buy salads or ice cream, it may still lead to higher sales of fries, and if you run an experiment to see whether it’s good for the bottom line, you might conclude that it is.

That’s only for the short term, however. After a while, many customers — such as those buying salads and ice cream — will stop buying those extra fries, and instead just tune out those irrelevant sales pitches. Some even might stop visiting the stores that annoy them. In the long term, sales will go down.

The same is true for search results that show users items they are likely to buy, but that are not relevant to what they need now. Over time, such customers become irritated and tune out. The short-term gain of irrelevant pitches comes at the expense of customer trust, and that’s a much harder cost to measure.

E-commerce companies are at a crossroads, and it may just be high time they evaluate what’s worked and what hasn’t. That’s a sure way to improve customer retention and satisfaction, and eventually hike profits — by creating an easy-to-use, intuitive site with “great search.”