Understanding Intent as Part of Product Discovery
eCommerce search is expanding beyond its traditional role of retrieving relevant products to also include understanding shopper intent. While these goals are closely related, they are distinct problems that require different approaches in order to achieve an optimal solution. Large Language Models (LLMs) provide the contextual understanding needed to clarify shopper intent, whereas proven BM25-based search engines such as OpenSearch, Elasticsearch, and Solr use lexical search and search relevancy to locate and rank products with the precision needed for reliable transaction processing. This article discusses how these complementary technologies work together to create a more effective search experience.
Search need no longer be about just finding the thing you know you want. After all, if I know I want a red, short-sleeved t-shirt, a traditional search will be able to identify a list of products that match that description quickly and cheaply. Now, however, we can apply the power of LLMs to help a customer better understand the details of the product they should be searching for.
Carefully designed prompts that incorporate user input with descriptive information from a product catalog can leverage an LLM to discover details about the shirt that the customer may not have figured out on their own. It could lead them to a red, short-sleeved t-shirt or go in a completely different direction with a powder blue chiffon blouse. If all the customer knows is they need a top to go with their existing wardrobe but they’re not good at matching colors and styles, the LLM can assist in determining these details and make them more comfortable with the item they purchase.
The following process incorporates the best of what LLMs and traditional search can do to simulate a woman's experience of purchasing a blouse from a boutique. Through it, I want to create a concrete understanding of how these tools can work together to bring the online shopping experience closer to a real-world experience, including interactions like you might have with a knowledgeable, helpful sales assistant.
Using LLMs to Understand Shopper Intent
First we want to take advantage of the LLM to do things it is better at than a traditional search engine. A huge advantage it has over traditional search is the ability to make suggestions based on information that is not part of a product description. If a customer talked with a sales assistant and said they were looking for a blouse, the assistant might respond with something like, "What do you want it to go with?" This is completely unlike how a search engine works and provides a way for the customer to find what they want without actually knowing details about the product they are seeking. LLMs can simulate this sort of interaction with a prompt as follows:
Request: As a sales assistant in a boutique, a customer has arrived looking for a blouse. Provide a list of 3 kinds of apparel (with colors) worn in summer that a blouse would go with in a JSON array of strings.
Response: [ "white linen wide-leg trousers", "navy blue denim shorts", "olive green tiered midi skirt" ]
The response values could then be used as clickable links in a preformatted response such as, "Do you have something you'd like to pair your blouse with, such as white linen wide-leg trousers, navy blue denim shorts, or an olive green tiered midi skirt?" This could be displayed in a sidebar or other dialog area separate from the facet options provided by the search engine based on category selection of "blouse" the site visitor initially selected. Next, the details provided by the customer coupled with a RAG (Retrieval-Augmented Generation) input of styles, colors, fabrics, and patterns you have in stock (possibly queried from a traditional search engine) would be incorporated into a new LLM request:
Request:
As a sales assistant in a boutique, a customer has arrived looking for a blouse. The customer has stated they would like it to go with, "light brown tiered midi skirt". You have the following styles, colors, fabrics, and patterns available. Provide a JSON object with 5 common combinations under a node of "casual" and 2 combinations that would be considered eye catching under the node "stylish". Combinations do not need to use all four elements and should include an additional field ("display_text") with a concise, friendly phrase that can be used in a comma separated list.
Styles (node name: style):
"Classic Button-Down Blouse",
"Peplum Shirt",
"Wrap Shirt",
"Tunics",
"Tie-Neck Blouse (Pussy-Bow)",
"Off-the-Shoulder Shirt",
"Cold-Shoulder Shirt",
"Peter Pan Collar Blouse",
"Smocked Shirt",
"Ruffle Blouse",
"Bodysuit Shirt",
"Crop Top Shirt",
"Kimono Blouse",
"Cowl Neck Shirt",
"Asymmetrical Shirt",
"Boho/Peasant Blouse",
"Mandarin Blouse",
"Sleeveless Shell Top",
"Henley Blouse",
"Corset-Style Shirt"
Colors (node name: color):
"Crisp White",
"Alabaster",
"Ivory",
"Jet Black",
"Dove Grey",
"Charcoal Gray",
"Camel",
"Emerald Green",
"Ruby Red",
"Sapphire Blue",
"Midnight Blue",
"Deep Plum",
"Teal",
"Dusty Rose",
"Blush Pink",
"Powder Blue",
"Lavender",
"Sage Green",
"Periwinkle",
"Peach",
"Burnt Orange",
"Mustard Yellow",
"Burgundy",
"Olive Green",
"Chocolate Brown",
"Coral",
"Fuchsia",
"Cobalt Blue",
"Marigold",
"Mint Green"
Fabrics (node name: fabric):
"Silk",
"Polyester",
"Cotton Poplin",
"Chiffon",
"Rayon",
"Linen",
"Satin",
"Georgette",
"Chambray",
"Crepe de Chine"
Patterns (node name: pattern):
"Solid",
"Floral",
"Polka Dot",
"Pinstripe",
"Houndstooth",
"Leopard Print",
"Plaid",
"Paisley",
"Geometric",
"Abstract"
Response:
{
"casual": [
{
"style": "Classic Button-Down Blouse",
"color": "Crisp White",
"fabric": "Cotton Poplin",
"pattern": "Solid",
"display_text": "A crisp white cotton classic button-down"
},
{
"style": "Boho/Peasant Blouse",
"color": "Ivory",
"fabric": "Linen",
"pattern": "Solid",
"display_text": "A breezy ivory linen boho peasant blouse"
},
{
"style": "Sleeveless Shell Top",
"color": "Sage Green",
"fabric": "Rayon",
"pattern": "Solid",
"display_text": "A soft sage green sleeveless shell"
},
{
"style": "Tie-Neck Blouse (Pussy-Bow)",
"color": "Blush Pink",
"fabric": "Chiffon",
"pattern": "Floral",
"display_text": "A romantic floral chiffon tie-neck blouse in blush pink"
},
{
"style": "Wrap Shirt",
"color": "Chocolate Brown",
"fabric": "Satin",
"pattern": "Solid",
"display_text": "A sleek chocolate brown satin wrap shirt"
}
],
"stylish": [
{
"style": "Corset-Style Shirt",
"color": "Burnt Orange",
"fabric": "Linen",
"pattern": "Solid",
"display_text": "A striking burnt orange linen corset-style top"
},
{
"style": "Off-the-Shoulder Shirt",
"color": "Black",
"fabric": "Silk",
"pattern": "Leopard Print",
"display_text": "A bold leopard print silk off-the-shoulder blouse"
}
]
}
The LLM has not been asked to retrieve or invent products, so the list provided should not be seen as such. Instead, the LLM has translated shopper intent into a list of structured inputs to use when searching the actual product catalog. The customer came in looking for some kind of shirt, and the LLM has generated a list of the kinds of products that would better fulfill their specific need. This is similar to a sales clerk responding with, "Oh, I think a sleek chocolate brown satin wrap would go great with that!" and then escorting the customer to the appropriate clothing racks and shelves where items of that type can be found. The actual product is still unknown, but the LLM has developed enough context to point you in the right direction, so to speak.
Turning Shopper Intent into Product Discovery
Now that we have a list of the types of useful products, they can be presented to the user in a dialog window on their screen. It could look something like this:
It looks like we have a few options that could work for you! Any of the following should pair nicely: A crisp white cotton classic button-down A breezy ivory linen boho peasant blouse A soft sage green sleeveless shell A romantic floral chiffon tie-neck blouse in blush pink A sleek chocolate brown satin wrap shirt Or if you'd like to make an impression, you might go with: A striking burnt orange linen corset-style top A bold leopard print silk off-the-shoulder blouse
Each of the options could be a hyperlink that launches a new query with additional parameters based on the associated attribute field and value pairs. Before the shopper makes a selection, a search and discovery optimization platform such as FindTuner could use these attributes to adjust query weights, promote matching products, elevate relevant facet values, or prepare curated and visual navigation options. Selecting one of the options could launch queries with definitive filter specifications to reduce the result set entirely.
Doing the final query in a traditional search engine comes with guarantees that LLMs cannot compete with. Primarily, the results that are returned will definitely be products from your catalog, with accurate SKU, pricing, and other details. An LLM-generated response does not provide the same deterministic guarantee, even when the model is grounded through RAG using catalog data. Additionally, traditional search executes in milliseconds on your search infrastructure, while LLM requests introduce network and inference latency that is typically measured in seconds rather than milliseconds. Some query modifications may depend on shopper preferences, inventory conditions, margins, promotions, or other proprietary information that you may not wish to transmit to an external LLM provider.
LLMs and Search: Better Together
LLMs and search engines each have distinct and critical roles to play in the new realm of search. LLMs can help a customer figure out details about what they want in ways search engines are simply unable to replicate, but it comes at the cost of precision. Meanwhile, search engines generally perform product discovery best when shopper intent can be expressed through terms, attributes, filters, or other signals represented in the index. This means they rely on terms the user may not know or exhaustive navigation that frustrates users, but those terms can be incorporated into the context the LLM develops. Leveraging these technologies as complements to one another creates a far more expansive search experience than either can alone.