Paul Sonneveld
Hi, everyone, and welcome to Marketplace Masters. I'm Paul Sonneveld, and today we're going to look at a shelf on Amazon that most brands have never ordered it, the AI Shopping Assistant. Every brand watching this has an organic rank strategy and a paid ad strategy. Two shelves, well understood, heavily optimised. But shoppers are increasingly asking Amazon's AI what to buy. Amazon itself says that the Rufus engine behind Alexa for shopping helped over three hundred million customers in 2025. And the assistant's answer is a third shelf entirely, one that plays by very different rules. And until now, nobody could really tell what those rules are.
Now, joining me today is Christian Umbach. He's the co-founder and CEO of Autopilot, a company dedicated to boosting merchant profitability and preserving consumer choice for billions worldwide. Now, before Autopilot, he co-founded and led Xapix.io, an API and enterprise machine learning platform that helped companies orchestrate and exchange data across systems. Trained as a systems engineer through MIT System Design and Management Fellowship, he's worked at the edges of hard operational problems with experience connected to Uber, Hyperloop, and Lufthansa. The thought line of his career is consistent. Take a messy manual system, understand it deeply, and then let it run on its own. Christian, it is so good to have your Marketplace Masters today. Welcome to the show.
Christian Umbach
Thanks, Paul. Excited to join and share a bit about our story and on our thesis of the third shelf.
Paul Sonneveld
Yeah, excellent. I think it's a super interesting topic, this whole concept of this third shelf. I must admit, I probably fall into that category where I don't pay enough attention to it. So yeah, let's get into the conversation and before I do, for those that are watching this live, if you do have questions for christian you know I'll be asking him a number of questions, but if you've got questions of your own, don't hesitate to put them in the LinkedIn comments or on the YouTube comments, and I will put them through him. So we're gonna have a very interesting discussion today, but let's start here. Most brands, and this was in my intro too, most brands describe Amazon as a two-shelf game, organic search and paid ads. Give us really the one-line case for why the AI system is generally a third shelf, as opposed to just a smarter search result search box. Why is it a genuine third shelf?
Christian Umbach
Yep. So we see it as a genuine third shell because it's working under different inputs, different assumptions, and most importantly, it's producing different results. And that's really what our study here that we ran across a large set of ASINs, across seventy thousand products focused on, which was, OK, is it different or is it following the patterns of organic search? Is it following patterns of paid search, or does it maybe build on some of these patterns that actually has a pattern on its own? And we confirmed that through the study, and happy to share more about that today.
Paul Sonneveld
Yeah, I look forward to really getting into that. Maybe but before that like how do you even study an AI assistant at scale I mean it's kind of sort of, how you do the study is kind of an interesting question in itself for me like what methodology did you come up with, what do you actually measure,how many queries data points, give us a sense of the work undertaken and how you went about that question.
Christian Umbach
Yep. So obviously, what we wanted to find out through the study is what do humans see, what do shoppers see today, not agent shoppers, but you and I, when we go to Amazon.com and search for products. So the way for us to approach this study was really, we didn't want to use any scraping on the site because Amazon detects that, and either you get shut down but it certainly impacts the search results that you're receiving. So the way we approach that is we designed about two thousand non-branded queries across different categories. So we said we want to test two hundred, two hundred fifty queries within the apparel space, within supplements, within home and kitchen, within beauty and within a couple of other categories.
We also had a set of branded query tests, but for the purpose of what we're sharing today, it's mainly focused on the unbranded side. So we took two thousand queries, and we would literally by hand go to one the traditional search and look for you know sneakers for example or running shoes and then on most of the pages as you're logged in and we were always logged in with a US account. You would also already see the Alexa search bar. So in those cases, we would also there go in and say, okay, what is the best running shoe, running sneaker? And then basically like our job was to look at what are the Alexa results that are showing up versus what does the detail, what does the search result page look like and what's paid versus organic there.
And at the end of the day, then take, those two thousand queries, you know, that gives you about a thousand just ASIN results if you kind of, yeah, or kind of distinct ASINs across all of the searches. If you just add up the full detail, the full search result page, and we would have like within that set then about thirteen, thirteen and a half thousand Alexa-specific ASIN recommendations. So we would see those thirteen thousand recommendations that Alexa gave us. Here's a hundred ten thousand products that came up on the search result page, you know, where they paid, where they're organic, where, you know, what's kind of their placement, if they were both. And then we saw that, you know, obviously, there are some duplicate ASINs in there. So you could say that the study overall captured about seventy thousand unique ASINs.
Paul Sonneveld
That is a very comprehensive effort. So, of course, the first question, the first question that I'm sure every brand has on their mind, I'm really curious, is know if you compare those results right Alexa to the standard search result you know, is the assistant recommending the same products? Going back to your like your point you made about this is a genuine third shell, is it recommending the same product that kind of the SEO and and the the ad strategy are built around, or are you seeing some very different results?
Christian Umbach
Yep, and we'll actually have some numbers to share on that one as well. So at the very core, we saw that sixty four percent of the top AI picks or the Alexa picks were not part or are not part of the organic top. And I have to show you this a little bit on a different view on this data. The number one suggestion that Alexa makes still around fifty five percent of the time is also within the organic top ten. But then if we look at, for example, go further down the Alexa suggestions, and what we did was basically we kept asking it, don't only give me one or two products, what are other products there? So if we look at suggestion number eight, for instance, that would only be in the organic top ten, about eighteen percent of the time overall.
And what we took from that, and obviously within organic, you only have so much space, but that there is a clear diversification of results. The reason why a product is in the organic top ten, which builds around Amazon's quality, sales velocity and reviews and all of that gives you also a higher chance of winning with Alexa. But it is not a guaranteed ticket into that. And a different way, our friends at Marketplace polls took that same view and presented the data in a slightly different way, basically saying, organic top ten sort of give you a six percent chance of getting into the Alexa results. But then The other side, I think that for us was probably the most fascinating result that, forty percent of the results that Alexa gave us, and we never asked more than ten suggestions, but forty percent of those results are not anywhere, whether it's sponsored or organic, on the natural, traditional results search page.
Paul Sonneveld
Yeah, that's so interesting because we know how heavily weighted that first set of results is in terms of the sales against that particular search term. So being able to somehow squeeze your way in there, maybe through Alexa, through AI, is definitely an incremental opportunity, particularly for those brands that maybe haven't established that really strong ranking yet. And of course, you know if there was marketers here you know the first question is like I know I've heard about ChatGPT, there's some you know advertising engine there you know there's something going on there, is there a way I can influence the results here with ad spend in terms of Alexa? Can I essentially buy my way into Alexa's and AI recommendations on Amazon?
Christian Umbach
Yep, and we have a similar question. Basically, the answer that we found was if an advertised product is coming up in Alexa, your chances, first of all, are lower being in Alexa than they are being on the actual search results page. So, you know, twenty point seven percent of overall results that we saw on the general pages, like the search results page, were sponsored in the Alexa pics on fourteen percent of those were sponsored. And then if your hopes are that maybe you're not showing up on the top there, but you can buy your way into it.
Unfortunately, that's not a viable path. Only two point four percent of results that came up in Alexa and were kind of advertised were not on the first advertising is not necessarily a path towards creating visibility there within Alexa. Obviously, the downstream effects of advertising, which is you get more visibility, you get more sales, and that sales velocity now gets you up there, but then that overall will bring you both then over time on the first page, but also increase your chance of getting into the Alexa recommendations.
Paul Sonneveld
So right now, hard to buy your way into that. In fact, it doesn't help. Of course, we don't know what the Amazon advertising team is doing. I'm sure they're thinking about this, but that's what it is. That's what it looked like right now. So that begs the question, if you can't buy your way into it, what can you do? What actually influences whether the assistant picks up your products? I'm not sure whether you guys got into this in the study, but what are the signals that mattered most? How much is actually given by the actual listing and the structured content and all of those things versus things like reviews, availability, seller reputation, those secondary things? What did you learn there?
Christian Umbach
So some of the core learnings on that side, for one, all of the results that Alexa suggested had a rating of four, four point zero or above. They were all available. And at the same time, from kind of a velocity perspective, we could see that there was an overarching amount where like the listings would have kind of a callout of sold fifty last month, sold one hundred last month. For example, so some indication of velocity. It's obviously hard for us to tell whether you know those are you could say the the highest moving items like with within that category I yeah we just can't go go that far to I guess conclude that, But it's certainly listings where Amazon felt okay we're giving the shopper suggestions here where we have different proof points available you know the listing itself, they had decent quality in terms of bullet points, in terms of titles and so forth, but then also sort of sales velocity, review velocity, review quality, those different bits and pieces coming together.
So I think as a brand looking at what's the playbook today to achieve that, I do think it is kind of covering those different bases, which is to work around reviews. In the early days of a product lifecycle, doing Vine is a helpful step there in finding a way on how you do those and get reviews in at scale. And then I think the other part is more kind of on the data feed level or listing optimization level, making sure that you have the right signals in your product catalogue itself to help Amazon understand like, the persona that it understands as kind of the shopper of like, oh, you know, it's a college student back just kind of moving into their first dorm room right now. So that's kind of the shopping mode that they're in. Now, how does it match to the products that it's surfacing? And kind of do we see a reflection of that. That's at least what we're you know reading from parts of the data but I think you know from a statistical significance perspective there just realistically it gets a little bit you know softer.
Paul Sonneveld
Yeah, I understand. There's so many variables trying to work out what the prompt behind Amazon's AI system is in terms of how they evaluate the results and then present something back to you. That's an interesting exercise. But I'm sure, as you mentioned before, some really interesting glimpses of where hints and where things point to. Now, I know you've just done the study. If you were to repeat this over time, it'd be really interesting to see whether things move and shift, whether you have some time series data. But did you get into any of that? Did you do any tests around that in terms of the stability of the recommendations? For example, if you go back the next day or the week after, is Alexa still recommending the same products, or is it a completely different set? Is there an element of randomness to it, or does it change? Did you pick up any insights on that question?
Christian Umbach
We tested it just on a smaller scale. So basically, before we properly ran the two thousand queries, kind of testing it with a set of five to ten, like how does it change day by day? And we saw that we really ran the study in June 2026. So we did see good consistency there such that we said, OK, like for this initial jump that we're going to do, we're going to focus on a one-time run. But I would overall expect that as the entire space is developing and Alexa is going to continue developing and get fine-tuned, that we're going to see results evolve. And I think especially around the importance of maybe better understanding particular signals over time. And then we'll see what's going to happen on the advertising side.
I think that's kind of an area where we'll see new advertising products emerge as well that kind of give a more clear kind of playing field. Obviously, Amazon launched kind of sponsored product prompts, I believe, or the rollout was in March this year. And we're gonna start seeing more of that, I believe. We didn't come across too much in the study that we ran. But I think, you know, whether it's Alexa or AI search tools outside, like ChatGPT, Claude, Gemini, and so forth, like everyone's playing around with, okay, how do we create the best user experience? And to a certain degree, I think advertising can be part of a good user experience as well.
Paul Sonneveld
So I wanted to go back to I mean, obviously, just I've got some questions around the broader ChatGPT, Claude and GeminI and all those models. But before we get there, I just wanted to go a little bit deeper still on maybe some of the category differences, because you mentioned you picked a whole range of ASINs across different categories, you know, home, electrical, apparel and the like. In terms of you know, Alexa recommending a set of products versus the organic search results, you know, where do you see sort of a category or department-by-department view? Were there differences, you know, in terms of Alexa pushing recommendations, maybe that were outside the top ten? Anything that is worth comparing, contrasting, based on your findings in the study?
Christian Umbach
Yep. So we actually have a few numbers on that as well. The main takeaway for us was that if we looked at apparel versus kind of the or at least home and kitchen beauty and health and supplements like other was maybe a bit of a let's leave out other in this context but the line between like does rank protect versus not protect was least strong for apparel. So it was the most diversified results that we were getting, and while for all kinds of this thesis of thirty six percent of your of the Alexa results still came from the top ten organic for apparel, this was much lower. And maybe that's due to just the nature of what's called custom preferences in apparel. That's where I would be looking overall, I think, in terms of Amazon understanding preferences. We both like blue shirts, for example, and Amazon knows that we're buying blue shirts. It will impact the way that it is surfacing results. And that does not necessarily maybe follow the same pattern today on the traditional search results pages.
And just in apparel, I think it's probably the category with the highest level degree of what's called custom preferences, customisation. So we see that as a reflection of that. Which basically for brands playing in that category, means that you both have the highest opportunity to say, OK, here's actually the full range of products. And here's a number of different use cases, for example, that these products are good for. And we actually see that around apparel really nicely, for instance, in some of the work we do around seasonality, which is you have those seasons, like Halloween is going to be coming up pretty soon, where everything orange gets a sales spike around Halloween. So it's kind of have these weird nuances of very kind of minI tentpole events or micro-seasonal demand.
Paul Sonneveld
Yeah, I think, look, it does make sense on a category-by-category basis there, just because the nature of the category and the consumer choice, and maybe AI being able to play a stronger role in recommending things based on preferences, based on what other people have done there. That's certainly interesting. So sort of goes back to the kind of the top line question for me, which is if I'm a brand owner, say like I'm in apparel, right? And in apparel, it matters even more than, you know, it doesn't help in supplements. Although, you know, I can't read exactly. I think it says fifty three percent. That's still quite a bit.
But in apparel, like as a brand owner in apparel, and I'm watching this video. What are some practical things I can do really to try and boost my featuring in AI recommendations? I know you've already given us like some sort of you know correlations that you've observed, maybe not statistically significant at this time, we know that spending money on advertising. I know Amazon's done that, obviously, their prompt space, but I'm sure there's more coming. But in the meantime, what are some practical things that I can do to improve my chances and odds to show up?
Christian Umbach
So apparel specifically is actually one of the big winners, for example, of Amazon's latest title policy and introduction of item highlights because apparel, a lot of categories within apparel used to live under this restriction of you have a hundred and twenty-five character title, and then you go into the listing from there. Like now, just from a pure real estate perspective, yes, the title is getting shorter, but you get a dedicated field, you know, a hundred and twenty-five characters just for item highlights to basically show all the great things of the product being waterproof and following specific compliance measures, for example. So it's almost like an added marketing field there to call that out.
I think the other part, and with Autopilot Brand, we do a lot of work with apparel brands. For example, one of the core areas where like automation works very, very well is to sort of lift the overall standard of the product catalogue. And you can, you know, that kind of starts on the sort of on the foundation of the catalogue in terms of like, are, is there a proper reflection of all of the attributes and so forth? But then the interaction point for the shopper is if you look at a lot of apparel products on Amazon today, it's kind of barely like, three to five bullet points. But if there are five bullet points, it's usually kind of just made in the US, like cotton and just buzzwords. It's not really leveraging that space. And obviously, you sell a hundred products. It's actually ten thousand listings. So it's not that easy to wrangle as a catalogue. We see a very, very heavy impact and lift for conversion rates, and you know through that then growth traffic growth as well on actually getting hands on there and bringing this in.
Paul Sonneveld
Yeah that's, I mean, it makes so much sense, I mean, certainly, apparel has always been the challenge. So much range, such a wide curve or variance in sizes, seasonality, new drops, depending on the type of apparel you're in. It's always a much heavier lift than other categories. But I guess there are much better tools out there to help with this these days. And it certainly seems to pay off. At least your stats seem to suggest that. So that's really interesting. Let's sort of go a little bit wider now, beyond Amazon, you know, search results in models like ChatGPT, Claude, Gemini, and obviously search results may be pointing back to your Shopify site or to even to the ASIN listing or the PDP. You know, did you form a view as part of your study on how things work in relation to those, you know, those search engines, let's call them that. It's probably not the correct term. You know, any differences, similarities, what observations can you share with us?
Christian Umbach
Yep. So one thing that we came across, which I think got published in the course of June as well, while we were just kind of wrapping up our own study and, you know, we found it really interesting from an insights point of view. I think this was a study done by Peak AI here published in Search Engine Land. So in their case, they basically looked at ChatGPT results and were trying to understand where is ChatGPT sourcing from.
And they could see that forty five percent from an exact match perspective of the carousel products that ChatGPT would recommend were directly coming out of the Google Shopping top forty. Which for us was kind of an interesting finding because we've just seen that like on the thirty six percent of the I guess that's called Alexa products carousel Alexa recommendations came from the organic top ten. And you know obviously, if you go further down the page like that number increases.
But on the ChatGPT side you have this strong tie-in into the Google Shopping kind of ecosystem today at least from from a data perspective. So for us, that was, I guess, an initial finding on the same level, you could say, as we were looking at with our study. But then, obviously, the much bigger question outside of that is like, okay, well, it's nice to see the numbers, but how do we actually play there? Like, what's the playbook to be visible there? And that's an area that we've been increasingly passionate about because if we look at what's the strength of brands that are doing well on Amazon is, you actually have a lot of data that, whether you like it or not, Amazon requires you to provide to Amazon and to fill in all of your different catalogue attributes. Amazon's been going through these waves of ensuring that those are of higher and higher quality. Because for Amazon itself, it is this, you could say, low-cost feed of information to understand the products better.
Now, ChatGPT, Claude, Grok, and others, they don't have this luxury. They don't have a, at least at scale, low-cost data feed, basically, into them today. So we started looking into, okay, like what's, how can we actually take some of the learnings? And I think this is most critical to brands that very have a strong Amazon footprint overall, because you're basically like Amazon is walling you from the search world out there. I guess, you know, ChatGPT will go to Google, and Google has some of your Amazon product data as well, kind of, you know, as a bridge, but it's still very, very limited. It's cash, it's not, you know, it's not complete by all means. So we started taking that understanding, you know, right now that's running with at or so where we have took a lot of the context that, you know, the brand also had through their Shopify side, through other data sources. And we start creating agent sites specifically, they're very much linked to their Shopify site if they have one, for instance.
Really with the goal of saying like okay like we we're expecting not only human readers to come to the shopify side but bots are playing an important role and let's make this data available as far as we we can as far as kind of it makes sense for for the brand in terms of saying like you know here are the products here's what they're good at, here are some of the unique selling points around them here are my sort of my item highlights you could say and then signaling that basically for for bots this information and then the part that's easiest to say, okay, like a bot came to the site, hey, great, meta scraping it, clause scraping it, they're all, you know, gather the information and they're coming back the next day or the next week to see if there's anything new.
But then we actually started seeing like, oh, this is showing up in citations that are measured through AI visibility tools. In Google Analytics, you can now see kind of AI assistant traffic as well. So we see for the brands that are running, the AI assistant traffic is getting up. There's add to cart reported around that traffic. I think it's still very early. It's kind of the wild west, I think, in many ways of what's called search and discovery around the agent platforms.
But I do think that the foundation of good data feeds, good comprehensive understanding of your customer base, their use cases, is something that Amazon has very very much trained us as on and that you know becomes a real asset now in a much broader world of search where you know it's from the example of an Amazon focus brand you open up the data out there Obviously, to with the goal of bringing more traffic than Amazon because that's where you want to sell, that's where maybe you've built a brand so far and you don't necessarily need to change that. But the, I think it's opening kind of a much broader visibility spectrum, but beyond kind of the playbook of saying like, oh, you've been strong on Amazon, now you've got to go to TikTok. I think that part is already much more established in the sort of CMO's mind today.
Paul Sonneveld
Yeah, that's really interesting. To me, it's like maybe content optimisation isn't dying yet, right? It's almost like now you've got to think about, what is the content for the actual consumer, a person, and how do you optimise a PDP or maybe a separate page or whatever platform that lives on for AI bot discovery?
Christian Umbach
Yeah. Exactly.
Paul Sonneveld
And I think that's a really interesting, and that's kind of a new lever, right? Super interesting. Now, we are well over time, but we did have one of our audience members ask a question. I want to do the right thing and just answer or at least ask the question. I'll leave the answering to you before we wrap up. So the question comes from CA ECOM. And the question is, can you estimate the degree of impact that Cosmo has on traditional search in 2026? Are search results being like reordered based on customer location, time of day? Any insights into that question? Let me make sure I've got the right page here. Here we go.
Christian Umbach
Yep. I guess to dissect it a little bit, obviously, we're halfway through 2026 and we're obviously all itching to understand what's what Amazon launches next and what's going to happen next. I think overall, and taking because of pure Amazon.com perspective here, I think Alexa is going to continue to take more real estate, and Amazon will nudge more users into kind of another search versus a traditional search in that regard. So the share of traditional search is going to decrease over time.
And I think we already see that if you go to a search bar today, once in a while, it actually directs you directly to an Alexa for shopping search. It doesn't even allow you to kind of perform traditional search, and that's way out. And then I think a couple of foundational items that have held true under traditional search will continue to hold true which is you know amazon knows that they are creating a happy customer. If they're allowing you to buy a product that is good in terms of well-reviewed, well, good ratings as well. And speed obviously matters for satisfaction as well. I think its ability to understand, okay, how fast can we get a product to the person that is looking to shop here is certainly one. And they want to avoid that. You just saw something which may be a great product, but it needs to shift over from the other coast, and maybe comes at the risk of someone going to Walmart and just getting it within twenty-four hours at Walmart.
So I think some of those metrics will hold true. But overall, from a search discovery perspective, a lot, a lot more will go towards Alexa and certainly towards the models outside of Amazon as well, where I think OpenAI had the stat that about twenty percent of all ChatGPT queries are already have some level of shopping intent. And I think last year there was a study that like, two point one percent were specifically product searches, uh, which at the time were like, I guess, fifty million a day. And I think what today would be like, well, well over a hundred billion product searches.
Paul Sonneveld
Yeah. Thank you for that answer. Much appreciated. We are well over time, so we're going to have to wrap up. Christian, thank you so much for joining me today. That was a really phenomenal session. Such new ground here. I really suspect that more than a few people just added, oh, we have to audit our presence on this third shelf, our catalogue quality, all of those attributes. I'm sure it's been a bit of a wake-up call for many. So thank you for conducting the study and giving and sharing with us those insights that just lift the lid on this space that obviously is rapidly evolving. Now, I was just going to say, if people, I know you're about to publish the full results of the study. If people are interested in getting their hands on that, what's the best way to do that?
Christian Umbach
Yep. We'll have to publish it through the order. We do a weekly newsletter, so we'll make sure that folks get their hand on that. Otherwise, it's theoneorder.substack.com. But we'll also link it to the webinar here. Yeah, we'll put it in the notes and make sure you've got all those links and get your hands on those study results.
Paul Sonneveld
Okay, Christian, thank you so much. It's been great.
Christian Umbach
Thank you so much, Paul, and have a wonderful day, everyone.
Paul Sonneveld
Thank you. OK, everyone, before we wrap, a couple of quick reminders. This session was recorded, and registrants will get the replay landing in their inbox in the next twenty-four hours. So feel free to forward it to whoever owns your Amazon channel within your business. As Christian mentioned, if you want to hear more from him or the study, we'll make sure that links are in the show notes so you can access those. And of course, if you're getting value out of Marketplace Masters, the best thing you can do for us is to either hit follow or subscribe on YouTube and forward this episode to another brand owner and share some of the content that we're producing for you. I'm Paul Sonneveld. I'll be back next week with another live conversation. Until then, keep building and take care.
Here's the link to the full study >> full study plus a 700-page Alexa results brand library