Large language models (LLMs) are changing search and information retrieval. It’s critical to understand this shift to generative information retrieval.
There’s no one else I trust more to learn about this paradigm shift than Dawn Anderson, Managing Director of Bertey. Some of the topics covered in our interview:
Evolving SEO beyond traffic and rankings.
Understanding the biggest change in information retrieval – ever.
What SEOs should learn now.
A preview of her upcoming presentation at SMX Advanced 2025 in Boston.
Reminder: SMX Advanced returns June 11-13 in Boston. Get your tickets now!
This transcript has been edited for length and clarity.
Danny Goodwin: Hey everybody, this is Danny Goodwin, editorial director of Search Engine Land, and I am joined today by Dawn Anderson. Dawn, looking forward to seeing you speak at SMX Advanced in Boston. Please introduce yourself to the class. Who are you and what do you do?
Dawn Anderson: Yeah, hi. Thank you for the invitation. Really looking forward to being in Boston, too. I’m Dawn Anderson. I am based in Manchester in the UK. I run a very boutique agency consultancy called Bertey, named after my dog. And I’ve been in SEO for nearly 18 years now. So I’m still enjoying it really every day.
Danny Goodwin: And we’re looking forward to your session at SMX, which will be on June 12, which is a Thursday. We’ll get into that in just a little bit, but since I have you and as you mentioned, you’ve been around for quite a while in the industry like myself, but you look way better than I do from being around this industry for so long. I wanted to start by asking you – a lot of change has happened in the last year or two. Based on what you’re seeing as an industry, what do you think that a lot of us are doing right and maybe what are some things something that we’re doing wrong right now with all this, breakneck AI change.
Dawn Anderson: I think those people that have been focusing on journeys and understanding their users For years and years and years, the whole of the digital marketing broader strategies has had an approach that’s been fairly omnichannel. Sort of be everywhere be a bit omnipresent, get the halo effect. But even though it’s been said in the SEO spaces, I think we’ve all still very much focused on what we consider to be the lion’s share has been just go for rankings just go for traffic from organic and don’t think about how it all touches up with everything else all connects with all these other channels.
Dawn Anderson: So I think those that are doing well and will do well and continue to do so will be those that really see organic as a single touch point that must be tied with all of these other channels very very well. And understands user journeys, understands the whole research cycle, which is getting longer and longer and longer. Obviously, now we’ve also got all these different chatbots in the mix as well. The AI search elements there that’s making people almost even more inquisitive because people are intuitively going to learn about things and research. Everybody’s a researcher nowadays. So I think those that are focusing on journeys are doing the right thing.
Danny Goodwin: Your session ties in with my next question I wanted to ask. Your session title from search ranking to answer synthesis. for the longest time, organic has been all about driving traffic and getting rankings. That’s changing and it’s throwing people a little bit for a loop. They’re seeing click-through rates traffic go down in the last year or so. At this point and heading forward, how can SEO still provide value to our clients and stakeholders?
Dawn Anderson: So I have thought about this quite a bit because I must admit when I read the intro to Mark Najork’ss keynote 2023 SIGIR, which is the Special Interest Group for Information Retrieval, one of the big information retrieval conferences. The intro said Mark wants to give people answers rather than send people to resources.
Dawn Anderson: And I thought, this is one of the biggest research engineers at Google on search.” That really worried me. And then I read the paper about delphic cost by Andrei Broder and he was talking about how we could search is really difficult people when they start having to go through 10 blue links and go everywhere and do their own researching and let’s bring the answer to them. I was worried. But then I started to think about the fact that … as creators and publishers we are in a commercial contract. We provide the content. We optimize. We do a sterling job as SEOs to actually help them improve their productiveness from the web if nobody’s going to go and tell those devs to remove that robots noindex blocked robots from the homepage
Dawn Anderson: So that there is an un unspoken agreement between us that we get traffic and we’ve seen in a number of articles out there that, as creators, we are seen as a necessary evil. So I’m not worried. I think that we’ll come to a compromise where they realize that They have to send traffic. They had to make it worth everybody’s while. Otherwise, what will happen is people will just stop creating good stuff.
Dawn Anderson: Publishers will start blocking Google or any other bots that come crawling in scraping and eventually they’ll just end up with a downward spiral of AI generated content. … You end up with this situation called catastrophic forgetting. So it’ll all go just to nothing to a really bad cess bit of content. So I think ultimately we’re in an agreement that’s unspoken and they’ll sort it out to some extent.
Dawn Anderson: We will put pressure on them as an industry. Not just us. The IR field will put pressure on Google as another field because they’re not happy with the way that things are going either. Generally speaking, there’s a really interesting paper by Emily Bender and [Chirag] Shah that says that they’re really concerned that Google are rushing ahead into all of this. So I think there’s going to be pressure and what’ll happen is we’ll find a nice balance where it kind of works itself out. We’ve seen it before with links. We’ve seen it with Penguin where, they had all those. We adapt. SEOs adapt and then they adapt. So it’s a partnership that has to that will prevail, I believe.
Danny Goodwin: Absolutely. We hope so for sure. Obviously there’s a lot of change going on. Change is not always a bad thing. So is there anything in SEO that you are really excited about? Is there anything going on that just kind of gets you really excited about the future of search and SEO and information retrieval.
Dawn Anderson: At the moment I’m finding search really exciting again. I’m keen to see how it evolves over this next two, three years. How far they do go with the AI modes and so forth. We’ve got all these surfaces like the universal aspects of search. I think they may expand out further.
Dawn Anderson: There was an interview with Sundar Pichai who was talking about how they are really keen to send traffic to the open web. They agonize over sending traffic to the open web in many different types of ways. I think the ways in which we will get traffic are going to expand, which leaves many opportunities for us to get really creative. Ten blue links really is well gone in my mind. There’s just so many different ways – imagery, video, lots of different answers that you can be creating. Not to mention things like a merchant center and the local business profiles. So, just many aspects that is very untapped at this moment in time. And for me it’s about just getting a rounded strategy.
Danny Goodwin: Yeah. Absolutely. Yeah. And I saw an interesting stat today. I believe it said if you rank number one, you have a 25% chance of being shown. And then I don’t know if it was AI Overview or AI Mode. And I was just like, “Wow, I remember the days when you had a 100% chance of ranking number one when you ranked number one.”
Dawn Anderson: And interestingly enough, I was looking at some rank tracking is a funny thing for me. It’s always been a bit just a finger in the air. And I think it is increasingly so because with all the different aspects of ranking now that go on in the late stages of ranking that take into consideration all the context and localization, personalization and so forth.
Dawn Anderson: There’s rank trackers are always wrong, but they are just a very good guide for me. It’s more like a don’t hang your hat on it, but just see it as a general direction. Are you in the top K? And if you’re in the top 20, whatever, you’ve been fetched as highly relevant and then things will get shuffled around anyway with the machine learning late-stage ranking. So, I was looking at something today and it saw it was in Semrush, funnily enough, and I saw three sites all in position one for the same query and…
Dawn Anderson: I was like, that’s interesting.” So, obviously some sort of AI Overview that they’ve all just hit number one because I’ve never seen that before. So, it was either a data anomaly or it was a link or something that had been detected on an AI Overview of sorts and I think it was probably that. Yeah.
Dawn Anderson: I mean obviously we’ve got this situation in that they have said that we will get the data but it will all be longterm. So that’s a big relief. That’s a big relief because if they were saying we’re not going to show you then that would be really worrying over time.
Danny Goodwin: Yeah. It’s kind of interesting too that a lot of people have seen their impressions going up and they’re like why is this happening? It’s like because you’re in an AI overview and that’s all one block. So if you’re in an AI overview you rank number one and you’re going to get more impressions because everyone sees this.
Dawn Anderson: Yeah. when I look at some projects, I’m seeing impressions that are absolutely huge as they’ve been in the historical data I have access to at this moment, 16 months. I’m like, but the click-through rate is going down.
Dawn Anderson: But then I think they’re going to improve on it. I think this is just a short-term … I mean, it’s probably very expensive for some brands. This moment in time, but I think what might happen is on the back end there’ll be all sorts of this convergence of a model. So it fetches generative but on the front end I think it maybe begin to look like we’ve seen it look over the years so the front end is the same, but the model is one converging model yeah in the background and I think if I remember rightly that Glenn or Lily or one of the others on Twitter shared that they’d seen in an AI Mode test what looked like just 10 blue links but it was AI Mode and I think I’m wondering whether they’re going to eventually maybe go back to that comfortable place that people as searchers understand, but on the back end of it, there’s all this other generative stuff going on.
Dawn Anderson: So, I think it’s just let’s see what happens. I’m not worried about my job, let’s put it that way, this moment in time. In fact, I’m more excited because I think we’ll have to get more creative and that is a great place to be.
Danny Goodwin: Another good transition. You just mentioned that you’re not worried about your job. I know some people are, but I’m curious, is there anything that has you worried now with all these changes going on in SEO?
Dawn Anderson: Yeah, I’m a bit worried about some of the I think we’re in a very RankBrainy time again in where everybody’s talking about things that it’s a little bit like just repeating things that are not necessarily true. It’s kind of an exciting time, but it’s going to be a time where there’s going to be quite a lot of misinformation flying around. I think there’ll be a lot of blog posts that are not right. None of these are easy concepts for us to understand or for anybody to understand. So, I think we need to get our learning hats on again. Just knuckle down and get to grips with what all the different concepts mean. I’m always advocating for learning about information retrieval because it opened my eyes to understanding search a lot better than I had done previously when I stumbled upon it all those years ago. And this is a part of it.
Dawn Anderson: So, I think people should really just knuckle down and just learn and maybe go on LinkedIn less and blog less until you learn about it. That’s all. And especially if you’re going to go on to ChatGPT and say, “Please create post for me for LinkedIn about AI Overviews,” which I think there may be a bit of that going on.
Danny Goodwin: I think you’re right. We’ve talked about a lot here. based on everything you’re seeing right now, do you feel like SEO is heading toward growth decline or maybe some reinvention over the next, couple years here.
Dawn Anderson: I think we’re always reinventing ourselves. I mean, if you remember, the big changes. You’ve been through more changes in SEO than I have because you’ve been around longer than me. But we reinvent ourselves every day in SEO. I think this is probably the biggest reinvention we’ll have to go through. I mean, this is the biggest paradigm shift in the information retrieval world ever.
Dawn Anderson: So it’s going from index, retrieve, rank to just generate a model, that’s going to take over time. So it’s going to be different. So I think we will reinvent ourselves. We will probably become much more aligned with trying to understand the way that agents are doing things on behalf of humans. But we must remember that the agents are trying to emulate human behavior.
Dawn Anderson: So they’re still trying to be like humans. So, we must still try and understand the humans. Like search engines have only really ever tried to emulate what human information needs are. Let’s not do things for search engines. Let’s do things for humans. And it’s the same with agents. Agents are just cutting out another layer and we just have to understand the human that’s behind those search needs.
Danny Goodwin: Absolutely. At SMX Advanced, you will be there. You’re coming to Boston. And I’m not sure if you have the longest flight of anybody speaking.
Dawn Anderson: I think probably tying with James Brockbank because he’s coming from the same airport. So think he has a lot longer to get to the airport than I do. So I’ll let him take that win.
Danny Goodwin: There you go. but yeah, thank you for coming to Boston for that in just a couple weeks. and your session is: From search ranking to answer synthesis, how to weather the AI storm and totally ties into everything we’ve already talked about. But someone’s watching this thinking about coming or they’re definitely coming and they want to see you speak, can you give us maybe a preview of what you’ll talk about?
Dawn Anderson: I’m going to talk about kind of how we’ve got here, the bit of a storm that’s been brewing in the information retrieval world. The fact that this is the biggest paradigm shift ever in all search that we’ve ever known. Cannot really underestimate the size of that. I’m gonna give away if you like six steps to consider don’t be realistic let’s not be chicken lickings thinking that the sky is falling in think for the future understand your user a few insights into what might happen next really. So yeah hopefully it’ll be useful
Danny Goodwin: I think it will be. I’ve seen you speak before and your stuff is always notch head of the class type of stuff. Always learn so much from you and I’m looking forward to this session as well.
Dawn Anderson: Thank you.
Danny Goodwin: That will be on June 12. SMX Advanced is June 11-13 live in Boston. making our grand return to in-person after five years of going online after COVID. And yeah we’re super excited about it and Dawn, super looking forward to moderating your session as well and just generally catching up in the real world. So thank you again for taking the time here.
Dawn Anderson: No problem. Thank you for the invitation, Danny.
Danny Goodwin: All right, thanks, Dawn. Thanks everybody.