Diary of an SEO: The Biggest Shift in Search We’ve Ever Seen

If there’s one universal truth in SEO, it’s that everything eventually changes. I’ve been in this industry long enough to weather countless shifts that forced us to rethink our strategies. Every time Google rolled out a major update – from Panda and Penguin (which forced us to clean up low-quality content and shady links) to Hummingbird and RankBrain (which pushed us to focus on user intent over exact keywords) – the same cycle played out. People panicked, proclaimed “SEO is dead,” and scrambled to adapt. But each time, SEO didn’t die; it evolved. Those who learned quickly and adjusted not only survived, they often thrived.

This time, though, feels different. It’s not just another algorithm tweak or a new ranking factor. It’s a fundamental change in how people search for information. For more than two decades, the search model stayed familiar: users typed a query, Google served a list of websites, and people clicked through to those sites. Now that model itself is being upended. AI-powered search is starting to pull answers directly into Google’s results, meaning users get what they need without ever clicking a website. AI assistants can handle tasks like research, product comparisons, and even purchases end-to-end, never once sending the user to a traditional search result.

For the first time, the question isn’t “How do I rank higher on Google?” – it’s “Will traditional search even matter five years from now?” If AI agents become the primary way people find and interact with information, then SEO won’t be about optimising for Google’s algorithm anymore. It will be about optimising for how AI selects and presents content. In other words, we’ll have to ensure our brands and content are visible to an AI-driven gatekeeper. That’s an entirely new kind of challenge, and it’s what makes this shift the biggest I’ve ever seen in my career.

For businesses, this change is unsettling. Many have relied on Google traffic as the lifeblood of their digital strategy, and if users stop clicking through to websites, traditional metrics like organic traffic and click-through rates could crater. As SEO professionals, we have to redefine our roles. It’s no longer just about getting to page one of search results; it’s about staying visible when “search” as we know it is fundamentally changing. That means understanding how AI-driven search processes information, learning to optimise content for AI outputs, and finding new ways to reach audiences beyond the old Google-centric tactics.

I’m embracing this challenge head-on. In this piece, I’ll share my journey through past evolutions of SEO, the hard lessons those experiences taught me, and how I’m applying them to navigate the AI-driven future of search. It’s a personal story of adaptation – and a roadmap for anyone who wants to stay ahead in SEO’s next chapter.

The Early Days: Learning SEO Through Trial and Error

When I started in SEO, there were no courses or degrees in “search engine optimisation.” It was something you fell into by accident. I learned by rolling up my sleeves and tinkering – making mistakes, breaking things, fixing them, and slowly figuring out what worked. Back then, Google was far from the AI-driven powerhouse it is today. The algorithm was relatively easy to manipulate, and rankings were often won by those who discovered little hacks and loopholes before Google caught on. In many ways, it felt like the Wild West of digital marketing.

I vividly remember devouring Matt Cutts’ videos like they were gospel. Matt was the closest thing we had to an inside source at Google, dropping hints about what to do (and not do). Some SEOs bristled at his guidance, but many of us hung on every word, trying to read between the lines. At the time, SEO was both simple and chaotic. You could buy a bunch of backlinks and watch your site rocket up the rankings. You could cram keywords onto a page – even at the expense of readability – and still rank well. We built entire microsites full of thin content, played tricks like exact-match domains, spun articles into barely comprehensible copies, and for a while, those tactics worked.

It felt like a game, and if you knew the cheat codes, you could beat the system. But that era was short-lived. I soon learned that chasing quick wins wasn’t a sustainable strategy. Google grew smarter with each update, shutting down the loopholes and forcing SEOs to grow up, too. My early years taught me that while tactics might win you a battle, only solid strategy wins the war. And sometimes, the most memorable lessons come from the battles I lost.

Lessons From Failure: SEO Mistakes That Shaped My Approach

SEO is an industry built on trial and error, and error often makes the better teacher. I’ve made my share of mistakes over the years – some small, some costly, all educational. Two in particular still stick with me, because they fundamentally changed how I approach my work.

The first was a website migration gone wrong. On paper, this site move had everything covered: every URL redirect mapped perfectly, internal links updated, no obvious technical issues. I expected a seamless transition. Instead, within days of the relaunch, organic traffic plummeted. Conversions dried up, and rankings tanked across the board. I was baffled. Every technical checklist item was ticked, yet something was clearly off. After countless hours combing through data and double-checking the implementation, it finally hit me: the problem wasn’t technical at all – it was human. The new site looked and felt completely different to users. The navigation changed; the content was arranged in unfamiliar ways. In ironing out technical kinks, we’d ignored how real people were using the site. Loyal visitors got confused and frustrated, and Google’s ranking drop reflected that disruption in user experience. This failure drove home a crucial point: SEO isn’t just about appeasing algorithms; it’s about serving users. A technically perfect site is worthless if users can’t find what they need. Ever since, I approach big changes with two questions in mind: “Can Google handle this?” and “Can our users handle this?” – and I refuse to sacrifice one for the other.

Another hard lesson came from a client who was hooked on short-term wins. They had enjoyed sky-high rankings and were raking in revenue, but their strategy was a house of cards: low-quality, keyword-stuffed content; aggressive paid link schemes; dozens of thin pages created solely to rank. I warned them repeatedly that they were on borrowed time – that this approach would crumble when (not if) Google caught on. Their response: “It’s working now, so why change it?” And for a while, they kept climbing the ranks, convinced I was overly cautious. Then, as predictably as sunrise, a core Google update rolled out and their traffic fell off a cliff. Years of “success” were erased in a week. That phone call, where the client finally acknowledged, “We should have listened,” was painful for both of us. It reinforced something I now tell every business: quick wins can become quick losses. If you’re doing something in SEO that you’d be afraid to tell Google about, you’re living on borrowed time. That client’s downfall cemented my commitment to sustainable tactics – focusing on quality content, user value, and longevity over any short-lived tricks.

Each failure, as gut-wrenching as it was in the moment, became part of my education. They taught me to look beyond the immediate goal of “more traffic now” and focus on the bigger picture – keeping that traffic next month, next year, and beyond. In an industry as volatile as ours, avoiding disaster can be just as important as chasing success.

The End of the Click: How AI Is Upending Search as We Know It

Through all the Google updates and strategy pivots, one thing has remained constant for over twenty years: people search for something, see a list of results, and click through to websites. That click—delivering a visitor to a site—has always been the endgame of SEO. Now, AI is changing the rules of that game entirely.

Today, AI-powered search results and assistants are doing something unprecedented – they’re giving people what they want without requiring a click. Google’s search results page is no longer just ten blue links; it’s increasingly a curated answer engine. Ask a question, and you might get a comprehensive AI-generated summary at the top of the page. Need a product recommendation? An AI tool might compare options and suggest one, saving you the trouble of visiting multiple websites. Have a simple question? Your smart assistant on your phone or smart speaker can answer it directly, often pulling from information it deems trustworthy – and you never see the source site behind that answer.

For users, this is a convenience boon. For SEOs and content creators, it’s a paradigm shift – and a bit of a nightmare. We’ve entered the era of zero-click search, where the traditional funnel of user -> Google -> your website is bypassed. I’ve watched analytics accounts where organic traffic drops by double-digit percentages, not because rankings fell, but because users stopped clicking results at all. Google and other AI-driven platforms are becoming the middleman that delivers answers and keeps the engagement to themselves.

This raises a sobering question: if nobody needs to click on our carefully optimised pages to get information, where does that leave us? If an AI assistant can gather data from countless sources and present the “best” answer without a user ever leaving the chat interface, how do brands get in front of their audience? We’re looking at a future where ranking first might not matter if the AI summarising results doesn’t give you credit or send you traffic. In essence, AI is becoming the new gatekeeper. If your content isn’t the one an AI chooses to quote or if your brand isn’t recognised as a trusted source, you might as well be invisible.

This is, without question, the biggest shift I’ve seen in my SEO career. It’s not a tweak or a new feature – it’s a whole new playing field. But as with every disruption, there will be those who adapt and those who lag behind. The way forward is to stop thinking purely in terms of “How do I get more clicks from Google?” and start asking “How do I make sure my brand and content are chosen by the AI?” That means understanding how these AI systems decide what information to present, and ensuring our content is optimised for that new kind of visibility. SEO isn’t dead by any means, but it’s evolving from Search Engine Optimization to Search Experience Optimization. It’s about staying visible and relevant when the click itself is no longer guaranteed.

Future-Proof Skills: Communication Over Tactics

For most of my career, being a great SEO meant being deeply technical. If you could audit a site blindfolded, reverse-engineer Google’s algorithm shifts, tweak meta tags and structure sites with the best of them, you were golden. I prided myself on those technical skills – and they’re still incredibly important. But the reality is, the ground beneath our feet is shifting. The value of pure technical know-how is being matched (and perhaps overtaken) by something less tangible: the ability to communicate and lead.

Why? Because as search becomes more complex – with AI, machine learning, and automation in the mix – simply executing tactics isn’t enough. Businesses are more sceptical and confused than ever about SEO. They hear that “SEO is dying” or that “content is all you need” or a hundred other hot takes. They see their traffic charts rising and falling unpredictably. In this environment, the SEO who thrives isn’t just the one who knows what to do, it’s the one who can explain what to do and why.

I’ve seen this firsthand. Not long ago, I started noticing that some of the most respected voices in our industry weren’t necessarily those with the deepest technical pedigree, but those who could translate the chaos of SEO into clarity for others. Take Mark Williams-Cook – a well-known name in our space. It’s not just Mark’s technical acumen that makes him stand out (and he has plenty); it’s his ability to break down complex search concepts into plain language and relatable insights. He makes technical SEO feel accessible rather than intimidating. People like him can articulate why a technical fix matters for the business bottom line, or how an algorithm update actually affects a real-world website, in terms anyone can grasp. That skill – marrying expertise with clear communication – is pure gold.

Realising this changed the trajectory of my own career. I spent years happy to work behind the scenes – running audits, tweaking code, fixing issues, and quietly driving results. I’d write detailed reports and feel my job was done. But I noticed those reports often went unread by busy executives, or the insights got lost in translation. To truly lead in this next era, I knew I had to get out of my comfort zone and become a better communicator. That’s why I’ve started calling myself a “technical marketer” rather than just an SEO. My job isn’t only to make websites rank – it’s to help businesses understand how to be visible in a world where search is changing daily, and to trust my guidance in navigating that change.

This shift in skillset also means putting myself out there more – sharing insights on video, speaking at conferences, hosting webinars, and engaging in public conversations about where SEO is headed. I won’t lie: trading the anonymity of my laptop screen for the glare of a camera or a roomful of peers was daunting at first. Hitting “record” or standing at a podium makes you vulnerable in a way writing an audit never does. There’s no backspace key when you’re live on a webinar. But this is exactly why it’s powerful. In an industry where so many prefer to stay in the shadows, those willing to step forward and speak with clarity and conviction naturally stand out.

The more I’ve leaned into communication, the more I’ve seen its impact. I’ve had business owners tell me that a short talk or a clear analogy helped them “get” SEO for the first time. I’ve seen how sharing knowledge openly (successes and failures) builds trust and a following. In an AI-driven future, we need champions who can demystify the technology and keep SEO relevant in the eyes of clients and the public. The takeaway for me is clear: the best SEO in the room isn’t the one who knows the most, it’s the one who can help everyone else understand what the heck the wonks are talking about. As search evolves, leadership and communication are becoming just as critical as technical chops.

The Power of Community and Collaboration

One thing I’ve learned over the years is that SEO is not a solo sport. In the early days I haunted forums and comment sections, piecing together insights from anyone willing to share their latest experiment or theory. Back then, information was scattered; you had a few blogs here, a few discussion threads there, and a whole lot of trial-and-error on your own. Today, the landscape is different – we have more SEO blogs, podcasts, Slack groups, and conferences than ever – but the signal-to-noise ratio can be just as challenging. There’s an overload of advice, much of it conflicting or outdated. That’s why I’ve become very intentional about the communities I engage with.

To stay ahead in an industry that changes weekly, you need to surround yourself with smart, forward-thinking people. I co-founded a community called TechSEO North with Nikki Halliwell for exactly this reason. When we host discussions at TechSEO North, we skip the surface-level stuff and ask the hard questions: What’s really working right now? What assumptions are we all making that might no longer be true? Where are the next big search disruptions going to come from? These are the conversations that spark innovation and help us anticipate change rather than just react to it.

Being part of a community has tangible benefits. First, you get early insights – often the best ideas or warnings about what’s changing don’t come from Google or Moz blog posts, but from a peer casually mentioning an odd trend they spotted or a test they ran. Second, you develop a habit of critical thinking. When you’re in a room (physical or virtual) with people who challenge your views, you’re forced to refine your reasoning or sometimes completely rethink it. That echo chamber that so many SEOs fall into – reading the same blogs and following the same Twitter (X) personalities – can be dangerous. It’s easy to start accepting industry consensus as truth without questioning it. A good community won’t let you get too comfortable; someone will always say, “Are we sure about that?” or “I tried that and got a different result.” It keeps you honest and sharp.

Finally, a community is a safety net. When things go wrong (and in SEO, things will go wrong), it’s invaluable to have trusted folks to bounce ideas off of. I remember a late night after a brutal algorithm update, trading theories in a Slack channel with a few SEO friends about why certain sites got hit. We pieced together patterns that none of us could have seen alone. That kind of collective problem-solving is the secret weapon many outsiders don’t see.

So if you’re an SEO trying to navigate the rapid shifts in our field, my advice is: find your people. Whether it’s a local meetup, an online mastermind group, or even a tight-knit X circle, invest in those relationships. Share your knowledge freely and absorb what others share. It’s no exaggeration to say that the right community can level up your career. In a field moving as fast as ours, none of us can afford to be lone wolves anymore.

Pushing Boundaries: New Projects for a New Era

Staying ahead of the curve in SEO isn’t just about talking – it’s about doing. As I’ve been preaching the gospel of adapting to AI-driven search and elevating our communication, I know I have to walk the walk. That’s why I’ve thrown myself into a couple of new projects that push me beyond my comfort zone and tackle the challenges our industry is facing head-on.

The first is an upcoming speaking gig at BrightonSEO, one of the world’s premier search marketing conferences. I’ll be talking about something pretty cutting-edge: using Cloudflare Workers for SEO. For the uninitiated, Cloudflare Workers allow you to run code at the edge (basically, on Cloudflare’s servers around the world) to modify web content on the fly. Why does this matter for SEO? Imagine being able to implement fixes or optimisations across a site without waiting on a developer or touching the CMS. With Workers, you can intercept a page as it’s served and tweak its content, headers, redirects – you name it – in real time. This means an SEO can, for example, roll out a meta tag change or a structured data snippet across thousands of pages instantly, or improve site performance by caching and modifying content at the network edge. It’s powerful stuff that most SEOs haven’t tapped into yet. I’m excited about this topic because it represents the kind of technical agility that will define the future of SEO. Even as AI changes how search engines display results, the underlying importance of a fast, accessible, well-structured website isn’t going away. Cloudflare Workers are a way to supercharge technical SEO in an era where we need to deliver results faster and at greater scale than ever. By sharing this at BrightonSEO, I’m aiming to spark more SEOs to think like developers and problem-solvers, not just content optimisers.

The second project is something I’m calling SEO Disasterpieces – a bit of a passion project born from seeing the same mistakes and crises play out across companies, again and again. The name (yes, inspired by Slipknot’s Disasterpieces) hints at the tone: we’re going to dive into the messy, chaotic failures in SEO that people usually prefer not to talk about. I’m assembling a series of case studies and stories on the biggest faceplants – algorithm updates that wiped out businesses overnight, migrations that tanked a site’s visibility, growth hacks that backfired spectacularly. And importantly, we’ll dig into what can be learned from each. The SEO industry has a bad habit of showcasing wins and sweeping losses under the rug. Everyone loves a good success story – “Look how I grew organic traffic 300% in six months!” – but those stories often gloss over how risky or rarefied that success might be. Meanwhile, thousands of sites suffer penalisations or drops, and we collectively don’t talk about it much, which means we don’t learn from it.

By bringing failures to the forefront, I want to inject some honesty and humility into the conversation. Every SEO consultant or agency has had campaigns that didn’t go as planned (and if they say they haven’t, they’re lying). Every business investing in SEO should know not just how to win, but how to fail gracefully – how to spot the warning signs of trouble, how to avoid common pitfalls, and how to recover when things go south. SEO Disasterpieces is my way of saying, “It’s OK, we’ve all been there – now let’s talk about it and get smarter.” I suspect these stories of crash-and-burn will be as educational (maybe more so) than the glossy success case studies, because they come with hard-earned lessons about resilience and wisdom in strategy.

Why pursue these projects now? Because the industry is at an inflection point. AI, automation, and the collapse of old-school tactics are forcing us all to rethink what we’re doing. I don’t want to be the guy who’s reacting to changes after they happen; I want to be out in front helping shape the conversation. By speaking on new technical solutions like Cloudflare Workers, I’m advocating for the kind of forward-thinking, technical adaptability we’re all going to need. By launching a project around failures, I’m challenging us to have a more open, growth-oriented mindset – one that will be crucial as we navigate uncharted territory. In short, I’m not content to just ride the waves of change; I intend to help steer the ship. These projects are stepping stones toward that goal.

SEO Isn’t Dying – It’s Just Changing (So What’s Next?)

Every few years, like clockwork, someone claims “SEO is dead.” I’ve heard it after every major Google update and every industry shake-up. Yet here we are – SEO is still around, just not in the same form as before. Today, though, I’ll admit the doomsayers are half-right: SEO as we’ve known it is on the way out. The old tricks and simple playbooks won’t carry us through the next five years. But that doesn’t mean SEO is dying; it means SEO is changing faster than ever, and we have to change along with it.

Let’s be clear-eyed about what’s happening. Traditional search – a person typing into a box and clicking a result – is sharing the stage with AI-curated experiences. We’re moving toward a world where a consumer might not use a search engine at all to get what they need. Instead, they’ll ask their phone or their car or their smart fridge a question, and an AI will deliver an answer or make a decision for them. In that world, a top Google ranking won’t be the holy grail it once was. The real currency will be trust and visibility across these AI-driven platforms. If an AI assistant trusts your brand and consistently pulls information from it, you win. If not, no amount of keyword optimization will save you.

The SEOs and businesses that refuse to acknowledge this shift are going to struggle. If you’re still clinging to the notion that this is just a phase – that users will always come back to the ten blue links, or that Google will forever need our websites to feed it – you’re in for a rude awakening. Some folks in our field are in denial, continuing with business as usual, treating AI like a toy or a fad. Meanwhile, the smart ones are experimenting and learning, figuring out how to make AI an ally rather than an adversary.

So, what does SEO look like in five years? I see a few big changes:

• Optimising for AI, not just search engines: We’ll spend less time obsessing over title tags and more time understanding how to feed the right data to AI systems so that our content is chosen and presented accurately. This might mean investing in structured data, APIs, or partnerships with AI platforms.
• Brand and authority will trump keywords: When AI decides what information to trust, it leans heavily on authority signals. Strong brands with recognised expertise will dominate. That means SEO and PR and content strategy all melt together. Your reputation in the industry (online and off) will directly influence your search visibility.
• New success metrics beyond organic traffic: As organic clicks decline in some categories, SEOs will need to help companies measure visibility in new ways. Maybe it’s being the most cited source in AI answers, or owning a high share of voice in a certain topic, even if the traffic comes through different channels. We’ll have to get creative in showing the value we provide.
• SEO as part of a broader “visibility strategy”: The job will evolve into ensuring a company’s information is accessible, accurate, and compelling across whatever platforms users turn to. That could be Google, or it could be Amazon Alexa, or a chatbot in a messaging app, or something we haven’t seen yet. I wouldn’t even be surprised if the job title changes for many of us from “SEO Specialist” to something like “Search Experience Manager” or “AI Content Strategist.”

All this might sound a bit daunting, but I find it invigorating. It’s a chance for those of us who’ve been doing this a while to reinvent ourselves and for newcomers to leapfrog ahead if they grasp the new rules quickly. I’ve already started refocusing my own efforts in these directions. I’m diving into how AI models like Google’s MUM or OpenAI rank and retrieve information. I’m working on future-proof technical solutions (like the Cloudflare Workers stuff) that give me an edge. I’m spending more time creating content and speaking, to build up that personal and brand authority. And I’m educating my clients relentlessly about why we can’t just chase last year’s SEO tactics – we need to build resilience and adaptability into everything we do.

If you’re reading this and wondering how to stay ahead, here’s my advice: embrace continuous learning and don’t get comfortable. Make it a point to understand AI, even at a basic conceptual level, because it’s going to affect our field more and more. Work on your brand – that could mean your personal brand or your company’s – because trust and recognition are invaluable in the algorithmic world. Start redefining success in your reports and conversations: include metrics like engagement, branded searches, or conversion quality, not just raw session counts. And perhaps most importantly, hone your ability to communicate. Be the translator between the tech and the takeaway. When you can explain to a CEO why their company should care about an AI update or a Core Web Vitals report, you become indispensable.

Lastly, plug into a network of people who are also looking forward. The worst thing you can do now is operate in a bubble, thinking you’ve got it all figured out. I guarantee you, none of us do. The next five years in SEO are being written as we speak, by people who are testing, sharing, and iterating. I plan to be one of them – out there in the mix, shaping what’s next.

The bottom line: SEO is not dying; it’s just not standing still. The coming years will split the field into those who evolved and those who faded away. I’ve made my choice. The question is, are you ready to make yours?

Disclaimer: I used OpenAI’s Deep Research model on this to turn my notes into an article. Technically, it reads well, but the delivery is poor.