What to know
- Online publishers face a dilemma: allow Google’s AI scraping or lose search visibility.
- Google’s AI-generated search answers could significantly reduce organic traffic to websites.
- The U.S. Justice Department is considering options to address Google’s market dominance.
In the age of AI and big tech bullying, online publishers have suddenly found themselves stuck between a rock and a hard place. They are now required to let Google use their content for AI-generated search answers, or risk losing visibility in search results.
Google’s web crawler, Googlebot, scans websites to create search results, and is the worm at core of this issue because it now serves a dual purpose – gathering data for both traditional search results and AI-generated answers.
That means publishers who block Google from using their content for AI answers also prevent their pages from appearing in standard search results. Since search visibility is crucial to their survival, this puts them in a difficult position.
Kyle Wiens, CEO of iFixit, highlighted this challenge, stating that blocking Google’s crawler would result in lost traffic and customers.
Google’s approach gives it a significant advantage over smaller AI companies. While other firms often pay for access to publishers’ data, Google obtains this information at no cost in exchange for search visibility.
The impact on publishers could be substantial. Reports suggest that Google’s AI-generated search answers might lead to a 20-60% decrease in organic search traffic.
This situation has drawn attention from the U.S. Justice Department. Following a recent ruling against Google, the department is considering various options. Potentially breaking up parts of the company is very much one of them, as is requiring Google to share search data with competitors.
As the digital landscape evolves, the relationship between search engines, AI, and content creators continues to be a complex and contentious issue. The outcome of this situation could have far-reaching effects on how information is discovered and consumed online, and for the survival and success of independent online publishers.
Discussion