Google is asking web content providers for permission to use their material in its Bard AI and future generative AI models. This move comes as Google aims to address concerns about the use of web content without consent and ensure ethical data collection.
To allow or disallow Google to use web content, providers can simply add or remove “User-Agent: Google-Extended” in their site’s robots.txt file. This file informs web crawlers about the content they can access.
While Google emphasizes its commitment to ethical AI development, this request raises questions about the use of data for AI training purposes. Although the word “train” doesn’t appear in Google’s statement, it’s evident that the data is used to train machine learning models.
The Vice President of Trust at Google, Danielle Romain, framed the request as a choice, asking content providers whether they want to “help improve Bard and Vertex AI generative APIs” and contribute to the enhancement of AI models’ accuracy and capabilities over time.
This approach highlights the importance of consent in data usage. However, some critics argue that Google’s request lacks authenticity, given that its AI models have already been trained on extensive user data without explicit consent.
This move by Google follows recent actions by platforms like Medium, which announced its decision to universally block web crawlers until a more refined solution is developed. Many other platforms are taking similar measures to address data usage concerns.
