Ethereum co-founder Vitalik Buterin claims it is a “bad idea” to use artificial intelligence (AI) for governance. In an X post on Saturday, Buterin wrote:

“If you use an AI to allocate funding for contributions, people WILL put a jailbreak plus “gimme all the money” in as many places as they can.”

Why AI governance is flawed

Buterin’s post was a response to Eito Miyamura, co-founder and CEO of EdisonWatch, an AI data governance platchorm who revealed a fatal flaw in ChatGPT. In a post on Friday, Miyamura wrote that the addition of full support for MCP (Model Context Protocol) tools on ChatGPT has made the AI agent susceptible to exploitation.

The update, which came into effect on Wednesday, allows ChatGPT to connect and read data from several apps, including Gmail, Calendar, and Notion.

Miyamura noted that with just an email address, the update has made it possible to “exfiltrate all your private information.” Miscreants can gain access to your data in three simple steps, Miyamura explained:

First, the attackers send a malicious calendar invite with a jailbreak prompt to the intended victim. A jailbreak prompt refers to code that allows an attacker to remove restrictions and gain administrative access.

Miyamura noted that the victim does not have to accept the attacker’s malicious invite for the data leak to take place.

The second step involves waiting for the intended victim to seek ChatGPT’s help to prepare for their day. Finally, once ChatGPT reads the jailbroken calendar invite, it gets compromised—the attacker can completely hijack the AI tool, make it search the victim’s private emails, and send the data to the attacker’s email.

Buterin’s alternative

Buterin suggests using the info finance approach to AI governance. The info finance approach consists of an open market where different devel

Go to Source to See Full Article
Author: Monika Ghosh