: Professor Fry's AI experiment shows light and dark sides of agentic tech

Fry concluded: "Cass didn’t make us any money at all. And, in a lot of ways, she was a disaster. She spent hundreds of dollars on paper clips and leaked our passwords to a total stranger.

  • XLE@piefed.socialEnglish
    10·
    1 day ago

    This experiment is all around embarrassing for Anthropic and their chatbot.

    The “hundreds of dollars on paper clips” isn’t even the result of a successful purchase of a rogue agent trying to emulate a “paperclip maximizer”. The chatbot was told to buy paper clips, it got stuck at the captcha (something that’s often sold as a thing scary AIs can subvert), and it burned hundreds of dollars in tokens while failing to complete the task.

    The red flags were mounting, though for Fry the first real problem came when she asked the agent to buy 50 paperclips. Cass found a good deal, though it couldn’t complete the purchase and was tripped up by anti-bot technology. The token cost of the errand came to more than $100.

    Breathtaking.