minus-squareaesopjah@sh.itjust.workstoTechnology@lemmy.world•Announcing ARC-AGI-3 - A benchmark that tests if AI can explore, learn, and adapt in unfamiliar situations. Humans score 100%. Frontier AI scores 0.26%.linkfedilinkEnglisharrow-up9arrow-down1·19 hours agoit’s also an odd metric since only 20-60% of the humans completed it. Very 60% of the time they complete it everytime energy. Ideally they’d run the bots multiple times through (with no context or training of previous run), but I guess that is cost prohibitive? linkfedilink
it’s also an odd metric since only 20-60% of the humans completed it. Very 60% of the time they complete it everytime energy.
Ideally they’d run the bots multiple times through (with no context or training of previous run), but I guess that is cost prohibitive?