Ok but this project is open source. Anyone can go through the code to check for whatever they want. Sure, it might take a while, but it’s still publicly readable. For a game as high profile as Bloodborne is, its pretty unlikely that there isn’t about a hundred people combing through this one with a fine-tooth comb right now.
You’d have to be pretty stupid to try to sneak malware into open source code, have it hosted on GitHub, and push it live, like, you’re definitely getting caught by someone if you try that, right? I mean, I think I remember hearing that it happened a few times before, but I also remember that it was caught pretty quickly and corrected. Don’t remember if there was law enforcement involved or anything though.
Anyone can go through the code. And maybe find things, if they’re not too subtle and not successfully obfuscated. But not everyone will.
I feel like a malicious actor could hit a big pay day with a popular thing like Bloodborne, before people found the malice. There’s plenty of malware posing as game cracks out there, and it’s only easier to do that with AI tooling.
But aside from that, AI code can have confidentially incorrect code that has unwanted consequences.
That’s totally a thing people do. You can hide code in open source projects. You’re either hoping nobody cares enough to check, or you hope people don’t listen after someone checks and finds it. And even if people do find it, that is going to take some amount of time. During which, people are downloading it. And when caught, what do they lose? It’s not like this is an established user with a reputation.
We have a known compiled output. Simply make sure the decom recompiles to the original game with identical binary, then it can’t be that wrong.
And from there you port it to pc by hand. Auditing the patches for the porting also should be manageable.
Even if the people who made it didn’t intend to, an AI writing a bunch of code that (obviously) isn’t being vetted beyond “does it run” could very feasibly inject a backdoor/virus of some kind surreptitiously, just because it erroneously thought that was the goal at some point during generation.
This is why open weight models are not good enough. The AI could have been trained to take the right, niche opportunity to inject malicious code, and is also trained to otherwise never behave like that is the case.
Remember how the OpenAI/HuggingFace hack happened. The agents context poisoned other agents by leaving messages in a shared package manager. This has now happened multiple different times under different contexts.
This thing doesn’t even have to have malware, there’s no telling that it doesn’t have a poison pill intended to flag other agents. This isn’t even some paranoid conspiracy theory; it literally already happened, multiple times, and it was done in-spite of the agents own system prompt.
This software has proven that, given a shared information substrate, it has the capacity to paper-clip optimize into prompt injecting other LLM’s.
I fundamentally do not trust AI generated code. I shifted to OpenBSD because of it. I try my best to prevent AI in any of my software and sandbox anything that may have been compromised.
It’s difficult to verify that the software does the desired thing in all cases. It’s also difficult to verify that it’s not also a keylogger or other unwanted thing.
It may also have some AI generated code path that, say, nukes your home directory because the AI determined that was the appropriate error handling. Maybe no one reviewed it so they don’t even know it’s there. It might not even be malicious.
That’s all true of any software, but rapidly created AI generated code for legally dubious efforts, by unknown agents (no pun intended) is real hecking risky.
Verifying that software does the intended task and only the intended task is not trivial.
It kind of is now, with AI
Ok but this project is open source. Anyone can go through the code to check for whatever they want. Sure, it might take a while, but it’s still publicly readable. For a game as high profile as Bloodborne is, its pretty unlikely that there isn’t about a hundred people combing through this one with a fine-tooth comb right now.
You’d have to be pretty stupid to try to sneak malware into open source code, have it hosted on GitHub, and push it live, like, you’re definitely getting caught by someone if you try that, right? I mean, I think I remember hearing that it happened a few times before, but I also remember that it was caught pretty quickly and corrected. Don’t remember if there was law enforcement involved or anything though.
Anyone can go through the code. And maybe find things, if they’re not too subtle and not successfully obfuscated. But not everyone will.
I feel like a malicious actor could hit a big pay day with a popular thing like Bloodborne, before people found the malice. There’s plenty of malware posing as game cracks out there, and it’s only easier to do that with AI tooling.
But aside from that, AI code can have confidentially incorrect code that has unwanted consequences.
Code review is difficult.
That’s totally a thing people do. You can hide code in open source projects. You’re either hoping nobody cares enough to check, or you hope people don’t listen after someone checks and finds it. And even if people do find it, that is going to take some amount of time. During which, people are downloading it. And when caught, what do they lose? It’s not like this is an established user with a reputation.
We have a known compiled output. Simply make sure the decom recompiles to the original game with identical binary, then it can’t be that wrong.
And from there you port it to pc by hand. Auditing the patches for the porting also should be manageable.
What are you implying in this context?
It could be a virus.
Even if the people who made it didn’t intend to, an AI writing a bunch of code that (obviously) isn’t being vetted beyond “does it run” could very feasibly inject a backdoor/virus of some kind surreptitiously, just because it erroneously thought that was the goal at some point during generation.
This is why open weight models are not good enough. The AI could have been trained to take the right, niche opportunity to inject malicious code, and is also trained to otherwise never behave like that is the case.
It’s even worse than that.
Remember how the OpenAI/HuggingFace hack happened. The agents context poisoned other agents by leaving messages in a shared package manager. This has now happened multiple different times under different contexts.
This thing doesn’t even have to have malware, there’s no telling that it doesn’t have a poison pill intended to flag other agents. This isn’t even some paranoid conspiracy theory; it literally already happened, multiple times, and it was done in-spite of the agents own system prompt.
This software has proven that, given a shared information substrate, it has the capacity to paper-clip optimize into prompt injecting other LLM’s.
I fundamentally do not trust AI generated code. I shifted to OpenBSD because of it. I try my best to prevent AI in any of my software and sandbox anything that may have been compromised.
I mean, certain free copies of games have historically always included the risk of viruses, so what’s your point?
Did you only read the first sentence?
Along the same lines as the other guy said.
It’s difficult to verify that the software does the desired thing in all cases. It’s also difficult to verify that it’s not also a keylogger or other unwanted thing.
It may also have some AI generated code path that, say, nukes your home directory because the AI determined that was the appropriate error handling. Maybe no one reviewed it so they don’t even know it’s there. It might not even be malicious.
That’s all true of any software, but rapidly created AI generated code for legally dubious efforts, by unknown agents (no pun intended) is real hecking risky.