I don’t understand why this happens; why would you ever be working with a live production DB in the first place? Why would’t you do all your development and testing on a mock? If it’s data which is too large to store the schema can still be mocked; and if it’s data it should be backed up and generally read only. If you’re having to manually fuss with user data you’re doing something wrong.
Because they don’t know what they’re doing. A tool is only as useful as the person verifying its output. Vibe coders have dumb shit like this happen to them all the time because they don’t actually possess the skill set to perform the task correctly, with or without a bot that writes the actual code for them.
Well, sure, but even if a developer correctly tests on a development environment, they still probably have prod settings laying around on their filesystem for the times where they need to put out fires in prod. That’s kinda what it looks like what happened here - the LLM found a .env file with prod settings, and used that config to run destructive tests. All the more reason to not give an LLM side wide reaching access to your computer, but it’s not necessarily an indication that someone regularly tests in prod.
We have to remote into a management server to access databases outside of our local dev environment. Allowing direct access to production from your development environment seems crazy to me.
Some places deploy to prod multiple times a day. Trying to coordinate access through a management server in situations like that can get cumbersome. It’s also easier to audit who’s doing what if they connect from their personal environment. Different strokes for different folks of course, but it’s certainly not uncommon to be able to connect to prod directly from a dev box.
Prod deployments should come from a build server though and if you’re deploying multiple times a day you should have a well built CI pipeline to handle it.
Remoting into a management server would still have people using an account on the management server and their personal db login credentials.
We also don’t give devs write access to production and all scripts run in production are run by our DBA after they have been approved by the lead Dev, operations and the DBA.
This might be overkill for some systems, but we won’t ever accidentally delete our prod data. Our system requires this level of scrutiny due to the nature of our systems.
Sure, that’s a very, very robust way of doing things. And I’m not trying to say what you’re describing is wrong either. I’m just saying that some places don’t have the knowledge/ability to set things up that way, so they play fast and loose with prod access. Couple that with giving LLMs free reign over the computer with those prod settings, and you end up in situations like in the original post.
It’s not unrealistic, is all I’m trying to claim. It’s a failure on several levels, but it’s not unrealistic.
that is absolutely NOT the way that works in any functional environment
your ability to deploy quickly comes directly from your automation, and those automation tools - NOT developers - have the secrets in them
deploy to prod multiple times per day isn’t some win by itself… the ability for large teams (not 1 fuckwit and a goon squad of agents) to deploy without breaking things and in ways that are safe is the win here… anyone can deploy to prod multiple times per day… but anyone isn’t netflix (the originators of the “multiple times per day” line) with the uptime they achieve over years while doing it
I don’t understand why this happens; why would you ever be working with a live production DB in the first place? Why would’t you do all your development and testing on a mock? If it’s data which is too large to store the schema can still be mocked; and if it’s data it should be backed up and generally read only. If you’re having to manually fuss with user data you’re doing something wrong.
Because they don’t know what they’re doing. A tool is only as useful as the person verifying its output. Vibe coders have dumb shit like this happen to them all the time because they don’t actually possess the skill set to perform the task correctly, with or without a bot that writes the actual code for them.
Well, sure, but even if a developer correctly tests on a development environment, they still probably have prod settings laying around on their filesystem for the times where they need to put out fires in prod. That’s kinda what it looks like what happened here - the LLM found a .env file with prod settings, and used that config to run destructive tests. All the more reason to not give an LLM side wide reaching access to your computer, but it’s not necessarily an indication that someone regularly tests in prod.
We have to remote into a management server to access databases outside of our local dev environment. Allowing direct access to production from your development environment seems crazy to me.
Some places deploy to prod multiple times a day. Trying to coordinate access through a management server in situations like that can get cumbersome. It’s also easier to audit who’s doing what if they connect from their personal environment. Different strokes for different folks of course, but it’s certainly not uncommon to be able to connect to prod directly from a dev box.
Prod deployments should come from a build server though and if you’re deploying multiple times a day you should have a well built CI pipeline to handle it.
Remoting into a management server would still have people using an account on the management server and their personal db login credentials.
We also don’t give devs write access to production and all scripts run in production are run by our DBA after they have been approved by the lead Dev, operations and the DBA.
This might be overkill for some systems, but we won’t ever accidentally delete our prod data. Our system requires this level of scrutiny due to the nature of our systems.
Sure, that’s a very, very robust way of doing things. And I’m not trying to say what you’re describing is wrong either. I’m just saying that some places don’t have the knowledge/ability to set things up that way, so they play fast and loose with prod access. Couple that with giving LLMs free reign over the computer with those prod settings, and you end up in situations like in the original post.
It’s not unrealistic, is all I’m trying to claim. It’s a failure on several levels, but it’s not unrealistic.
that is absolutely NOT the way that works in any functional environment
your ability to deploy quickly comes directly from your automation, and those automation tools - NOT developers - have the secrets in them
deploy to prod multiple times per day isn’t some win by itself… the ability for large teams (not 1 fuckwit and a goon squad of agents) to deploy without breaking things and in ways that are safe is the win here… anyone can deploy to prod multiple times per day… but anyone isn’t netflix (the originators of the “multiple times per day” line) with the uptime they achieve over years while doing it