Just say no.
We use a software to help us take notes is that okay? FUCK NO.
Luckily, I know something that would probably help her get over the trauma…
Article:
Error by AI scribe during medical appointment leaves patient devastated
By the Specialist Reporting Team’s Paige Cockburn
Posted 13 Aug 2026 at 2:55pm13 August 2026 at 2:55pm, updated 13 Aug 2026 at 4:04pm13 August 2026 at 4:04pm
An error in Rebecca Green’s medical records falsely claimed she had taken psychedelic mushrooms. (ABC News: Madeline Grace)
When Rebecca Green arrived for her first appointment with a urologist, she was asked whether she consented to having the appointment transcribed by artificial intelligence (AI).
She said yes without hesitation.
“I don’t want to be in her bad books or be that annoying patient who takes up more of her time,” she said.
But that decision would cause her a lot of stress, with the AI scribe making up the serious but false claim that Ms Green was taking illegal drugs.
“I was gobsmacked and in tears … it made up that I take [psychedelic] mushrooms. I have never done mushrooms in my life,”
she said.
More GPs and hospitals are using AI scribes. What are the risks?
AI tools are becoming more prevalent in healthcare, helping overworked doctors to carry the workload. But it has come with concerns.
New research by a privacy watchdog has raised serious concerns about AI being used in medical settings due to its propensity to “hallucinate” false symptoms or conditions, and warned patients were at risk.
Ms Green only picked up the mistake after her kidney stone surgery in March, when she read the post-operative letter her specialist sent to her GP.
It stated Ms Green micro-dosed mushrooms and that could be the reason for prior bleeding around the kidneys.
“For this letter to say something that wasn’t based in any evidence but could potentially have huge consequences for me … it was quite scary,” she said.
“It felt very impersonal to me. I wasn’t getting the level of care and familiarity from the doctor that I would be hoping for in such a vulnerable situation.”
Rebecca Green discovered the error on her medical record after reading a post-operative letter from her specialist. (ABC News: Madeline Grace)
Ms Green was particularly stressed as she was receiving workers’ compensation and felt any mention of illegal drugs on her medical record could impact her case.
Australia’s peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations. Ms Green said it was clear her doctor did not do this.
“If you had checked that letter you would have been able to correct that before it was sent out … why aren’t you in tune with each of your patients and what’s going on?”
What to know before saying yes to your doctor using AI
Health practitioners are using generative AI to help them record and summarise patient information, but some patients are saying no.
After Ms Green made a complaint, the urologist sent a letter apologising.
“Please be assured that we take the accuracy of our documentation very seriously, and we are truly sorry that this occurred,” the letter stated.
The doctor said she had not been able to determine how the claim about mushrooms came to be included, but it appeared to be from an error during the “dictation or transcription process”.
The urologist corrected the correspondence and in her letter said she would review how AI was used in her practice.
The doctor did not provide any responses to the ABC’s questions about what changes, if any, have been made since.
AI confuses which breast has cancer
Ms Green feels lucky she spotted the error, but is worried some people might not be aware of mistakes on their medical record, which could have repercussions for their health, their career or their insurance.
This week the ABC revealed instances where private health insurers were asking providers for their patients’ full clinical history as part of “aggressive” auditing practices.
AI scribes have become prevalent within the healthcare sector. The Royal Australian College of GPs (RACGP) last year estimated that 40 per cent of GPs were regularly using them, but that number was considered conservative.
The peak body for GPs in Australia has estimated that 40 per cent are regularly using AI scribes at work. (ABC News: Tim Swanston)
Research shows having AI transcribe an appointment can reduce the administrative burden for GPs and allow them to focus more on their patient, rather than the computer.
But for every benefit, there are risks.
A recent report by advocacy group Digital Rights Watch (DRW) found Australians were being exposed to unregulated AI medical scribes, sometimes without their consent, and errors had the potential to change clinical decision-making.
Do you have a story to share about AI in health care?
Email Paige Cockburn at cockburn.paige@abc.net.au
Examples of mistakes found in the DRW report included a case where a scribe recorded the wrong breast in a diagnosis of breast cancer and said someone had epilepsy, when they did not.
Chief executive of the Consumers Health Forum of Australia Elizabeth Deveny said she often heard complaints of AI mixing up words and patients only finding out by chance.
“People report both frustration in finding the original source of the error and then getting the record changed,” she said.
Elizabeth Deveny says it is frustrating for patients who have to get their records fixed when errors are made. (ABC News: Darryl Torpy)
Dr Deveny, who is currently involved in a major research project on AI medical scribes, said doctors may be too trusting of AI.
“There’s evidence that for the first few weeks of using AI, [people] check things pretty carefully and then at a point they stop checking because they assume it will be right.”
Perth GP Sean Stevens, who has been using a scribe for three years, said it was incumbent on the doctor to check every transcript, but believed the technology was becoming more accurate.
“Three years ago, I would have to correct multiple things on each transcription. These days it’s like one correction every third or fourth transcription.”
Sean Stevens says AI transcriptions are getting more accurate, but doctors still need to check them. (Supplied)
Dr Stevens is a big advocate of tech; he is the RACGP’s Digital Health and Innovation chair as well as an advisor for one of the biggest AI scribes, but he admits AI has its weaknesses and can produce incorrect results that sound plausible.
“The classic is, it will get the side of the body incorrect and say right when you’ve said left … so you need to check things pretty closely, especially things like drug doses,”
he said.
He said using AI did not mean he could squeeze more patients in, but it allowed him to run on time more often and produce more thorough referrals.
Consenting to AI’s involvement
The DRW found huge discrepancies in how consent is being obtained, with some doctors only putting a sign in the waiting room, which does not meet privacy law requirements.
Other times consent must be given when the appointment is booked online, and alarmingly some patients are even being refused appointments if they do not consent.
Dr Deveny said consent must be enthusiastic and fully informed, and patients should feel comfortable asking questions, even if it feels intimidating.
What the Consumer Health Forum recommends asking at your next medical appointment:
- Are you using AI today and if so, what are you using it for?
- Will the doctor review everything before it goes into my record?
- Will I be able to get a copy of the notes if I want them?
- What’s the process if I think something’s wrong?
The DRW report also identified that people might not be forthcoming about sensitive issues, like family violence, when they know AI is listening.
Dr Stevens said that was why he asked for consent in every appointment, even with regular patients.
“It’s very important patients disclose any and all things that are relevant to their health … and that they know they can say no [to AI] … as a doctor you have to respect that,” he said.
TGA flags breaches will be dealt with
There are around a dozen AI medical scribes available in Australia, but none are approved by the regulator — the Therapeutic Goods Administration (TGA) — because if the product claims to be solely for transcription and summarising, it is exempt from oversight.
The TGA only considers tools that do things like suggesting a diagnosis or treatment plan as a “medical device”.
AI changing the way GPs tend to their patients
Some GPs are now relying on artificial intelligence to take notes during patient consultations. So, how safe is our sensitive information?
Currently, no AI scribes have gone through the regulation process, despite some claiming to offer clinical suggestions.
The TGA has acknowledged this means some scribes were potentially in breach of the law.
The TGA has spent the last year reviewing AI scribes and says its findings will be released in the coming weeks, and any breaches will be acted upon.
Chief executive of Patients Australia Lisa Robins believed all scribes should be regulated, as summarising a patient still involved decision-making by AI.
Dr Stevens had concerns that extending regulation would only stifle future innovation.
The DRW is calling on the newly established AI Safety Institute to test AI scribes and publish a list of approved tools.
This would be similar to New Zealand’s approach, where the national AI advisory group reviewed tools and last year gave approval for clinicians to use two AI tools: Heidi Health and iMed.
Ms Deveny said ensuring doctors were using AI safely could also be enforced in other ways, such as compulsory checks and balances built into the software, or during practice accreditation, which happens every three years.
The error on her medical record has left Rebecca Green reconsidering if she would give consent to AI transcription again. (ABC News: Madeline Grace)
Ms Green said her trust in AI has been splintered by what happened and now would only give consent for AI if she had an established relationship with the doctor.
“I love AI … but the essential ingredient is the human. At the end of the day what’s happened here is more about the lack of human involvement,”
she said.
The patient was devastated? You should have seen how many times the AI apologized.
As far as I’m concerned, using AI of any sort inside of a doctor’s office is a grave violation of HIPAA
Let me tell you about how medical transcriptionists used to do it: Sister in law was one, she didn’t seem to care about HIPAA too much, she and all her colleagues did lose their jobs about a 15-20 years ago when AI transcription software took over their jobs, but before that she was frequently given jobs because her predecessors made serious mistakes in their transcriptions.
If she didn’t care about HIPAA, I’m glad she lost the job lol.
If only we could fire AI too.
Ai transcription is everywhere. You cant get away from it
Yeah? Watch me.
Watch me and millions like me push back everywhere and slowly change the world.
Speech-to-text (a type of AI) has been around for a fair good years, but now they’re being paired with generative AI. This is the kind of thing that shouldn’t be paired with a generative LLM. A language model to identify words? Yes. Generative? No.
Yes. It technically does improve accuracy (homophone correction, for example) but the occasional errors are worse and way less obvious.
Australia’s peak regulator of medical practitioners, AHPRA, said clinicians must always check all output from an AI scribe for accuracy to ensure they meet their professional obligations.
Show me one single AI user that checks the output of their AI, and I will show you my unicorn
I just asked SlopGPT a question and checked the output. Can I see your unicorn?
(That was a lie. I just really want to see a unicorn.)
Not gunna get me with the ole “wanna see a unicorn” trick twice.
I read “medieval” appointment and was thoroughly confused, and intrigued.
deleted by creator
I’ve used AI for medical transcription before and it is absolutely incredible. It’s not just a word for word text, but a detailed clinical breakdown of everything discussed.
All you have to do is simply read the damn thing thoroughly and correct whatever minor errors pop up.
Takes 95% less time.
All you have to do is fire the human who is trained and rarely makes a mistake and replace them with a paid machine that almost always makes a mistake and double check everything yourself (or hire another human to double check it) because someone’s life literally depends on the entirety of the text being absolutely correct.
AI does work at a level that makes unpaid interns look like career professionals. All for the low cost of completely fucking the environment.
Glad you like it.
I’m unsure if there’s an ethical issue with putting patients medical records in the data machine.
There exists online and offline ML.
Transcription is not a task that needs a full blown LLM, summarization probably does, and if being done offline or on a server with trusted verifiable lack of logging should be fine. That does not mean send it to ChatGPT API.
I would hope they store the raw conversation file encrypted with patient records so they can retranscribe later as tools improve.
Medical AI has guardrails out the wazoo. By the time it’s made available to a physician, it’s probably been reviewed by a council, vetted by IT, and negotiated heavily by Legal. If the data leaves the premises, it’ll be thoroughly protected and isolated.
That said, fuck AI. I’d give up medical transcription to get the environment, economy, and sanity back. Musk’s data center is running dozens of natural gas turbines in a low-income, primarily Black neighborhood 11 miles down the road from St. Jude Children’s Research Hospital. It’s positively ghoulish.
Horseshit. Medical AI is using Oracle services. Larry mother fucking Elison is storing and processing this data.
The medical industry seems so ignorant about this. They think the argument is just about AI. It’s also about the fact that I don’t trust the people you’re partnered with.
I get so pissed when the AI consent is baked into other paperwork with no opt out. I will leave.
Edit: https://www.oracle.com/customers/beacon-health/
Edit: they might not all be, but I’m not authorizing your arbitrary “AI use” clause. You might as well be, now or at some point down the road.
The agreement is only as good as the hospital’s vetting, so I may be a little optimistic after seeing a very thorough and protracted process on my side. If you work for a healthcare system that’s okay with Beacon and Oracle, then it may be time to evaluate whether they’re a good healthcare system to work for.
Clue: Medical transcription has been done by “AI” for the past 20 years. Source: sister in law was a medical transcriptionist making $80K per year, until Dragon Naturally Speaking and friends decimated her job market.
It needs to be a system and a contractual framework that complies with the medical privacy regulations of the place where it’s being used (in the US, HIPAA). I have some doubts about whether specific companies or people will actually comply, or the enforcement of those laws by certain governmental authorities, but I don’t think it’s conceptually impossible to have this kind of tech be compliant with those types of regulations.
Exactly. It is a tool for professionals to use to speed up existing processes. It is not a replacement for people, regardless of what the marketers tell you so you buy their product. I don’t understand how there aren’t thousands of lawsuits against these AI companies for their blatantly false advertising.
AI keeps proving this day by day when people try to make it a replacement and have it blow up in their faces every day.
blatantly false advertising.
SEC is neutered, what makes you think anyone in charge of advertising rules hasn’t been replaced by a 17 year old called Goebbels Gobbler or something with a DOGE laptop.
SEC is neutered
This article is about an incident in Australia. The SEC has no bearing on what Australia can do.
The AI bubble is an American phenomenon and their SEC isn’t slowing down the insanity. Their lack of a functional SEC is everyones problem.
Hell, I just found out a few months ago my father’s office (he’s a lawyer) still did transcriptions of court proceedings manually! Full blown lawyers wasting expensive man hours doing that. As soon as I found out I got some random whisper cpp frontend and slapped it on a random computer
I hope his clients are aware
As long as they still have someone verifying and correcting mistakes, that’s a decent solution. It doesn’t need to be complicated. Whisper is by no means perfect, but from my experience it handles 90% fine, only having to listen and correct 10% is a huge improvement over transcribing everything manually.
The issues with so much of the current AI use is implementations that completely remove trained human oversight from the equation because the AI companies position it as a complete replacement for employees.
from my experience it handles 90% fine, only having to listen and correct 10% is a huge improvement over transcribing everything manually.
I’m curious how this is supposed to work? How do you only listen to 10%? How are you supposed to know which 10% to listen to? Wouldn’t you still need to listen to it all to find that 10%?
Yes, but listening to it all once and correcting the mistakes as you hear/see them would be faster than transcribing the entire thing, then proof reading.
All these articles, nobody posting error budgets or Delta’s with real doctors in this shit.
But we all know doctors are perfect with perfect notes every time and every nurse who gives you a dose 🤷♂️
God forbid that doctor take 2 minutes to confirm accuracy.
but it’s 90% accurate!
Ignoring that this means that one in every ten patients is getting inaccurate notes.
This will literally lead to people dying.
How accurate are the doctors? How accurate are they both in tandem? Are they higher than 90%?
Dual systems working on a decision could be significantly safer.
But Lemmy hates AI and loves to huff its own farts
It’s a fair point. Unpopular in this community, but fair. Of course, nuance is rarely rewarded in online discussion
“If you’re not with us, you’re against us. Huh. If you’re not an apple, you’re a banana.”
-Terry Pratchett, Thud
Fediverse has turned into the truth social of disenfranchised progressives. Don’t get me wrong I agree with many themes in this place, but the tribalism and unwillingness to refrain from immediately labeling a topic or person a total bad actor is pretty high. Everyone is a mustache twirling billionaire, redneck who hates trans and are Nazis, or a progressive hero.
I like to eat apples and bananas.
