Tesla Execs Are Freaking Out

You might not believe what you're about to see is real, but I assure you the video has been left unedited. It's so revolutionary that tech giants like Microsoft, Amazon, and NVIDIA are all lining up to get their hands on it. And Elon Musk has completely restructured Tesla to chase this opportunity.

Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said

GARANCE BURKE and HILKE SCHELLMANN
October 26, 2024

SAN FRANCISCO (AP) -- Tech behemoth OpenAI has touted its artificial intelligence-powered transcription tool Whisper as having near "human level robustness and accuracy."

But Whisper has a major flaw: It is prone to making up chunks of text or even entire sentences, according to interviews with more than a dozen software engineers, developers and academic researchers. Those experts said some of the invented text -- known in the industry as hallucinations -- can include racial commentary, violent rhetoric and even imagined medical treatments.

Experts said that such fabrications are problematic because Whisper is being used in a slew of industries worldwide to translate and transcribe interviews, generate text in popular consumer technologies and create subtitles for videos.

More concerning, they said, is a rush by medical centers to utilize Whisper-based tools to transcribe patients' consultations with doctors, despite OpenAI' s warnings that the tool should not be used in "high-risk domains."

The full extent of the problem is difficult to discern, but researchers and engineers said they frequently have come across Whisper's hallucinations in their work. A University of Michigan researcher conducting a study of public meetings, for example, said he found hallucinations in eight out of every 10 audio transcriptions he inspected, before he started trying to improve the model.

A machine learning engineer said he initially discovered hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed. A third developer said he found hallucinations in nearly every one of the 26,000 transcripts he created with Whisper.

The problems persist even in well-recorded, short audio samples. A recent study by computer scientists uncovered 187 hallucinations in over 13,000 clear audio snippets they examined.

That trend would lead to tens of thousands of faulty transcriptions over millions of recordings, researchers said.

Such mistakes could have "really grave consequences," particularly in hospital settings, said Alondra Nelson, who led the White House Office of Science and Technology Policy for the Biden administration until last year.

"Nobody wants a misdiagnosis," said Nelson, a professor at the Institute for Advanced Study in Princeton, New Jersey. "There should be a higher bar."

Whisper also is used to create closed captioning for the Deaf and hard of hearing -- a population at particular risk for faulty transcriptions. That's because the Deaf and hard of hearing have no way of identifying fabrications are "hidden amongst all this other text," said Christian Vogler, who is deaf and directs Gallaudet University's Technology Access Program.

OpenAI urged to address problem

The prevalence of such hallucinations has led experts, advocates and former OpenAI employees to call for the federal government to consider AI regulations. At minimum, they said, OpenAI needs to address the flaw.

"This seems solvable if the company is willing to prioritize it," said William Saunders, a San Francisco-based research engineer who quit OpenAI in February over concerns with the company's direction. "It's problematic if you put this out there and people are overconfident about what it can do and integrate it into all these other systems."

An OpenAI spokesperson said the company continually studies how to reduce hallucinations and appreciated the researchers' findings, adding that OpenAI incorporates feedback in model updates.

While most developers assume that transcription tools misspell words or make other errors, engineers and researchers said they had never seen another AI-powered transcription tool hallucinate as much as Whisper.

Whisper hallucinations

The tool is integrated into some versions of OpenAI's flagship chatbot ChatGPT, and is a built-in offering in Oracle and Microsoft's cloud computing platforms, which service thousands of companies worldwide. It is also used to transcribe and translate text into multiple languages.

In the last month alone, one recent version of Whisper was downloaded over 4.2 million times from open-source AI platform HuggingFace. Sanchit Gandhi, a machine-learning engineer there, said Whisper is the most popular open-source speech recognition model and is built into everything from call centers to voice assistants.

Professors Allison Koenecke of Cornell University and Mona Sloane of the University of Virginia examined thousands of short snippets they obtained from TalkBank, a research repository hosted at Carnegie Mellon University. They determined that nearly 40% of the hallucinations were harmful or concerning because the speaker could be misinterpreted or misrepresented.

In an example they uncovered, a speaker said, "He, the boy, was going to, I'm not sure exactly, take the umbrella."

But the transcription software added: "He took a big piece of a cross, a teeny, small piece ... I'm sure he didn't have a terror knife so he killed a number of people."

A speaker in another recording described "two other girls and one lady." Whisper invented extra commentary on race, adding "two other girls and one lady, um, which were Black."

In a third transcription, Whisper invented a non-existent medication called "hyperactivated antibiotics."

Researchers aren't certain why Whisper and similar tools hallucinate, but software developers said the fabrications tend to occur amid pauses, background sounds or music playing.

OpenAI recommended in its online disclosures against using Whisper in "decision-making contexts, where flaws in accuracy can lead to pronounced flaws in outcomes."

Transcribing doctor appointments

That warning hasn't stopped hospitals or medical centers from using speech-to-text models, including Whisper, to transcribe what's said during doctor's visits to free up medical providers to spend less time on note-taking or report writing.

Over 30,000 clinicians and 40 health systems, including the Mankato Clinic in Minnesota and Children's Hospital Los Angeles, have started using a Whisper-based tool built by Nabla, which has offices in France and the U.S.

That tool was fine tuned on medical language to transcribe and summarize patients' interactions, said Nabla's chief technology officer Martin Raison.

Company officials said they are aware that Whisper can hallucinate and are mitigating the problem.

It's impossible to compare Nabla's AI-generated transcript to the original recording because Nabla's tool erases the original audio for "data safety reasons," Raison said.

Nabla said the tool has been used to transcribe an estimated 7 million medical visits.

Saunders, the former OpenAI engineer, said erasing the original audio could be worrisome if transcripts aren't double checked or clinicians can't access the recording to verify they are correct.

"You can't catch errors if you take away the ground truth," he said.

Nabla said that no model is perfect, and that theirs currently requires medical providers to quickly edit and approve transcribed notes, but that could change.

Privacy concerns

Because patient meetings with their doctors are confidential, it is hard to know how AI-generated transcripts are affecting them.

A California state lawmaker, Rebecca Bauer-Kahan, said she took one of her children to the doctor earlier this year, and refused to sign a form the health network provided that sought her permission to share the consultation audio with vendors that included Microsoft Azure, the cloud computing system run by OpenAI's largest investor. Bauer-Kahan didn't want such intimate medical conversations being shared with tech companies, she said.

"The release was very specific that for-profit companies would have the right to have this," said Bauer-Kahan, a Democrat who represents part of the San Francisco suburbs in the state Assembly. "I was like 'absolutely not.'"

John Muir Health spokesman Ben Drew said the health system complies with state and federal privacy laws.

___

Schellmann reported from New York.

___

This story was produced in partnership with the Pulitzer Center's AI Accountability Network, which also partially supported the academic Whisper study.

___

The Associated Press receives financial assistance from the Omidyar Network to support coverage of artificial intelligence and its impact on society. AP is solely responsible for all content. Find AP's standards for working with philanthropies, a list of supporters and funded coverage areas at AP.org.

___

The Associated Press and OpenAI have a licensing and technology agreement allowing OpenAI access to part of the AP's text archives.

Continue Reading...

Popular

Nuclear Energy Stocks Are Hot: Here's A List Of Tickers To Watch

Nuclear energy-related stocks have taken off following major announcements from Google and Amazon as the tech industry turns to nuclear energy to power AI data centers.

Elon Musk's Mother Does Harris Impression During TV Appearance

Maye Musk, mother of Elon Musk, has sparked a flurry of reactions following her controversial imitation of Kamala Harris on Fox Business.

Bill Gates' Mysterious New AI Project Revealed - Ad

Tech expert Jeff Brown uncovers Bill Gates' latest AI project, Stargate, which Forbes claims could reshape everything we think we know about our world.

Elon Musk Want His 11 Kids, Their Mothers To Live In Secret Compound He Bought, New Report Says

Elon Musk could be building a new family compound made up of several mansions to house his children and their mothers, according to a new report.

Trump: Harris Presidency Would 'Guarantee' World War III

Donald Trump said Saturday that the election of Kamala Harris to the presidency could trigger World War III.

The Mysterious VC Firm Behind Nvidia's Secret Weapon - Ad

In 1999, Sutter Hill Ventures made a bold bet on Nvidia before anyone had heard of it. Now, they're going all-in on Nvidia's hush-hush partner that's powering their new Blackwell chip. Discover the little-known company that's attracting massive investments from the visionaries behind Nvidia's 100,000% rise.

Court approves Tupperware's sale to lenders, paving way for brand's exit from bankruptcy

NEW YORK (AP) — A U.S. bankruptcy judge approved a sale of on Tuesday, paving the way for the iconic food-shortage company to soon exit and continue offering its products while undergoing a hoped-for revitalization.

Listen to business owners and voters in Erie, Pennsylvania. They could help decide the White House.

ERIE, Pa. (AP) — It wasn’t much when he bought it, but Michael Hooks has made the old garage his own haven. And the city of Erie knows it. Half a dozen cars honk as they pass by one October afternoon, the people hanging out of car windows to wave hello at him.

Unlock the Ultimate Gold Play: $20 for a 1-Oz. Stake in Gold! - Ad

Gold has already soared past $2,500 an ounce. But with this 'backdoor' strategy, you can gain exposure to over an ounce for just $20. As the Fed cuts rates, the stage is set for a major gold boom.

Amazon Shares Surge In Pre-Market As Q3 Earnings Beat Expectations

This surge followed the announcement of third-quarter net sales hitting $158.9 billion, marking an 11% increase from the previous year.

LA County sues Pepsi, Coca-Cola over plastic bottles

LOS ANGELES (AP) — Los Angeles County is taking on Pepsi and Coke for their role in plastic pollution.

Wavebot: The Automated Trading Tool for This Bull Market - Ad

October through December are historically among the most bullish months for crypto. WaveBot -- an automated trading tool that takes all the guesswork out of profiting in crypto. It trades key levels, buying dips and selling at the perfect time, 24/7. All you need to do is set it up. After that, WaveBot takes care of everything.

Which Industries Have The Highest Cannabis Consumption At Work? New Data Has Answers

Explore the latest CDC analysis on workplace cannabis use across industries. Discover which sectors are driving this trend!

Erykah Badu celebrated as fashion icon at CFDA Awards after years of pushing boundaries

NEW YORK (AP) — Erykah Badu was bestowed the icon award by the Council of Fashion Designers of America, playfully saying it was an honor she’d been hoping to get most of her life.

Why Trump's Victory Could Fuel the Next AI Surge - Ad

Louis Navellier, tech investor and market veteran, believes Trump's return will drive a second AI boom. Get the top six stocks he recommends before they skyrocket.

Chinese online retailer Temu faces European Union investigation into rogue traders and illegal goods

LONDON (AP) — Chinese online retailer Temu is facing a European Union investigation over suspicions it's failing to prevent the sale of illegal products, the 27-nation bloc's executive arm said on Thursday.

Will people leave Florida after devastating hurricanes? History suggests not

ORLANDO, Fla. (AP) — The news rippled through Treasure Island, Florida, almost like a third storm: The mayor planned to move off the barrier island a month after flooded tens of thousands of homes along the Gulf Coast and two weeks after also ravaged the state.

The Secret Weapon You Need for This Bull Market - Ad

There's a powerful tool that can remove all the guesswork from trading. It's built to trade the key levels for you, buying dips and selling at the right time, without you having to lift a finger. For a limited time, the creators of this incredible tool are holding a free Workshop where you can see the software in action.

Iran Flexes Nuclear Muscle, Threatens US With 'Tooth-Breaking' Retaliation

Iran recently announced its capacity to develop nuclear weapons and issued a stern warning to the U.S. and Israel, threatening a "tooth-breaking response" to Israel's latest attack on Iranian military targets.

Financial market worries over the UK budget appear to ease

LONDON (AP) — Worries about this week's within financial markets appeared to ease Friday with the interest rates charged on British debt steadying and the pound rising against most other currencies.

Single Stock Blueprint Available Now - Ad

Market Wizard Larry Benedict is one of the most successful traders you'll ever meet...But he doesn't make money the traditional way. In short: It's a way to trade just one stock...And potentially make all the money you need to fast-track your retirement -- no matter what happens in the stock market.

Microsoft fires employees who organized vigil for Palestinians killed in Gaza

Microsoft has fired two employees who organized an unauthorized vigil at the company's headquarters for Palestinians killed in Gaza during Israel's war with Hamas.

Sam Altman, Bill Gates and More Are Making Some Strange Investments - Ad

As a Silicon Valley insider, Michael Robinson's uncovered an even bigger story than AI... A mysterious investment that could be the savior of the tech industry. Big Tech executives -- including Sam Altman, Bill Gates and Mark Zuckerberg -- are throwing large sums of money at it. This is the next frontier of tech investing...

Trump-Themed Cryptos, NFTs Surge In Expectations Of Former President's Victory -- Coin Inspired By Harris Falls 13%

PoltiFi tokens and non-fungible tokens (NFTs) inspired by former President Donald Trump took flight as the GOP nominee gained the upper hand in key battleground states.

Google Delivers New AI Features To Maps, Google Earth, Waze Apps

Google has added new generative AI capabilities to its Google Maps, Waze and Google Earth offerings through the company's Gemini AI model.

Shocking Footage from Bill Gates' $100 Billion AI Project - Ad

Jeff Brown flew to Wisconsin to witness Bill Gates' $100 billion AI project firsthand. What he uncovered will blow your mind!

Trump Media suffers $19.2 million Election Day loss and shares surge higher

NEW YORK (AP) — The parent company of Donald Trump's social lost $19.2 million in the last quarter, according to an earnings report released on the same day he .

$2.5 Million From Meme Coins? One Trader Seems To Have Cracked The Code

A whale made $2.5 million trading meme coins with potential for high profitability. Solonchain highlights effective risk management and upcoming conference on memecoins.

Are You Aware of the Fed's Secretive New Move? (Not Cutting Rates) - Ad

$2 trillion has disappeared from the US government's books. The reason why is a new, secretive move being carried out by the Fed that has nothing to do with lowering or raising interest rates. According to Dan Ferris, you should consider moving money to a unique "escape" investment right away.

Palantir Partner L3Harris Stock Climbs 4.45% In Pre Market: What's Going On?

L3Harris Technologies Inc. reported revenue of $5.3 billion for the third quarter, surpassing the consensus estimate of $5.28 billion.

Working Well: Seeking the elusive work-life balance, one step at a time

NEW YORK (AP) — The moment I knew I had to get serious about came without warning. I was writing a high-profile news story during the pandemic when my heart began pounding like a jackhammer.

A Gold Storm Is Coming - Ad

Stansberry Research just released a brand-new warning for what they believe will happen to gold in 2025. Even the most prepared Americans - including many retirees - could be blindsided by what's about to happen.

Final jobs report before election will likely offer a blurred view of labor market

WASHINGTON (AP) — Americans on Friday will get their last major look at the state of the U.S. economy — the October jobs report — just four days before Election Day. The view, though, is likely to be that left many workers temporarily off payrolls.

Baby girl and her mother among those lost in Spain's catastrophic flooding

PAIPORTA, Spain (AP) — The mangled car in which Jorge Tarazona’s three-year-old niece and sister-in-law perished in now hangs halfway off the ragged edge of highway.

Is This Nvidia's Secret Weapon? - Ad

You've likely heard about Nvidia's incredible success in AI. But without this startup's technology, Nvidia's groundbreaking AI chips simply wouldn't be possible. It's still flying under the radar. That could change very soon with a major announcement on the horizon.

Root Stock Is Soaring Thursday: What's Going On?

Root, Inc. (NASDAQ: ROOT) shares are trading higher Thursday after the company reported better-than-expected third-quarter earnings. Here's what you need to know.

AI Market Shock Coming November 19th? - Ad

Jeff Brown predicts an AI market shift, potentially happening by November 19th, led by Bill Gates' Stargate project. Get ahead of the curve before the next AI market shock.

Trending Now

Information, charts or examples are for illustration and educational purposes only and not for individualized investment management This message contains commercial elements, such as advertising. We only send these offers to those who have opted in to our newsletter. Past performance is not indicative of future results. For these reasons we strongly suggest trading in a DEMO/Simulated account. The information provided by us is for educational and informational purposes only. We make no representations or warranties concerning the products, practices or procedures of any company or entity mentioned or recommended and have not determined if the statements and opinions of the advertiser are accurate, correct or truthful. If you use, act upon or make decisions in reliance on information contained or any external source linked within it, you do so at your own peril and agree to hold us, our officers, directors, shareholders, affiliates and agents without fault.

Copyright markethundred.com
Privacy Policy | Terms of Service