FORGET Crypto, FORGET Stocks... Do THIS Instead...

With everyone scrambling for the next hot AI stock or cryptocurrency... One man is saying FORGET them all. Because his strategy could lead to bigger gains than you'd see from stocks, crypto or anything else you might be seeing. It's shown his readers gains of 85% in 14 days... 55% gains in 2 days... And even 222% in 8 days!

AI 'gold rush' for chatbot training data could run out of human-written text

MATT O'BRIEN
June 06, 2024

Artificial intelligence systems like ChatGPT could soon run out of what keeps making them smarter -- the tens of trillions of words people have written and shared online.

A new study released Thursday by research group Epoch AI projects that tech companies will exhaust the supply of publicly available training data for AI language models by roughly the turn of the decade -- sometime between 2026 and 2032.

Comparing it to a "literal gold rush" that depletes finite natural resources, Tamay Besiroglu, an author of the study, said the AI field might face challenges in maintaining its current pace of progress once it drains the reserves of human-generated writing.

In the short term, tech companies like ChatGPT-maker OpenAI and Google are racing to secure and sometimes pay for high-quality data sources to train their AI large language models - for instance, by signing deals to tap into the steady flow of sentences coming out of Reddit forums and news media outlets.

In the longer term, there won't be enough new blogs, news articles and social media commentary to sustain the current trajectory of AI development, putting pressure on companies to tap into sensitive data now considered private -- such as emails or text messages -- or relying on less-reliable "synthetic data" spit out by the chatbots themselves.

"There is a serious bottleneck here," Besiroglu said. "If you start hitting those constraints about how much data you have, then you can't really scale up your models efficiently anymore. And scaling up models has been probably the most important way of expanding their capabilities and improving the quality of their output."

The researchers first made their projections two years ago -- shortly before ChatGPT's debut -- in a working paper that forecast a more imminent 2026 cutoff of high-quality text data. Much has changed since then, including new techniques that enabled AI researchers to make better use of the data they already have and sometimes "overtrain" on the same sources multiple times.

But there are limits, and after further research, Epoch now foresees running out of public text data sometime in the next two to eight years.

The team's latest study is peer-reviewed and due to be presented at this summer's International Conference on Machine Learning in Vienna, Austria. Epoch is a nonprofit institute hosted by San Francisco-based Rethink Priorities and funded by proponents of effective altruism -- a philanthropic movement that has poured money into mitigating AI's worst-case risks.

Besiroglu said AI researchers realized more than a decade ago that aggressively expanding two key ingredients -- computing power and vast stores of internet data -- could significantly improve the performance of AI systems.

The amount of text data fed into AI language models has been growing about 2.5 times per year, while computing has grown about 4 times per year, according to the Epoch study. Facebook parent company Meta Platforms recently claimed the largest version of their upcoming Llama 3 model -- which has not yet been released -- has been trained on up to 15 trillion tokens, each of which can represent a piece of a word.

But how much it's worth worrying about the data bottleneck is debatable.

"I think it's important to keep in mind that we don't necessarily need to train larger and larger models," said Nicolas Papernot, an assistant professor of computer engineering at the University of Toronto and researcher at the nonprofit Vector Institute for Artificial Intelligence.

Papernot, who was not involved in the Epoch study, said building more skilled AI systems can also come from training models that are more specialized for specific tasks. But he has concerns about training generative AI systems on the same outputs they're producing, leading to degraded performance known as "model collapse."

Training on AI-generated data is "like what happens when you photocopy a piece of paper and then you photocopy the photocopy. You lose some of the information," Papernot said. Not only that, but Papernot's research has also found it can further encode the mistakes, bias and unfairness that's already baked into the information ecosystem.

If real human-crafted sentences remain a critical AI data source, those who are stewards of the most sought-after troves -- websites like Reddit and Wikipedia, as well as news and book publishers -- have been forced to think hard about how they're being used.

"Maybe you don't lop off the tops of every mountain," jokes Selena Deckelmann, chief product and technology officer at the Wikimedia Foundation, which runs Wikipedia. "It's an interesting problem right now that we're having natural resource conversations about human-created data. I shouldn't laugh about it, but I do find it kind of amazing."

While some have sought to close off their data from AI training -- often after it's already been taken without compensation -- Wikipedia has placed few restrictions on how AI companies use its volunteer-written entries. Still, Deckelmann said she hopes there continue to be incentives for people to keep contributing, especially as a flood of cheap and automatically generated "garbage content" starts polluting the internet.

AI companies should be "concerned about how human-generated content continues to exist and continues to be accessible," she said.

From the perspective of AI developers, Epoch's study says paying millions of humans to generate the text that AI models will need "is unlikely to be an economical way" to drive better technical performance.

As OpenAI begins work on training the next generation of its GPT large language models, CEO Sam Altman told the audience at a United Nations event last month that the company has already experimented with "generating lots of synthetic data" for training.

"I think what you need is high-quality data. There is low-quality synthetic data. There's low-quality human data," Altman said. But he also expressed reservations about relying too heavily on synthetic data over other technical methods to improve AI models.

"There'd be something very strange if the best way to train a model was to just generate, like, a quadrillion tokens of synthetic data and feed that back in," Altman said. "Somehow that seems inefficient."

------------

The Associated Press and OpenAI have a licensing and technology agreement that allows OpenAI access to part of AP's text archives.

Continue Reading...

Popular

NATO is deploying eyes in the sky and on the Baltic Sea to protect vital cables. Here's why and how

ABOARD A FRENCH NAVY FLIGHT OVER THE BALTIC SEA (AP) — With , the French Navy surveillance plane scouring zoomed in on a cargo ship plowing the waters below — closer, closer and closer still until the camera operator could make out details on the vessel's front deck and smoke pouring from its chimney.

Nvidia faces a reckoning as an upstart rival raises questions about Wall Street's darling

NEW YORK (AP) — The superstar run for Nvidia’s stock the last few years has been astonishing. So was its tumble Monday, which caused $595 billion in wealth to vanish. That’s about as much as PepsiCo, McDonalds, Starbucks and Target are worth, combined.

Crypto Stocks Are Heating Up - Ad

At the center of this transformation is a game-changing platform that's experiencing explosive growth with $41 million in revenue in just nine months. The stock has doubled in the last 45 days, but this is only the beginning.

Dubai's ceaseless boom is putting strains on its residents

DUBAI, United Arab Emirates (AP) — — and some residents are starting to feel burned.

Hawaii wildfire victims spared from testifying after last-minute deal over $4B settlement

HONOLULU (AP) — Lawyers representing victims of a deadly Hawaii wildfire reached a last-minute deal averting a trial Wednesday to determine how to split a .

Robotics Meets Public Safety Innovation - Ad

Advanced security robots, real-world impact, and a valuation gap too big to ignore. This robotics stock is one to watch.

A South Florida luxury condo project is planned for site where building collapse killed 98 people

SURFSIDE, Fla. (AP) — A Dubai-based developer plans to build a 12-story luxury condominium project on the South Florida site where in 2021, killing 98 people.

Elon Musk Is 'Kind Of Glad' Many Didn't Pick Up A Tesla FSD Subscription: Here's Why

Tesla Inc CEO Elon Musk said on Wednesday that the company will likely have to update hardware for customers whose vehicles are equipped with an older version of the AI hardware called Hardware 3, for it to achieve fully unsupervised autonomous driving with full self-driving (FSD) driver assistance software.

Top-Secret Military Base in New Mexico Desert - Ad

Out in the New Mexico desert sits a top secret government facility... The scientists here are studying a new technology that will change the way wars are fought... Once you see this thing for yourself -- you'll know in your heart it could be coming for all of us. That's why I've taken steps to help protect myself and my family from what could be unleashed as early as April 11.

Stock market today: Asian stocks advance ahead of the Fed's rate decision as panic over AI fades

HONG KONG (AP) — Asian stocks advanced Wednesday in thin Lunar New Year trading following a rebound on Wall Street driven by tech stocks as the panic over Chinese AI company DeepSeek faded.

Japanese automaker Nissan says it plans job and production cuts in the U.S.

TOKYO (AP) — Nissan is slashing production at its U.S. plants and offering buyouts to factory workers there as part of the Japanese automaker’s urgent efforts to return to profitability.

Elon's Next Millionaire-Making Project? - Ad

This could create more wealth than all of his previous ventures... combined.

NASA's 2 stuck astronauts take their first spacewalk together

CAPE CANAVERAL, Fla. (AP) — NASA’s took their first spacewalk together Thursday, exiting the International Space Station almost eight months after moving in.

6 Clean Energy Stocks Face Challenges, Opportunities Ahead Of Earnings

Analyst revises ratings/price forecast on clean energy stocks ahead of earnings release. Highlights key areas to watch for each company.

5-Second Trump Clip will Give You Chills - Ad

If you have any money in the markets, take a moment to see this presentation with a 5-second clip of a shocking prediction President Trump made during his victory speech... Because according to legendary investor Louis Navellier, who correctly predicted Trump's win... This prediction is about to become a reality.

Kansas City Battles Major Tuberculosis Outbreak, 67 Active Cases, Hundreds Under Watch

Kansas City faces one of the worst U.S. tuberculosis outbreaks, with 67 active cases and 384 people under monitoring.

Trump White House Puts DeepSeek Under National Security Scanner: 'Wake-Up Call To The American AI Industry'

The White House is reviewing China's DeepSeek AI for potential national security risks, amid growing concerns over intellectual property theft and its impact on U.S. AI dominance.

Crypto Stocks Are Heating Up - Ad

At the center of this transformation is a game-changing platform that's experiencing explosive growth with $41 million in revenue in just nine months. The stock has doubled in the last 45 days, but this is only the beginning.

Stryker's Strategic Moves Impress Analysts, Spine Implant Sale and NARI Deal Set to Drive Future Growth

Wall Street analysts upgraded Stryker Corp (NYSE: SYK) after its Q4 report beat expectations, with strong sales and earnings growth projected for 2025.

Dan Ives Expects Meta And Microsoft To Stand Firm On $60 - $80 Billion AI Spending Plans Amid DeepSeek's Disruptive AI Model

Wedbush Securities analyst Dan Ives expects Meta Platforms Inc. and Microsoft Corp. to stand firm on their ambitious artificial intelligence spending plans during their upcoming earnings calls, despite recent market jitters over Chinese AI startup DeepSeek's emergence.

Elon Musk: "AI Will Run Out of Electricity ... in 2025" - Ad

This AI boom is going to push America's power grid to the brink... Elon Musk even recently said that AI could run out of electricity by 2025. This crisis may just reveal one of the greatest investment ideas we are ever going to find. My research has recently uncovered two companies that could make a massive difference today.

Gary Black Calls Tesla's Austin Robotaxi Launch By June 'Most Bullish' Takeaway, But Wants Answers On Auto Margins And Next-Gen EV

Tesla Inc. bull Gary Black highlighted the company's planned autonomous ride-hailing service in Austin as a significant development while expressing concerns about automotive margins and the forthcoming affordable electric vehicle's design.

MrBeast And Group Of Investors, Including Roblox CEO, Lock In Over $20 Billion For TikTok Takeover

A group of American investors, including YouTube star MrBeast and Roblox CEO David Baszucki, has secured over $20 billion for a potential TikTok takeover, but ByteDance has yet to respond to their bid.

The $20 Stock Powering NVIDIA, TESLA, and Microsoft - Ad

The biggest AI firms in the world... All rely on this single company. And right now, you can get in for only $20 - but not for long.

Bank of America's Sensitive Customer Data Compromised in Third-Party Hack - Is Your Account Safe?

Bank of America (NYSE: BAC) has confirmed a data breach involving a third-party software provider that led to the exposure of sensitive customer data.

Judge Blocks Trump's Order To Freeze Federal Grant Money

A federal judge in the District of Columbia on Tuesday afternoon temporarily blocked an executive order given by President Donald Trump that imposed a freeze of all federal grants and loans, putting $3 trillion in funding in jeopardy and entire industries on edge. 

Seven Unknown AI Stocks That Could Dominate the Next Six Years - Ad

The original "Magnificent Seven" stocks generated 16,800% over the last 20 years. But now a new set of AI stocks is set to take over. Alex Green dubs them "The Next Magnificent Seven." And he's arguing that just $1,000 in each could turn into more than $1 million in less than six years.

Despite chaos over Trump White House's funding pause, FAFSA forms and student loans still available

A temporary freeze imposed briefly this week by the White House on federal grants and loans left many students wondering about the impact to the used to apply for financial aid.

Robert Kiyosaki Foresees Bitcoin Surpassing US Dollar As 'Good Money'

Robert Kiyosaki, the acclaimed author of “Rich Dad Poor Dad,” has reiterated his preference for Bitcoin (CRYPTO: BTC) over

This Coin Could Surge Like Bitcoin Did Back in 2013... - Ad

A new coin is emerging in the crypto world. And investing in it now could end up like Bitcoin or Ethereum during their first bull runs.

Ozempic Receives FDA Approval To Treat Kidney Disease

The FDA on Tuesday approves Ozempic, the weight loss drug made by Novo Nordisk, for the treatment of chronic kidney disease in patients who also have type 2 diabetes. 

Alex's Mystery Guest Makes Shocking Trump Revelation - Ad

Over the next 4 years, Trump is about to create something extraordinary. A real chance for everyday Americans to build serious wealth... wealth that would dwarf what we saw in his first term. We're talking about a potential economic transformation that could mint 20 million new millionaires. Put everything else aside and watch this special event right now.

Europe's economy showed zero growth at end of 2024 as Germany, eurozone's biggest economy, struggled

FRANKFURT, Germany (AP) — Europe’s economy stagnated at the end of last year as its former growth engine, Germany, finished a second straight year of shrinking output, officials said Thursday.

Is This Defi Coin Your Next "10-Bagger" Investment? - Ad

Our #1 pick in decentralized finance is largely overlooked by mainstream investors (for now). This governance token could skyrocket as DeFi adoption surges with the smart money and institutions.

UK Faces Triple Threat as Economic Uncertainty Hits

The United Kingdom faces a decline in stock market listings, demographic crisis, and diminishing tax revenue with an exodus of high earners.

Crypto Stocks Surge Amid Market Boom - Ad

Crypto stocks are on fire as Bitcoin hits all-time highs. One under-the-radar company is leading the charge, managing $2.1 billion in client assets and seeing record trading volumes. With crypto adoption skyrocketing, this could be a game-changing opportunity.

Trending Now

Information, charts or examples are for illustration and educational purposes only and not for individualized investment management This message contains commercial elements, such as advertising. We only send these offers to those who have opted in to our newsletter. Past performance is not indicative of future results. For these reasons we strongly suggest trading in a DEMO/Simulated account. The information provided by us is for educational and informational purposes only. We make no representations or warranties concerning the products, practices or procedures of any company or entity mentioned or recommended and have not determined if the statements and opinions of the advertiser are accurate, correct or truthful. If you use, act upon or make decisions in reliance on information contained or any external source linked within it, you do so at your own peril and agree to hold us, our officers, directors, shareholders, affiliates and agents without fault.

Copyright markethundred.com
Privacy Policy | Terms of Service