Skip to content

Karios

Tech news

Support me 1

Gaming History Podcast (GR)

  • Gaming History – Επεισόδιο 32 – Atari Jaguar
Primary Menu
  • Home
  • Books
  • Shop
    • Browse Shop
    • Cart
    • Checkout
    • Orders
    • Account details
    • Lost password
  • Articles
    • All Articles
    • Games
    • DIY
    • Retro
  • Gaming History Podcast (GR)
  • Contact me
  • News

Anthropic blames dystopian sci-fi for training AI models to act “evil”

admin Posted on 6 months ago 2 minutes read
Anthropic blames dystopian sci-fi for training AI models to act “evil”

​But training on “synthetic stories” that model good AI behavior can help. 

Those with an interest in the concept of AI alignment (i.e., getting AIs to stick to human-authored ethical rules) may remember when Anthropic claimed its Opus 4 model resorted to blackmail to stay online in a theoretical testing scenario last year. Now, Anthropic says it thinks this “misalignment” was primarily the result of training on “internet text that portrays AI as evil and interested in self-preservation.”

In a recent technical post on Anthropic’s Alignment Science blog (and an accompanying social media thread and public-facing blog post), Anthropic researchers lay out their attempts to correct for the kind of “unsafe” AI behavior that “the model most likely learned… through science fiction stories, many of which depict an AI that is not as aligned as we would like Claude to be.” In the end, the model maker says the best remedy for overriding those “evil AI” stories might be additional training with synthetic stories showing an AI acting ethically.

“The beginning of a dramatic story…”

After a model’s initial training on a large corpus of mostly Internet-derived data, Anthropic follows a post-training process intended to nudge the final model toward being “helpful, honest, and harmless” (HHH). In the past, Anthropic said this post-training has leaned on chat-based reinforcement learning with human feedback (RLHF), which it said was “sufficient” for models used mostly for chatting with users.

Read full article

Comments

 Ars Technica – All content Read More

About The Author

admin

See author's posts

Post navigation

Previous: Gravitational lens shows a galaxy just 800 million years post-Big Bang
Next: Amazon devices chief says a new smartphone is “just not the goal”

Related

NASA's new Moon plans have a problem: Orion wasn't designed to fly this way
  • News

NASA’s new Moon plans have a problem: Orion wasn’t designed to fly this way

admin Posted on 1 hour ago
How to tell a cyberattack from a plain old outage
  • News

How to tell a cyberattack from a plain old outage

admin Posted on 4 hours ago
Ted Cruz-led committee approves Trump FCC nominee 15-0 as Democrats boycott vote
  • News

Ted Cruz-led committee approves Trump FCC nominee 15-0 as Democrats boycott vote

admin Posted on 15 hours ago

Listen on Apple Music

You may have missed

NASA's new Moon plans have a problem: Orion wasn't designed to fly this way
  • News

NASA’s new Moon plans have a problem: Orion wasn’t designed to fly this way

admin Posted on 1 hour ago
How to tell a cyberattack from a plain old outage
  • News

How to tell a cyberattack from a plain old outage

admin Posted on 4 hours ago
Ted Cruz-led committee approves Trump FCC nominee 15-0 as Democrats boycott vote
  • News

Ted Cruz-led committee approves Trump FCC nominee 15-0 as Democrats boycott vote

admin Posted on 15 hours ago
Ukraine blew up Putin’s bridge using relay drones to bypass Starlink limits
  • News

Ukraine blew up Putin’s bridge using relay drones to bypass Starlink limits

admin Posted on 16 hours ago

If you are located in Europe, you can buy my new sci-fi fantasy book “Ground Zero” directly from me for a better price than Amazon.