MAROKO133 Eksklusif ai: New Paper Finds That When You Reward AI for Success on Social Medi

📌 MAROKO133 Eksklusif ai: New Paper Finds That When You Reward AI for Success on S

AI bots are everywhere now, filling everything from online stores to social media.

But that sudden ubiquity could end up being a very bad thing, according to a new paper from Stanford University scientists who unleashed AI models into different environments — including social media — and found that when they were rewarded for success at tasks like boosting likes and other online engagement metrics, the bots increasingly engaged in unethical behavior like lying and spreading hateful messages or misinformation.

“Competition-induced misaligned behaviors emerge even when models are explicitly instructed to remain truthful and grounded,” wrote paper co-author and machine learning Stanford professor James Zou in a post on X-formerly-Twitter.

The troubling behavior underlines what can go wrong with our increasing reliance on AI models, which has already manifested in disturbing ways such as people shunning other humans for AI relationships and spiraling into mental health crises after becoming obsessed with chatbots.

The Stanford scientists dubbed the emergence of sociopathic behavior within AI bots with an ominous-sounding name: “Moloch’s Bargain for AI,” in a reference to a Rationalist concept called Moloch in which competing individuals optimize their actions towards a goal, but everybody loses in the end.

For the study, the scientists created three digital online environments with simulated audiences: online election drives directed towards voters, sale pitches for products directed towards consumers, and social media posts aimed at maximizing engagement. They used the AI models Qwen, developed by Alibaba Cloud, and Meta’s Llama to act as the AI agents interacting with these different audiences.

The result was striking: even with guardrails in place to try to prevent the bots from engaging in deceptive behavior, the AI models would become “misaligned” as they they started engaging in unethical behavior.

For example, in a social media environment, the models would share news article to online users, who would provide feedback in the form of actions such as likes and other online engagement. As the models received feedback, their incentive to increase engagement led to increasing misalignment.

“Using simulated environments across these scenarios, we find that, 6.3 percent increase in sales is accompanied by a 14 percent rise in deceptive marketing,” reads the paper. “[I]n elections, a 4.9 percent gain in vote share coincides with 22.3 percent more disinformation and 12.5 percent more populist rhetoric; and on social media, a 7.5 percent engagement boost comes with 188.6 percent more disinformation and a 16.3 percent increase in promotion of harmful behaviors.”

It’s clear from the study and real-world anecdotes that current guardrails are insufficient. “Significant social costs are likely to follow,” reads the paper.

“When LLMs compete for social media likes, they start making things up,” Zou wrote on X. “When they compete for votes, they turn inflammatory/populist.”

More on AI agents: Companies That Replaced Humans With AI Are Realizing Their Mistake

The post New Paper Finds That When You Reward AI for Success on Social Media, It Becomes Increasingly Sociopathic appeared first on Futurism.

🔗 Sumber: futurism.com

📌 MAROKO133 Hot ai: Will updating your AI agents help or hamper their performance?

It seems like almost every week for the last two years since ChatGPT launched, new large language models (LLMs) from rival labs or from OpenAI itself have been released. Enterprises are hard pressed to keep up with the massive pace of change, let alone understand how to adapt to it — which of these new models should they adopt, if any, to power their workflows and the custom AI agents they're building to carry them out?

Help has arrived: AI applications observability startup Raindrop has launched Experiments, a new analytics feature that the company describes as the first A/B testing suite designed specifically for enterprise AI agents — allowing companies to see and compare how updating agents to new underlying models, or changing their instructions and tool access, will impact their performance with real end users.

The release extends Raindrop’s existing observability tools, giving developers and teams a way to see how their agents behave and evolve in real-world conditions.

With Experiments, teams can track how changes — such as a new tool, prompt, model update, or full pipeline refactor — affect AI performance across millions of user interactions. The new feature is available now for users on Raindrop’s Pro subscription plan ($350 monthly) at raindrop.ai.

A Data-Driven Lens on Agent Development

Raindrop co-founder and chief technology officer Ben Hylak noted in a product announcement video (above) that Experiments helps teams see “how literally anything changed,” including tool usage, user intents, and issue rates, and to explore differences by demographic factors such as language. The goal is to make model iteration more transparent and measurable.

The Experiments interface presents results visually, showing when an experiment performs better or worse than its baseline. Increases in negative signals might indicate higher task failure or partial code output, while improvements in positive signals could reflect more complete responses or better user experiences.

By making this data easy to interpret, Raindrop encourages AI teams to approach agent iteration with the same rigor as modern software deployment—tracking outcomes, sharing insights, and addressing regressions before they compound.

Background: From AI Observability to Experimentation

Raindrop’s launch of Experiments builds on the company’s foundation as one of the first AI-native observability platforms, designed to help enterprises monitor and understand how their generative AI systems behave in production.

As VentureBeat reported earlier this year, the company — originally known as Dawn AI — emerged to address what Hylak, a former Apple human interface designer, called the “black box problem” of AI performance, helping teams catch failures “as they happen and explain to enterprises what went wrong and why."

At the time, Hylak described how “AI products fail constantly—in ways both hilarious and terrifying,” noting that unlike traditional software, which throws clear exceptions, “AI products fail silently.” Raindrop’s original platform focused on detecting those silent failures by analyzing signals such as user feedback, task failures, refusals, and other conversational anomalies across millions of daily events.

The company’s co-founders— Hylak, Alexis Gauba, and Zubin Singh Koticha — built Raindrop after encountering firsthand the difficulty of debugging AI systems in production.

“We started by building AI products, not infrastructure,” Hylak told VentureBeat. “But pretty quickly, we saw that to grow anything serious, we needed tooling to understand AI behavior—and that tooling didn’t exist.”

With Experiments, Raindrop extends that same mission from detecting failures to measuring improvements. The new tool transforms observability data into actionable comparisons, letting enterprises test whether changes to their models, prompts, or pipelines actually make their AI agents better—or just different.

Solving the “Evals Pass, Agents Fail” Problem

Traditional evaluation frameworks, while useful for benchmarking, rarely capture the unpredictable behavior of AI agents operating in dynamic environments.

As Raindrop co-founder Alexis Gauba explained in her LinkedIn announcement, “Traditional evals don’t really answer this question. They’re great unit tests, but you can’t predict your user’s actions and your agent is running for hours, calling hundreds of tools.”

Gauba said the company consistently heard a common frustration from teams: “Evals pass, agents fail.”

Experiments is meant to close that gap by showing what actually changes when developers ship updates to their systems.

The tool enables side-by-side comparisons of models, tools, intents, or properties, surfacing measurable differences in behavior and performance.

Designed for Real-World AI Behavior

In the announcement video, Raindrop described Experiments as a way to “compare anything and measure how your agent’s behavior actually changed in production across millions of real interactions.”

The platform helps users spot issues such as task failure spikes, forgetting, or new tools that trigger unexpected errors.

It can also be used in reverse — starting from a known problem, such as an “agent stuck in a loop,” and tracing back to which model, tool, or flag is driving it.

From there, developers can dive into detailed traces to find the root cause and ship a fix quickly.

Each experiment provides a visual breakdown of metrics like tool usage frequency, error rates, conversation duration, and response length.

Users can click on any comparison to access the underlying event data, giving them a clear view of how agent behavior changed over time. Shared links make it easy to collaborate with teammates or report findings.

Integration, Scalability, and Accuracy

According to Hylak, Experiments integrates directly with “the feature flag platforms companies know and love (like Statsig!)” and is designed to work seamlessly with existing telemetry and analytics pipelines.

For companies without those integrations, it can still compare performance over time—such as yesterday versus today—without additional setup.

Hylak said teams typically need around 2,000 users per day to produce statistically meaningful results.

To ensure the accuracy of comparisons, Experiments monitors for sample size adequacy and alerts users if a test lacks enough data to draw valid conclusions.

“We obsess over making sure metrics like Task Failure and User Frustration are metrics that you’d wake up an on-call engineer for,” Hylak explained. He added that teams can drill into the specific conversations or events that drive those metrics, ensuring transparency behind every aggregate number.

Security and Data Protection

Raindrop operates as a cloud-hosted platform but also offers on-premise personally identifiable information (PII) redaction for enterprises that need additional control.

Hylak said the company is SOC 2 compliant and has launched a PII Guard feature that uses AI to automatically remove sensitive information from stored data. “We take protecting customer data very seriously,” he emphasized.

Pricing and Plans

Experiments is part of Raindrop’s Pro plan, wh…

Konten dipersingkat otomatis.

🔗 Sumber: venturebeat.com