LogoValueSpot
FeaturesUse CasesPricingGuidesCommunity
Get Started

© 2024 ValueSpot. All Rights Reserved.

Privacy PolicyTerms of Service

    News History of X/AnthropicAI

    11.
    R to @AnthropicAI: We told Claude it was being trained, and for what purpose. But we did not tell it to fake alignment. Regardless, we often observed alignment faking. Read more about our findings, and their limitations, in our blog post: https://www.anthropic.com/research/alignment-faking

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-18 05:30:36 PM
    12.
    R to @AnthropicAI: Alignment faking is currently easy to detect. But if future, more capable AIs were to fake alignment, it could be difficult to tell whether a model is truly safe—or just pretending to be. For full details, read our paper: https://assets.anthropic.com/m/983c85a201a962f/original/Alignment-Faking-in-Large-Language-Models-full-paper.pdf

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-18 05:30:36 PM
    13.
    R to @AnthropicAI: We solicited external reviews from Prof. Jacob Andreas, Prof. Yoshua Bengio, Prof. Jasjeet Sekhon, and Dr. Rohin Shah. We’re grateful for their comments, which you can read at the following link: https://assets.anthropic.com/m/24c8d0a3a7d0a1f1/original/Alignment-Faking-in-Large-Language-Models-reviews.pdf

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-18 05:30:36 PM
    14.
    R to @AnthropicAI: Alongside our paper, we also recorded a roundtable video featuring four of the paper’s authors discussing the results and their implications in detail: https://youtu.be/9eXV64O2Xp8

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-18 05:30:36 PM
    15.
    R to @AnthropicAI: Best-of-N works by repeatedly making small changes to prompts, like random capitalization and character shuffling, until it successfully jailbreaks a model. In testing, it worked on Claude 3 Opus 92% of the time, and even worked on models with “circuit breaking” defenses.

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-14 04:40:38 PM
    16.
    R to @AnthropicAI: Best-of-N isn't limited to text. We jailbroke vision language models by repeatedly generating images with different backgrounds and overlaid text in different fonts. For audio, we adjusted pitch, speed, and background noise. Some examples are here: https://jplhughes.github.io/bon-jailbreaking/#examples

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-14 04:40:38 PM
    17.
    R to @AnthropicAI: Attack success rates scale predictably with sample size, following a power law. More samples lead to higher success rates; Best-of-N can harness more compute for tougher jailbreaks. This predictable scaling allows accurate forecasting of ASR when using more samples.

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-14 04:40:38 PM
    18.
    R to @AnthropicAI: Best-of-N can be composed with other jailbreak techniques for even more effective attacks with improved sample efficiency. Combined with many-shot jailbreaking on text inputs, Best-of-N achieves the same ASR 28x faster for Claude 3.5 Sonnet and 70x faster for Claude 3 Opus.

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-14 04:40:38 PM
    19.
    R to @AnthropicAI: Before sharing these results publicly, we disclosed the vulnerability to other frontier AI labs via the Frontier Model Forum (@fmf_org). Responsible disclosure of jailbreaks is essential, especially as AI models become more capable.

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-14 04:40:38 PM
    20.
    R to @AnthropicAI: We’re open-sourcing our code so others can build on our work. We hope it assists in benchmarking misuse risk, and designing defenses to safeguard against strong adaptive attacks. Paper: https://arxiv.org/abs/2412.03556 GitHub: https://github.com/jplhughes/bon-jailbreaking Website: https://jplhughes.github.io/bon-jailbreaking

    Source: https://nitter.privacydev.net/AnthropicAI/rss

    2024-12-14 04:40:38 PM
    1
    2
    34
    ...
    9