安全与密码学2024arxiv.org
ArtPrompt：针对对齐 LLM 的 ASCII 艺术越狱攻击
ArtPrompt: ASCII Art-Based Jailbreak Attacks Against Aligned LLMs
进阶 · 约 45 分钟读完前置知识：LLM 基础、对齐与越狱概念、提示注入基础
导读
这篇论文提出 ArtPrompt，一种利用 ASCII 艺术字混淆提示中敏感词的越狱攻击方法，用来测试对齐后的 LLM 是否会绕过安全限制。它关注模型能否识别非语义形式的提示，并展示 ASCII 艺术可以成为攻击面。具体实验设置和防御效果以原文为准。
HN 关注点：LLM 安全对齐的脆弱性，以及这类越狱是否只是在重复已知的指令/数据分离架构问题。
HN 评论精选
这几条评论里：质疑 2 补充 1
质疑批评此类攻击论文只是列举症状，根因是指令与数据未分离的架构缺陷，公共讨论被误导。
我怀疑是否每种技术被绕过的方式都值得单独写论文。问题已知，是尚未解决的架构缺陷：我们依赖模型内部能力区分数据和指令，模型越强，混淆它的方式越多。指令与数据没有清晰分离，是许多计算机安全难题的根源，从 SQL 注入到 x86 把数据和代码视作可互换。自回归 Transformer 在当前认知下可能解决不了，需要分离输入和非图灵完备的指令语言。但真正的问题是公共讨论被这类细节、文化战争和重复内容消耗，大家只谈症状，不懂技术的人写监管时被误导。
gmerc查看英文原文
I wonder if we really need to have a paper for every way the technology can be subverted. We know what the problem is and we know it's an architecture shortcoming we have not solved yet. Generalized: "We rely on a model's internal capabilities to separate data from instructions. The more powerful the model, the more ways exist to confuse the process'. Not having a clear separation of instruction and data is the root cause for a fair share of computer security challenges we struggle with. From little bobby tables all the way to x86 architecture treating data and code as interchangeable (nevermind NX, other attempts at solving this later). Autoregressive transformers likely are not capable of addressing this issue with our current knowledge. We need separate inputs and a non turing complete instruction language to address it. We don't know how to get there yet. But none of this is the actual issue. The issue is that the entire public conversation is consumed by the bullshit details like this at the moment, the culture war is trying to get it's share too and everyone is recycling the same vomit over and over to drive engagement. Everyone is talking symptoms and projecting their hopes and fears into it and much less technically savy people writing regulation, etc are led astray about what the fundamental challenges are . It's all PR posturing. It's not about security or safety. It's stupid We discovered technology. It has limitations. We know what the problem is. We know what causes it. It has nothing to do with safety. We don't know yet how to fix it. We need to meet investor expections, so we create an entirely new level of Security Theatre that's a total diversion from the actual problem. We drown the world in a cesspool of information waste. We don't know how to fix it yet
补充缺乏 prompt 与注入数据的隔离，使基于 LLM 的产品难以可靠控制输出。
有意思也有点令人担忧：LLM 如此难以被控制不去做你不想让它做的事。我不喜欢 LLM 审查内容，但如果用 LLM 做产品（不是聊天服务），我希望完全控制它可能输出什么。没有类似 SQL 预处理语句的机制，也不区分提示和注入数据，这让控制很难。
matsemann查看英文原文
It's interesting, and a bit concerning, that it's so hard to control LLMs from doing things you don't want it to do. Sure, I don't like LLMs censoring stuff. But if I were to build a product using LLMs (aka not a chat service), I'd like to have full control of what it can potentially output. The fact that there is no "prepared statements" or distinction between prompts and injected data makes that hard.
质疑质疑把“不能仅靠语义解释提示”视为模型缺陷，因为人类语言理解也不只依赖语义。
“识别不能仅靠语义解释的提示”——人类本来也不只靠语义解释语言，为什么在聊天机器人身上这就算缺陷？
DinaCoder99查看英文原文
"recognizing prompts that cannot be solely interpreted by semantics" Humans certainly don't interpret language solely by semantics—why is this considered a flaw in chatbots?
相关资源
从这篇论文所有 HN 讨论帖的全部评论里整理出来的链接。
历次讨论
- 2024Hacker News 讨论帖145 分 · 55 评
同领域还值得读
如何入侵一台已关机的电脑：在 Intel ME 中运行未签名代码
How to hack a turned-off computer, or running unsigned code in Intel ME
这篇 Black Hat 2017 白皮书讨论 Intel Management Engine（ME）的安全问题：如何在 ME 中运行未签名代码，以及攻破处于关机状态的电脑的相关…
安全与密码学2017 · 543 分 · 107 评专业 · 约 40 分钟编辑精选比特币：一种点对点的电子现金系统（2008）
Bitcoin: A Peer-to-Peer Electronic Cash System (2008)
中本聪 2008 年发表的比特币白皮书，提出了无需可信第三方的点对点电子现金系统。
安全与密码学2017 · 536 分 · 394 评入门 · 约 45 分钟编辑精选GPRS 加密算法 GEA-1 的密码分析表明存在故意弱点
Cryptanalysis of GPRS Encryption Algorithms GEA-1 suggest intentional weakness
首次公开了对 GPRS 加密算法 GEA-1 和 GEA-2 的密码分析攻击。
安全与密码学2021 · 487 分 · 107 评专业 · 约 90 分钟编辑精选Signal 消息协议的形式化安全分析
A Formal Security Analysis of the Signal Messaging Protocol
用形式化方法分析 Signal 消息协议（TextSecure）的安全性：作者在多阶段密钥交换模型下，借助自动化证明工具对协议的密钥协商与 Double Ratchet 密钥更新…
安全与密码学2016 · 353 分 · 230 评专业 · 约 120 分钟编辑精选A2：模拟恶意硬件
A2: Analog Malicious Hardware
论文研究如何在芯片设计阶段植入模拟电路形式的恶意硬件后门，使其难以被常规检测发现。
安全与密码学2016 · 315 分 · 94 评专业 · 约 75 分钟编辑精选