Stories about Cross-Session Decomposition Attacks
1 related stories
Cross-Session Decomposition Attacks: Scaling Risk and Intent-Aligned Retrieval Defense
AI InsightThis work formalizes cross-session decomposition attacks as compositional safety risk and proves a conditional risk-transfer bound: the model's excess loss on allowed subqueries controls the gap between deployed and reference composed risk. Unlike prior focus on single-session jailbreaks, it reveals another side of scaling laws—wider transformers assign lower loss to held-out subqueries, potentially making decomposition attacks more stealthy.Key TakeawaySafety risk shifts from single-session to cross-session composition, with a theoretical risk bound.Why It MattersFirst theoretical link between scaling laws and compositional safety risk; safety evaluation must extend from single interactions to cross-session sequences.Who's Affected- AI ResearchersNeed to validate risk-transfer bounds across model scales.
- Model DevelopersNeed intent-aligned retrieval defense, not just stronger single-turn blocking.
- Cybersecurity PractitionersNew attack surface via cross-session decomposition; monitoring must cover multi-turn patterns.
- RegulatorsModel safety evaluation may need to account for cumulative cross-session risk.
What's NextWatch for follow-up work proposing deployable intent-aligned retrieval defenses and empirical attack reproductions.Importance 68/100