Anthropic: Clio – Privacy-Preserving Insights into Real-World AI Use
Summary of Read Full Report (PDF)
The paper introduces Clio, a privacy-preserving system using AI to analyze aggregated data from millions of AI assistant conversations. Clio identifies usage patterns, revealing common tasks and cross-cultural differences, without human review of individual conversations.
The system also enhances AI safety by detecting coordinated misuse and improving safety classifiers. The authors discuss Clio's limitations and ethical considerations, emphasizing its potential for pro-social applications and the importance of empirical transparency in AI governance.
They validate Clio's accuracy and privacy through extensive evaluations using both synthetic and real-world data.