Microsoft: The Impact of Generative AI on Critical Thinking – Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers

Summary of Read Full Report (PDF)

This research paper examines the effects of generative AI tools on the critical thinking skills of knowledge workers. A survey of 319 knowledge workers, analyzing 936 real-world examples of GenAI use, reveals that while GenAI reduces perceived cognitive effort, it can also decrease critical engagement and potentially lead to over-reliance.

The study identifies factors influencing critical thinking, such as user confidence in both themselves and the AI, and explores how GenAI shifts the nature of critical thinking in knowledge work tasks. The findings highlight design challenges and opportunities for creating GenAI tools that better support critical thinking.

Here are 5 key takeaways from the provided research on the impact of generative AI (GenAI) on critical thinking among knowledge workers:

• GenAI can reduce the effort of critical thinking, but also engagement. While GenAI tools can automate tasks and make information more readily available, this may lead to users becoming over-reliant on AI and reducing their own critical thinking and problem-solving skills.

• Confidence in AI negatively correlates with critical thinking, while self-confidence has the opposite effect. The study found that when users have higher confidence in AI's ability to perform a task, they tend to engage in less critical thinking. Conversely, those who have more confidence in their own skills are more likely to engage in critical thinking, even if it requires more effort.

• Critical thinking with GenAI shifts from task execution to task oversight. Knowledge workers using GenAI shift their focus from directly producing material to overseeing the AI's work. This includes verifying information, integrating AI responses, and ensuring the output meets quality standards.

• Motivators for critical thinking include work quality, avoiding negative outcomes, and skill development. Knowledge workers are motivated to think critically when they want to improve the quality of their work, avoid errors or negative consequences, and develop their own skills.

• Barriers to critical thinking include lack of awareness, motivation, and ability. Users may not engage in critical thinking due to a lack of awareness of the need for it, limited motivation due to time pressure or job scope, or because they find it difficult to improve AI responses. Also, some users may consider critical thinking unnecessary when using AI for secondary or trivial tasks, or overestimate AI capabilities.