AI-detection tools have flagged Pope Leo XIV’s social media posts as AI-generated, as reported by Pangram Labs. This detection was made using their model, which claims an accuracy rate of 99.98% and a false positive rate of just 1 in 10,000. Despite the sophistication of these tools, such detections remain contentious within the industry.
Pangram Labs’ detection model is stated to claim an accuracy of 99.98% and a false positive rate of 1 in 10,000. A December 2025 University of Chicago study ranked Pangram highest among the AI-detection tools it tested and reported a false positive rate for Pangram that was close to zero. These numerical claims and the study ranking are presented as assessments specific to Pangram’s model and are reported here without extrapolation to other detectors or broader performance contexts. This summary confines itself to the claims and the study finding as reported for Pangram and does not provide methodological analysis, validation beyond the cited study, or interpretive conclusions.
Pangram Labs’ browser extension flagged posts from Pope Leo XIV’s X account as AI-generated. In those detections, the extension identified the Pope’s warnings about AI’s effect on human thought and social structures as AI-written. The extension likewise flagged posts from blue-check influencers on X and writings published on Substack as AI-generated. The tool also flagged a post by Tim Cook commemorating Apple’s 50th anniversary as AI-generated.
These reported instances were covered in media reports that attributed the detections to Pangram’s extension and were reported by Decrypt citing Wired in coverage.
The utilization of AI detection tools has stirred controversy, mainly due to concerns about false positives. A notable example is when ZeroGPT labeled the U.S. Declaration of Independence as 97.93% AI-generated according to Christopher Penn. This highlights a potential flaw in AI detectors, emphasizing imperfections in their assessments.
A quote from industry experts notes that “Given that every detector is at least slightly imperfect, organizations still have to evaluate for themselves if and how to use them, trading off the potential for AI misuse with the risk of false accusations.”
These inaccuracies can lead to high-risk scenarios, such as academic institutions using AI detections to disqualify students, placing them on probation or suspension. Trust Insight reports also underscore broader concerns within the industry, as false positives challenge the credibility and reliability of AI detection tools, calling for careful evaluation in their deployment to avoid unjust outcomes.
Research conducted by Stanford University, Imperial College London, and the Internet Archive indicates that by mid-2025, AI-generated or AI-assisted texts accounted for approximately 35% of newly published websites. Additionally, Pangram Labs’ detection tools flagged posts from Pope Leo XIV, which included his warnings about AI’s impact on human thought and social structures, as being AI-generated. These findings were highlighted in a report sourced from Decrypt, citing Wired. This highlights the increasing prevalence and detection of AI-generated content, fueling ongoing discussions and considerations regarding the methodologies and implications of AI detection technologies.
