ibl.ai

Stanford University: Expanding Academia's Role in Public Sector AI

Summary of https://hai-production.s3.amazonaws.com/files/hai-issue-brief-expanding-academia-role-public-sector.pdf
Stanford HAI highlights a growing disparity between academia and industry in frontier AI research. Industry's access to vast resources like data and computing power allows them to outpace universities in developing advanced AI systems.
The authors argue that this imbalance risks hindering public-interest AI innovation and weakening the talent pipeline. To address this, the brief proposes increased public investment in academic AI, the adoption of collaborative research models, and the creation of new government-backed academic institutions. Ultimately, the aim is to ensure academia plays a vital role in shaping the future of AI in a way that benefits society.
Academia is currently lagging behind industry in frontier AI research because no university possesses the resources to build AI systems comparable to those in the private sector. This is largely due to industry's access to massive datasets and significantly greater computational power.
Industry's dominance in AI development is driven by its unprecedented computational resources, vast datasets, and top-tier talent, leading to AI models that are considerably larger than those produced by academia. This resource disparity has become a substantial barrier to entry for academic researchers.
For AI to be developed responsibly and in the public interest, it is crucial for governments to increase investment in public sector AI, with academia at the forefront of training future innovators and advancing cutting-edge scientific research. Historically, academia has been the source of foundational AI technologies and prioritizes public benefit over commercial gain.
The significant cost of developing advanced AI models has created a major divide between industry and academia. The expense of computational resources required for state-of-the-art models has grown exponentially, making it challenging for academics to meaningfully contribute to their development.
The growing resource gap in funding, computational power, and talent between academia and industry is concerning because it restricts independent, public-interest AI research, weakens the future talent pipeline by incentivizing students to join industry, and can skew AI policy discussions in favor of well-funded private sector interests.

University of Texas at Austin: Protecting Human Cognition in the Age of AI

Summary of https://arxiv.org/pdf/2502.12447
Explores the rapidly evolving influence of Generative AI on human cognition, examining its effects on how we think, learn, reason, and engage with information. Synthesizing existing research, the authors analyze these impacts through the lens of educational frameworks like Bloom's Taxonomy and Dewey's reflective thought theory.
The work identifies potential benefits and significant concerns, particularly regarding critical thinking and knowledge retention among novices. Ultimately, it proposes implications for educators and test designers and suggests future research directions to understand the long-term cognitive consequences of AI.
Generative AI (GenAI) is rapidly reshaping human cognition, influencing how we engage with information, think, reason, and learn. This adoption is happening at a much faster rate compared to previous technological advancements like the internet.
While GenAI offers potential benefits such as increased productivity, enhanced creativity, and improved learning experiences, there are significant concerns about its potential long-term detrimental effects on essential cognitive abilities, particularly critical thinking and reasoning. The paper primarily focuses on these negative impacts, especially on novices like students.
GenAI's impact on cognition can be understood through frameworks like Krathwohl’s revised Bloom’s Taxonomy and Dewey’s conceptualization of reflective thought. GenAI can accelerate access to knowledge but may bypass the cognitive processes necessary for deeper understanding and the development of metacognitive skills. It can also disrupt the prerequisites for reflective thought by diminishing cognitive dissonance, reinforcing existing beliefs, and creating an illusion of comprehensive understanding.
Over-reliance on GenAI can lead to 'cognitive offloading' and 'metacognitive laziness', where individuals delegate cognitive tasks to AI, reducing their own cognitive engagement and hindering the development of critical thinking and self-regulation. This is particularly concerning for novice learners who have less experience with diverse cognitive strategies.
To support thinking and learning in the AI era, there is a need to rethink educational experiences and design 'tools for thought' that foster critical and evaluative skills. This includes minimizing AI use in the early stages of learning to encourage productive struggle, emphasizing critical evaluation of AI outputs in curricula and tests, and promoting active engagement with GenAI tools through methods like integrating cognitive schemas and using metacognitive prompts. The paper also highlights the need for long-term research on the sustained cognitive effects of AI use.

University of Bristol: Alice in Wonderland – Simple Tasks Showing Complete Reasoning Breakdown in State-of-the-Art LLMs

Summary of https://arxiv.org/pdf/2406.02061
Introduces the "Alice in Wonderland" (AIW) problem, a seemingly simple common-sense reasoning task, to evaluate the capabilities of state-of-the-art Large Language Models (LLMs). The authors demonstrate that even advanced models like GPT-4 and Claude 3 Opus exhibit a dramatic breakdown in generalization and basic reasoning when faced with minor variations of the AIW problem that do not alter its core structure or difficulty.
This breakdown is characterized by low average performance and significant fluctuations in accuracy across these variations, alongside overconfident, yet incorrect, explanations. The study further reveals that standardized benchmarks fail to detect these limitations, suggesting a potential overestimation of current LLM reasoning abilities, possibly due to data contamination or insufficient challenge diversity.
Ultimately, the AIW problem is presented as a valuable tool for uncovering fundamental weaknesses in LLMs' generalization and reasoning skills that are not apparent in current evaluation methods.
Despite achieving high scores on various standardized benchmarks, many state-of-the-art Large Language Models (LLMs) exhibit surprisingly low correct response rates on the seemingly simple "Alice has brothers and sisters" (AIW) problem and its variations. Only a few large-scale closed models like GPT-4o and Claude 3 Opus show relatively better performance, while many others, including models claiming strong function, struggle significantly, sometimes even collapsing to a zero correct response rate.
The document highlights a significant discrepancy between the performance of LLMs on standardized reasoning benchmarks and on the AIW problem, suggesting that current benchmarks may not accurately reflect true generalization and basic reasoning skills. Models that score highly on benchmarks like MMLU, MATH, ARC-c, GSM8K, and HellaSwag often perform poorly on AIW, indicating a potential issue with the benchmarks' ability to detect fundamental deficits in model function. This suggests that these benchmarks might suffer from issues like test data leakage.
A key observation is the lack of robustness in SOTA LLMs, evidenced by strong performance fluctuations across structure and difficulty-preserving variations of the same AIW problem. Even slight changes in the numerical values within the problem statement can lead to drastically different correct response rates for many models. This sensitivity to minor variations points to underlying generalization deficits.
The study reveals that LLMs often exhibit overconfidence and provide persuasive, explanation-like confabulations even when their answers to AIW problems are incorrect. This can mislead users into trusting wrong responses, especially in situations where verification is difficult. Furthermore, many models struggle to properly detect mistakes and revise their incorrect solutions, even when encouraged to do so.
The AIW problem and its variations are presented as valuable tools for evaluating the robustness and generalization capabilities of LLMs, offering a method to reveal weaknesses that are not captured by standard benchmarks. The ability to create numerous diverse problem instances through variations addresses potential test set leakage issues. The introduction of a unified robustness score (R) is proposed to provide a more accurate model ranking by considering both average correct response rate and the degree of performance fluctuations across problem variations.

NIST: Adversarial Machine Learning – A Taxonomy and Terminology of Attacks and Mitigations

Summary of https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-2e2025.pdf
This NIST report explores the landscape of adversarial machine learning (AML), categorizing attacks and corresponding defenses for both traditional (predictive) and modern generative AI systems.
It establishes a taxonomy and terminology to create a common understanding of threats like data poisoning, evasion, privacy breaches, and prompt injection. The document also highlights key challenges and limitations in current AML research and mitigation strategies, emphasizing the trade-offs between security, accuracy, and other desirable AI characteristics. Ultimately, the report aims to inform standards and practices for managing the security risks associated with the rapidly evolving field of artificial intelligence.
This report establishes a taxonomy and defines terminology for the field of Adversarial Machine Learning (AML). The aim is to create a common language within the rapidly evolving AML landscape to inform future standards and practice guides for securing AI systems.
The report provides separate taxonomies for attacks targeting Predictive AI (PredAI) systems and Generative AI (GenAI) systems. These taxonomies categorize attacks based on attacker goals and objectives (availability breakdown, integrity violation, privacy compromise, and misuse enablement for GenAI), attacker capabilities, attacker knowledge, and the stages of the machine learning lifecycle.
The report describes various AML attack classes relevant to both PredAI and GenAI, including evasion, poisoning (data and model poisoning), privacy attacks (such as data reconstruction, membership inference, and model extraction), and GenAI-specific attacks like direct and indirect prompt injection, and supply chain attacks. For each attack class, the report discusses existing mitigation methods and their limitations.
The report identifies key challenges in the field of AML. These challenges include the inherent trade-offs between different attributes of trustworthy AI (e.g., accuracy and adversarial robustness), theoretical limitations on achieving perfect adversarial robustness, and the complexities of evaluating the effectiveness of mitigations across the diverse and evolving AML landscape. Factors like the scale of AI models, supply chain vulnerabilities, and multimodal capabilities further complicate these challenges.
Managing the security of AI systems requires a comprehensive approach that combines AML-specific mitigations with established cybersecurity best practices. Understanding the relationship between these fields and identifying any unique security considerations for AI that fall outside their scope is crucial for organizations seeking to secure their AI deployments.

Purdue University: The Emergence of AI Ethics Auditing

Summary of https://journals.sagepub.com/doi/10.1177/20539517241299732
Explores the emerging field of artificial intelligence ethics auditing, examining its rapid growth and current state through interviews with 34 professionals. It finds that while AI ethics audits often mirror financial auditing processes, they currently lack robust stakeholder involvement, clear success metrics, and external reporting.
The study highlights a predominant technical focus on bias, privacy, and explainability, often driven by impending regulations like the EU AI Act. Auditors face challenges including regulatory ambiguity, resource constraints, and organizational complexity, yet they play a vital role in developing frameworks and interpreting standards within this evolving landscape.
AI ethics auditing is an emerging field that mirrors financial auditing in its process (planning, performing, and reporting) but currently lacks robust stakeholder involvement, measurement of success, and external reporting. These audits are often hyper-focused on technical AI ethics principles like bias, privacy, and explainability, potentially neglecting broader socio-technical considerations.
Regulatory requirements and reputational risk are the primary drivers for organizations to engage in AI ethics audits. The EU AI Act is frequently mentioned as a significant upcoming regulation influencing the field. While reputational concerns can be a motivator, a more sustainable approach involves recognizing the intrinsic value of ethical AI for performance and user trust.
Conducting AI ethics audits is fraught with challenges, including ambiguity in interpreting preliminary and piecemeal regulations, a lack of established best practices, organizational complexity, resource constraints, insufficient technical and data infrastructure, and difficulties in interdisciplinary coordination. Many organizations are not yet adequately prepared to undergo effective AI audits due to a lack of AI governance frameworks.
The AI ethics auditing ecosystem is still in development, characterized by ambiguity between auditing and consulting activities, and a lack of standardized measures for quality and accredited procedures. Despite these limitations, AI ethics auditors play a crucial role as "ecosystem builders and translators" by developing frameworks, interpreting regulations, and curating practices for auditees, regulators, and other stakeholders.
Significant gaps exist in the AI ethics audit ecosystem regarding the measurement of audit success, effective and public reporting of findings, and broader stakeholder engagement beyond technical and risk professionals. There is a need for more emphasis on defining success metrics, increasing transparency through external reporting, and actively involving diverse stakeholders, including the public and vulnerable groups, in the auditing process.

Nature: The Mental Health Implications of AI Adoption – The Crucial Role of Self-Efficacy

Summary of https://www.nature.com/articles/s41599-024-04018-w
Investigates how the increasing use of artificial intelligence in organizations affects employee mental health, specifically job stress and burnout. The study of South Korean professionals revealed that AI adoption indirectly increases burnout by first elevating job stress.
Importantly, the research found that employees with higher self-efficacy in learning AI experience less job stress related to AI implementation. The findings underscore the need for organizations to manage job stress and foster AI learning confidence to support employee well-being during technological change. Ultimately, this work highlights the complex relationship between AI integration and its psychological impact on the workforce.
AI adoption in organizations does not directly lead to employee burnout. Instead, its impact is indirect, operating through the mediating role of job stress. AI adoption significantly increases job stress, which in turn increases burnout.
Self-efficacy in AI learning plays a crucial role in moderating the relationship between AI adoption and job stress. Employees with higher self-efficacy in their ability to learn AI experience a weaker positive relationship between AI adoption and job stress. This means that confidence in learning AI can buffer against the stress induced by AI adoption.
The findings emphasize the importance of a human-centric approach to AI adoption in the workplace. Organizations need to proactively address the potential negative impact of AI adoption on employee well-being by implementing strategies to manage job stress and foster self-efficacy in AI learning.
Investing in AI training and development programs is essential for enhancing employees' self-efficacy in AI learning. By boosting their confidence in understanding and utilizing AI technologies, organizations can mitigate the negative effects of AI adoption on employee stress and burnout.
This study contributes to the existing literature by providing empirical evidence for the indirect impact of AI adoption on burnout through job stress and the moderating role of self-efficacy in AI learning, utilizing the Job Demands-Resources (JD-R) model and Social Cognitive Theory (SCT) as theoretical frameworks. This enhances the understanding of the psychological mechanisms involved in the relationship between AI adoption and employee mental health.

ECIIA: The AI Act – Road to Compliance

Summary of https://www.eciia.eu/wp-content/uploads/2025/01/The-AI-Act-Road-to-Compliance-Final-1.pdf
"The AI Act: Road to Compliance," serves as a practical guide for internal auditors navigating the European Union's Artificial Intelligence Act, which entered into force in August 2024. It outlines the key aspects of the AI Act, including its risk-based approach that categorizes AI systems and imposes varying obligations based on risk levels, as well as the different roles of entities within the AI value chain, such as providers and deployers.
The guide details the implementation timeline of the Act and the corresponding obligations and requirements for organizations. Furthermore, it presents survey results from over 40 companies regarding their AI adoption, compliance preparations, and the internal audit function's understanding and auditing of AI. Ultimately, the document emphasizes the crucial role of internal auditors in ensuring their organizations achieve compliance and responsibly manage AI risks.
The EU AI Act is now in force (August 1, 2024) and employs a risk-based approach to regulate AI systems, categorizing them into unacceptable, high, limited, and minimal risk levels, with increasing obligations corresponding to higher risk. There's also a specific category for General Purpose AI (GPAI) models, with additional requirements for those deemed to have systemic risk.
Organizations involved with AI systems have different roles (provider, deployer, importer, distributor, authorised representative), each with distinct responsibilities and compliance requirements under the AI Act. The provider and deployer are the primary roles, with providers facing more extensive obligations.
Compliance with the AI Act has a phased implementation timeline with key dates starting from February 2025 (prohibited AI systems) through August 2027 (high-risk AI components in products). Organizations need to start preparing by creating AI inventories, classifying systems by risk, and establishing appropriate policies.
Internal auditors play a vital role in helping organizations achieve compliance with the AI Act by assessing AI risks, auditing AI processes and governance, and making recommendations. They need to ensure the implementation of AI Act requirements within their organizations.
A recent survey of over 40 companies revealed widespread AI adoption but a relatively low level of understanding of the AI Act within internal audit departments. Most internal audit departments are not yet leveraging AI, but when they do, it's mainly for risk assessment. Ensuring adequate AI auditing skills through training is highlighted as a need.

Harvard Business School: The Cybernetic Teammate – A Field Experiment on Generative AI Reshaping Teamwork and Expertise

Summary of https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5188231
This working paper details a field experiment examining the impact of generative AI on teamwork and expertise within Procter & Gamble. The study involved 776 professionals working on real product innovation challenges, randomly assigned to individual or team settings with or without AI assistance.
The research investigated how AI affects performance, expertise sharing across functional silos, and the social and emotional aspects of collaboration. Findings indicate that AI significantly enhances performance, allowing individuals with AI to match the output quality of traditional human teams. Moreover, AI facilitates the creation of more balanced solutions, regardless of professional background, and fosters more positive emotional responses among users.
Ultimately, the paper suggests that AI functions as a "cybernetic teammate," prompting organizations to reconsider team structures and the nature of collaborative work in the age of intelligent machines.
AI significantly enhances performance in knowledge work, with individuals using AI achieving a level of solution quality comparable to two-person teams without AI. This suggests that AI can effectively replicate certain benefits of human collaboration in terms of output quality.
AI breaks down functional silos and broadens expertise. Professionals using AI produced more balanced solutions that spanned both commercial and technical aspects, regardless of their professional background (R&D or Commercial). AI can also help individuals with less experience in product development achieve performance levels similar to teams with experienced members.
AI fosters positive emotional responses among users. Participants reported more positive emotions (excitement, energy, enthusiasm) and fewer negative emotions (anxiety, frustration) when working with AI compared to working alone without AI, matching or even exceeding the emotional benefits traditionally associated with human teamwork.
AI-augmented teams have a higher likelihood of generating exceptional, top-tier solutions. Teams working with AI were significantly more likely to produce solutions ranking in the top 10% of all submissions, indicating that the combination of human collaboration and AI can be particularly powerful for achieving breakthrough innovations.
AI is not merely a tool but functions as a "cybernetic teammate" that reshapes collaboration. It dynamically interacts with human problem-solvers, provides real-time feedback, bridges expertise boundaries, and influences emotional states, suggesting a fundamental shift in how knowledge work can be structured and carried out.

Baruch College: Not all AI is Created Equal – A Meta-Analysis Revealing Drivers of AI Resistance Across Markets, Methods, and Time

Summary of https://www.sciencedirect.com/science/article/pii/S0167811625000114
Presents a meta-analysis of two decades of studies examining consumer resistance to artificial intelligence (AI). The authors synthesize findings from hundreds of studies with over 76,000 participants, revealing that AI aversion is context-dependent and varies based on the AI's label, application domain, and perceived characteristics.
Interestingly, the study finds that negative consumer responses have decreased over time, particularly for cognitive evaluations of AI. Furthermore, the meta-analysis indicates that research design choices influence observed AI resistance, with studies using more ecologically valid methods showing less aversion.
Consumers exhibit an overall small but statistically significant aversion to AI (average Cohen’s d = -0.21). This means that, on average, people tend to respond more negatively to outputs or decisions labeled as coming from AI compared to those labeled as coming from humans.
Consumer aversion to AI is strongly context-dependent, varying significantly by the AI label and the application domain. Embodied forms of AI, such as robots, elicit the most negative responses (d = -0.83) compared to AI assistants or mere algorithms. Furthermore, domains involving higher stakes and risks, like transportation and public safety, trigger more negative responses than domains focused on productivity and performance, such as business and management.
Consumer responses to AI are not static and have evolved over time, generally becoming less negative, particularly for cognitive evaluations (e.g., performance or competence judgements). While initial excitement around generative AI in 2021 led to a near null-effect in cognitive evaluations, affective and behavioral responses still remain significantly negative overall.
The characteristics ascribed to AI significantly influence consumer responses. Negative responses are stronger when AI is described as having high autonomy (d = -0.28), inferior performance (d = -0.53), lacking human-like cues (anthropomorphism) (d = -0.23), and not recognizing the user's uniqueness (d = -0.24). Conversely, limiting AI autonomy, highlighting superior performance, incorporating anthropomorphic cues, and emphasizing uniqueness recognition can alleviate AI aversion.
The methodology used to study AI aversion impacts the findings. Studies with greater ecological validity, such as field studies, those using incentive-compatible designs, perceptually rich stimuli, clear explanations of AI, and behavioral (rather than self-report) measures, document significantly smaller aversion towards AI. This suggests that some documented resistance in purely hypothetical lab settings might be an overestimation of real-world aversion.

CSET: Putting Explainable AI to the Test – A Critical Look at Evaluation Approaches

Summary of https://cset.georgetown.edu/publication/putting-explainable-ai-to-the-test-a-critical-look-at-ai-evaluation-approaches/
This Center for Security and Emerging Technology issue brief examines how researchers evaluate explainability and interpretability in AI-enabled recommendation systems. The authors' literature review reveals inconsistencies in defining these terms and a primary focus on assessing system correctness (building systems right) over system effectiveness (building the right systems for users).
They identified five common evaluation approaches used by researchers, noting a strong preference for case studies and comparative evaluations. Ultimately, the brief suggests that without clearer standards and expertise in evaluating AI safety, policies promoting explainable AI may fall short of their intended impact.
Researchers do not clearly differentiate between explainability and interpretability when describing these concepts in the context of AI-enabled recommendation systems. The descriptions of these principles in research papers often use a combination of similar themes. This lack of consistent definition can lead to confusion and inconsistent application of these principles.
The study identified five common evaluation approaches used by researchers for explainability claims: case studies, comparative evaluations, parameter tuning, surveys, and operational evaluations. These approaches can assess either system correctness (whether the system is built according to specifications) or system effectiveness (whether the system works as intended in the real world).
Research papers show a strong preference for evaluations of system correctness over evaluations of system effectiveness. Case studies, comparative evaluations, and parameter tuning, which are primarily focused on testing system correctness, were the most common approaches. In contrast, surveys and operational evaluations, which aim to test system effectiveness, were less prevalent.
Researchers adopt various descriptive approaches for explainability, which can be categorized into descriptions that rely on other principles (like transparency), focus on technical implementation, state the purpose as providing a rationale for recommendations, or articulate the intended outcomes of explainable systems.
The findings suggest that policies for implementing or evaluating explainable AI may not be effective without clear standards and expert guidance. Policymakers are advised to invest in standards for AI safety evaluations and develop a workforce capable of assessing the efficacy of these evaluations in different contexts to ensure reported evaluations provide meaningful information.