The corporate world is increasingly formalizing artificial intelligence's role in the workplace, with companies now assigning AI systems positions on organizational charts and treating them as legitimate team members. What started as an experimental approach to boost efficiency has evolved into a widespread practice where AI agents operate alongside human staff. Yet researchers examining this phenomenon are discovering that organizations may be charging ahead without adequately grasping the operational hazards they are creating.
At a human resources conference last October, Emma Wiles, a Boston University professor specializing in workplace technology adoption, encountered two HR executives enthusiastically describing how treating AI as genuine employees represented a forward-thinking strategy for competitive advantage and productivity enhancement. This observation prompted Wiles and her colleagues from Boston Consulting Group to conduct deeper investigation into what was actually happening across the corporate landscape. What they uncovered was troubling: a systematic pattern of reduced scrutiny that undermined the very reliability companies sought to achieve.
Their experimental research, which included dozens of firms actively deploying AI systems in operational roles, revealed a striking behavioral phenomenon. When managers believed AI agents had produced documents, they performed substantially less rigorous error detection than when reviewing identical materials attributed to human colleagues. Mistakes that vigilant supervisors consistently identified in human-produced work went unnoticed when the work carried an AI label. This suggests that organizational structure itself—the decision to formalize AI as an employee—fundamentally alters how people evaluate quality and accountability.
Wiles theorized that managers operating within organizations that had integrated AI as formal team members had unconsciously shifted their mental models of responsibility. Placing AI on the organizational chart may inadvertently signal that oversight duties are diffuse or delegated elsewhere, perhaps to technical teams or senior executives who championed the adoption. Managers appeared to absolve themselves of direct accountability, reasoning that failures belonged to others' domains rather than their own supervisory purview. This psychological disengagement from quality control represents a serious vulnerability in systems designed to function reliably.
While companies have grown increasingly aware of documented AI shortcomings—algorithmic bias against minority groups, confidently stated but incorrect information, and data privacy violations—researchers are now identifying more subtle but equally consequential problems. These emerging pitfalls might theoretically be remedied through better management practices, such as holding supervisors directly responsible for AI subordinate performance. Yet most organizations appear unaware these issues even exist, let alone that corrective measures are necessary. As Wiles noted, the field is grappling with "unknown unknowns"—problems not yet fully catalogued that could systematically undermine operational effectiveness.
One particularly concerning flaw involves AI systems' demonstrated preference for work produced by other AI systems. When companies deploy AI models to evaluate résumés, those generated with AI assistance tend to receive higher ratings than equally qualified human-composed documents. Ohio State University operations professor Jane Yi Jiang and collaborators documented this bias in recent research that surprised some corporate recruiters into seeking guidance. However, this single identified problem likely represents only the tip of a much larger iceberg. The speed with which companies are implementing large language models throughout their operations—from pricing decisions to site selection for new facilities—suggests they are moving faster than their capacity to anticipate consequences.
The difference in how AI and humans approach strategic decisions reveals another hidden risk. Where humans typically seek cooperative, mutually beneficial outcomes when given decision-making authority, AI models trained on game-theoretic frameworks tend toward coldly calculating optimization strategies that can trigger destructive escalation. An AI system might recommend aggressively undercutting competitors in ways that provoke broader price wars, harming profitability across entire markets. Jiannan Xu, a doctoral researcher at the University of Maryland, observed that most large language models consistently misestimate human rationality, assuming humans operate more rationally than empirical behavior demonstrates. This misalignment creates scenarios where AI's "rational" recommendations lead to collectively worse outcomes.
Wiles' comprehensive survey of over one thousand corporate managers revealed the extent of this organizational shift. Approximately one-third of respondents indicated their firms referred to AI systems as team members or employees, while nearly one-quarter reported that their employers had literally assigned AI agents positions within official organizational structures. One manager even described their AI system by name—"Scout"—as an equivalent peer occupying a formal team slot. This isn't merely rhetorical; the language and structural positioning appear to meaningfully affect how managers exercise oversight responsibilities.
The experimental validation of this phenomenon came through Wiles' error-detection exercise. Managers received five error-containing documents with twenty minutes for review, with varied attributions: some described as AI-employee work, others as AI-tool output, and others as human work. Managers at companies with formalized AI employees caught substantially fewer errors when told they were reviewing AI-employee documents, despite making similar catches in other conditions. The organizational decision to integrate AI as a formal employee had somehow activated different psychological assumptions about responsibility, scrutiny, and accountability.
This psychological distinction matters because management practices for humans have evolved over centuries into reliable frameworks emphasizing supervisory accountability. When a direct report produces flawed work, managers naturally assume responsibility and implement closer monitoring. When tools malfunction, managers similarly treat themselves as accountable for implementing proper safeguards. But AI employees occupy a peculiar categorical space where traditional mental models seem to misfire, creating a responsibility vacuum that no one fully occupies. The managers surveyed did not uniformly reduce scrutiny across all AI scenarios—only when AI held formal employee status did accountability assumptions shift dramatically.
For Southeast Asian companies increasingly adopting AI systems across finance, customer service, supply chain management, and recruitment, this research carries particular significance. Regional economies competing for technological leadership may feel pressure to deploy AI rapidly without the institutional safeguards that older implementations gradually developed. Malaysian firms, Singaporean enterprises, and others across ASEAN face a choice about whether to learn from these documented pitfalls or repeat them. The research suggests that simply importing AI technology without deliberately designing corresponding management frameworks creates systematic blind spots that directly threaten operational reliability and decision quality.
Wiles emphasized that the fundamental shortcomings don't necessarily inhere in the technology itself but rather emerge from human adoption patterns lacking sufficient deliberation about potential failure modes. Organizations are moving to integrate AI into consequential decision domains with insufficient attention to how their own cognitive frameworks and management assumptions might fail under these novel circumstances. The gap between what companies believe they've implemented and what actually unfolds operationally appears to be widening as adoption accelerates. Without deliberate attention to these psychological and organizational dynamics, the promised productivity and cost advantages risk remaining unrealized while hidden costs compound.
The challenge ahead requires companies to acknowledge that anthropomorphizing AI—giving it names, titles, and positions—fundamentally alters organizational psychology in ways that undermine quality control. Rather than treating AI integration as primarily a technical problem, leaders need to recognize it as requiring new management disciplines that have yet to crystallize into standard practice. As Wiles concluded, while societies have accumulated reliable wisdom about managing human teams over centuries, the psychology of managing humanlike artificial intelligence remains largely uncharted territory. Companies genuinely moving forward will be those that deliberately pause to map this terrain rather than charging ahead "blind."
