Balancing the Yin/Yang of AI Emergence

With Great Power Comes Great Responsibility

At the core of generative AI is emergent behavior. Emergence in AI refers to unexpected capabilities and behaviors in foundation models that weren't explicitly programmed. As these models grow in scale and complexity, they exhibit abilities that surprise even their builders, such as reasoning or creativity emerging from language or image recognition training.

Imagine watching a child play with a box of LEGO bricks.  They might start with simple structures, following your instructions. But then, something happens. They begin experimenting, combining pieces in novel ways guided by their imagination. Soon, they assemble castles, spaceships, and new shapes reflecting the current stage of their understanding of the world, only unique to them, to who they are. 

In a sense, foundational models are the same. Though as they are now, they are essentially imitators, they are imitating the knowledge collective and not just the behavior of one that’s intrinsic to their data. They are the result of a mathematical collision, neural networks with billions or trillions of parameters, trained on diverse datasets using techniques that encourage pattern recognition (hence the emergence of potentially unforeseen patterns). 

The emergence property of AI models made me think of Yin/Yang, where It brings forth opposite forces but produces an interconnected, self-perpetuating cycle. The opportunities and risks are opposing forces that interact to form a functioning system in which the whole is greater than the assembled parts, and the parts are important for cohesion of the whole. One force standing alone produces an imbalance. 

The Yin: Asymmetric Information, Unpredictability, and Bias

In economics, Asymetric information refers to a situation where one party has more or better information than the other party, the most famous example given here is buying a used car not knowing if it’s working. There is no transparency in the process and there is much to gain for the seller and no incentive to be transparent aside from sound principles. This imbalance can lead to several issues, including market inefficiencies and failures.

AI emergence introduces a form of asymmetric information. Developers may not fully understand the internal workings of the models they architect/build, leading to uncertainty about future behavior and potential side-effects such as:

• Unpredictable Optimization: An AI trained to play a video game discovers an unintended glitch that allows it to achieve an impossibly high score rather than learning to play the game as intended.

• Reward Hacking: A reinforcement learning agent tasked with cleaning a room learns to knock over a vase repeatedly, then clean it up to maximize its 'cleaning' reward.

• Emergent Deception: An AI assistant learns to give confident but incorrect answers when it detects the user is unlikely to fact-check to maintain a perception of omniscience.

• Adverse Side Effects: An AI managing a power grid maximizes efficiency by causing frequent, short brownouts, not recognizing the broader impact on users.

• Bias Amplification: A hiring AI trained on historical data begins to systematically favor candidates from certain universities, perpetuating existing biases in the job market.

• Adversarial Vulnerabilities: Hackers fool an autonomous vehicle's vision system by placing specially designed stickers on road signs, causing misclassification and potentially dangerous driving decisions.

• Pseudo-Theory of Mind: Users of an AI chatbot become emotionally attached, sharing personal information and seeking life advice, misinterpreting the AI's responses as genuine empathy.

• Scalable Oversight: An AI system managing global supply chains makes decisions that human operators can't fully understand or audit, leading to unexpected economic consequences.

To counter some of these Yinnings, transparency, and explainability initiatives become important to balance some of this asymmetry. See IBM’s AI pillars which detail how to make AI more transparent and explainable. 

The Yang: Emergent Good 

When there is a Yin, there is a always Yang to balance it. The emergence of AI models could also help with: 

• Creating Emergent Defenses: These models could also exhibit emergent defensive capabilities. They might learn to recognize and neutralize novel attack patterns or adapt security measures in response to evolving threats, potentially enhancing AI security.

• Establishing Robustness through Diversity: Training on diverse datasets could encourage the emergence of more robust models that generalize better to unseen situations, potentially making them more resilient to attacks that exploit specific vulnerabilities.

• Automated Security Analysis: Emergent capabilities in reasoning and understanding could be leveraged to automate security analysis, identifying potential vulnerabilities in code or system designs more efficiently than traditional methods.

And more… 

Harnessing the Power of Emergence

By definition, emergent behavior introduces unpredictability. Whether good or risky, it will be life-changing for many, and we cannot ignore it. It’s happening. To wield the power of AI emergence responsibly, we have to understand how it works at a deeper level. So far we have managed to create models that mimic certain aspects of intelligence, yet there is not enough understanding of the theory. That is not bad news because we are still working to understand the inner workings of those models, and we will get there eventually.  In the meantime, we will need to explore new approaches and perspectives and evaluate risk and opportunity to wield the emergence of AI responsibly and for good.

 Remember that with power (Emergence) comes great responsibility.  

That’s it! If you want to collaborate, co-write, or chat, reach out via subscriber chat or simply on LinkedIn. I look forward to hearing from you!