The Transient Nature of Prompt Engineering: A Call for More Robust Language Models

Moving Beyond the Hack Towards Robust and User-Friendly Language Models

In tech, sometimes hacks live on and become permanent. It's important to recognize when a hack is needed and set a deadline with exit criteria before letting the hack live on to become a pattern some might regret.

One recent example is prompt engineering, a valuable HACK I'd say, considering the limitations of the initial versions of LLMs, e.g., data limitations, and how they were trained. Prompt engineering was (is) a stop-gap to formulate prompts/queries in a way that the LLM would understand them, similar to how they were trained and (instruction) tuned. 𝗜.𝗲., 𝗰𝗮𝗻 𝘄𝗲 𝗰𝗿𝗲𝗮𝘁𝗲 𝗽𝗿𝗼𝗺𝗽𝘁𝘀 𝘀𝗼 𝗰𝗹𝗼𝘀𝗲 𝘁𝗼 𝘁𝗵𝗼𝘀𝗲 𝗶𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻𝘀 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝘄𝗮𝘀 𝘁𝗿𝗮𝗶𝗻𝗲𝗱 𝗼𝗻 𝘁𝗼 𝗿𝗲𝗮𝗹𝗶𝘇𝗲 𝘁𝗵𝗲 𝗯𝗲𝘀𝘁 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲?

Don’t get me wrong, we need prompt engineering today because we are not in the desired state, where, ideally, a foundation model would comprehend the prompt regardless of how it was formatted or chained. From a usability standpoint, I'd consider it a UX bug. The question that I think we should be asking is, how do we get to that desired state? Not how do we prompt engineer better, especially in the long term.

Think about it, talking to a child, you'd have to formulate your communication intent to fit in their limited vocabulary and comprehension of the world versus talking to an educated adult who shares as much context with you and have a wider understanding of the world around them. You'd then use words without engineering them to communicate efficiently.

If I'd model that to a concept, I'd use Shannon's information theory and attribute the need for prompt engineering to channel noise. This noise can be a result of inadequate training data, suboptimal model architecture, or anyother limitation.

The question here is, how to reduce noise to elimiate the need to engineer Human language. Imagine you’re asking a friend for help. You’d typically be direct and informal: “Hey, I could really use some help with this.” You wouldn’t be saying, “You’re a great friend, you’re supposed to help me, here’s exactly how you can help…” because that level of detail isn’t necessary. Your friend understands your context and the nature of your request without needing you to spell it all out.

I don't see prompt engineering going away completely (not in the short-term), but at least for foundational models, we shouldn't have to rely on it as a pillar technique and acknowledge it as a hack, and put efforts in engineering and modelling beyond the prompt layer. It's not good UX.

TL;DR:

• Hacks can become permanent, so be wary of prompt engineering solidifying as a standard practice.

• Prompt engineering is a valuable but temporary fix for current LLM limitations.

• Ideal language models understand human language regardless of prompt formatting – no engineering needed.

• Reliance on prompt engineering highlights a UX flaw: models aren't robust enough.

• Instead of refining prompt engineering, focus on reducing "noise" (limitations in data, architecture, etc.).

• Like talking to a child vs. an adult, we want models to understand implicit context, not engineered prompts.

• Shannon's Information Theory: prompt engineering is needed due to channel noise, which we must reduce.

• Prompt engineering may remain relevant in niche areas, but not for foundational models.

• Invest in engineering and modeling beyond the prompt layer for truly robust and user-friendly language AI.

• Good UX means natural interaction – let's move beyond prompt engineering for the future of language models.

That’s it! If you want to collaborate, co-write, or chat, reach out via subscriber chat or simply on LinkedIn. I look forward to hearing from you!