Platforms, Products, APIs, and Indian Food
Navigating the Nuances of Platforms + Strategies for Success
According to Gartner, Platform Engineering is peaking the inflated expectations curve (just a couple of points behind AI!). But what is a Platform, what value does it provide, and what should be considered when Engineering one? I will explore these questions in this post.
⛔ For the busy folks, here is a TL;DR version:
Platform Principles: Create a foundation for streamlined development of value. I.e., Platforms provide tools, services, and automation to simplify and accelerate value delivery.
Platform Archetypes: Platforms come in various shapes and forms. Examples are Cloud platforms (Google Cloud, etc.), business platforms (Salesforce, etc.), and Internal Developer Platforms (IDPs). All offer unique value and are targeted to a set of personas.
Key Considerations:
APIs: Inspired by Amazon’s API mandate, platforms should provide consumable and persona-focused APIs that cover the right set of needs.
Purposeful Design: Understand the value you want to deliver and tailor a platform to serve those needs. Failure to do so results in overserving (surplus) or underserving (shortage). You will want to strike the balance (equilibrium).
Product of Platform vs. Platform for Product: Determine if the platform itself or the value it brings is the desirable.
Skills and Tools: Platform creation requires a wide range of skills (frontend, backend, infrastructure, etc.). Assemble the right people and tools to get the job done. Assemble your platform engineering fellowship.
To Remember: Keep your platform user-focused, simplify wherever possible, and continuously evolve it based on feedback.
What REALLY is platform engineering?
I’d define it simply as:
The foundation upon which value is being delivered
But it’s also a mashing of building blocks, tools, and services that enable the target persona (developers, ML engineers, data scientists, ...) to craft value.
There are many contexts in which you’ll see platforms mentioned. I will use the following format to describe them “_platform such as platform_name that provides _tools_services to enable _those_enabled to _value
Cloud Platforms, such as Google Cloud Platform, AWS, Azure, provide services (databases, networking, IDEs) for developers to use to generate value.
Business and Marketplace Platforms, such as Salesforce or Shopify, provide pre-built modules for customer management, payment processing, and e-commerce functionality to enable businesses to operate and sell online.
Social Media Platforms, such as Facebook or Twitter, provide social networking, content distribution, and API access to enable individuals and businesses to connect, engage with audiences, and build communities.
I left the best for last, Internal Developer Platforms (IDP), garnering attention, are no different. Let’s use the format above:
Internal Developer Platforms (IDPs) provide self-service portals, automation, and standardized workflows to enable developers to provision resources, deploy code for value-generating applications, and manage them easily.
Depending on the level of abstraction needed, a platform exposes a construct (or an Application Programming Interface aka API) to request for a service needed to provide value. Let’s talk more about this.
The API Mandate, the Chef’s Analogy, and Purposing Platforms
When I think about platforms nowadays, I think about two things:
The API Mandate - thanks to Jeff Bezos
Indian food (delicious) 🥘 - thanks to Kelsey Hightower
Food and APIs, what are you talking about, man? Let me explain 🙂
Amazon’s API mandate, as championed by Jeff Bezos at Amazon, essentially states that all functionalities within a platform should be accessible through well-defined APIs. I.e., all value generated by any team should be exposed via a well-defined interface, the side effect of that mandate was that Amazon was able to externalize those set of APIs to wider consumptions, and also what became AWS cloud platform (Augusto Marietti has a more detailed writeup on this here, good read!). That is probably one of the earliest appearances of a “platform”. The question is what kind of APIs to expose?
As highlighted in Working Backwards by Bill Carr and Colin Bryar, Amazon’s teams are grouped by mission vs. function, so depending on each team’s mission, and desired outcome, an API will emerge to allow others to consume the artifact produced by those Two-Pizza Teams (or should we call them distributed platform pizza teams 🙂).
Indian food is a specialized cuisine that is known for its unique taste and flavors. In one of those Twitter spaces, Kelsey Hightower gave the example of an Indian restaurant where quality is tied to specialty. You wouldn’t go to an Indian restaurant to eat Italian food. If that restaurant served Italian, you’d typically question the quality of the food! Sure, you can stock a kitchen with ingredients from all cuisines, but until you curate a menu, you might not have something of true value to offer.
The combined lesson here is to understand who you are serving and for what purpose, to build the right abstractions. Generalize too much, and you risk complexity, you overserve your target persona, and you drive them away from using your abstractions altogether (negative value). Specialize too much, and you risk undeserving the target persona, they don’t get what they need, and they will start building their own abstractions on top to fill the gaps (shadow IT).
In economics, the right terms to use are supply surplus and shortage. Supply surplus denotes a situation in which the quantity supplied of a good exceeds the quantity demanded. In the case of platforms here, too many APIs, too many unused templates, and too much functionality that has the side effect of increasing support times, engineering development time, etc. Whereas shortage represents the opposite scenario where the quantity demanded is greater than the quantity supplied, for example, developers need APIs to produce services of value, lack of APIs can be an example of a shortage. Here is an excerpt from Stevey’s Google Platform Rant describing an example of this:
Google+ is a prime example of our complete failure to understand platforms from the very highest levels of executive leadership (hi Larry, Sergey, Eric, Vic, howdy howdy) down to the very lowest leaf workers (hey yo). We all don't get it. The Golden Rule of platforms is that you Eat Your Own Dogfood. The Google+ platform is a pathetic afterthought. We had no API at all at launch, and last I checked, we had one measly API call. One of the team members marched in and told me about it when they launched, and I asked: "So is it the Stalker API?" She got all glum and said "Yeah." I mean, I was joking, but no... the only API call we offer is to get someone's stream. So I guess the joke was on me.
It’s a balancing act and no easy feat, but when you get there, you are a platform karate black belt!
Is the Platform the P-roduct OR the p-roduct
Platforms are value bridges, at least when considering why you would build one in the first place. If you treat the platform as the “main” thing, you’d be overlooking the purpose and the reason a platform exists in the first place, at the same time, if you JUST focus on the value the platform delivers, you might not even make it to where value is finable/consumable, well, you still need something, a vehicle of sorts to get there!
This has been a long-standing topic of discussion, and it is expressed in different terminologies/contexts such as:
The mean vs. the end
The platform product vs. the experience product
The P-roduct vs. the p-roduct (lower case and upper case)
Let’s take the last one. How can the p-product (lowercase) serve the uppercase P-roduct. How can we build platforms with purpose? You might have different goals depending on who you are (a supplier or a consumer) and your strategy.
Shreyas Doshi talked about the generic concept in one of his posts. Here are some of the examples/results based on surveys from his followers (who are mostly product builders) on what they consider the P-roduct:
Netflix: Most view licensed content as The Product, suggesting that content is more crucial than the technology (app or CDN) or algorithms.
Uber: The ride is viewed as The Product, more than the app or the matching algorithms.
Instacart: The groceries and the delivery system are almost equally viewed as The Product, indicating the importance of the end product and its delivery.
Amazon: The delivered item is considered The Product, underscoring the end result over the platform or logistics (the listings, the marketplace, etc).
YouTube: The creators are seen as The Product, highlighting the value of content creators over the technological features (search, the app, the video viewer).
As a supplier (a vendor), you might consider the platform itself as a P-roduct to streamline how platforms are built, making it easier to produce content on top. As a consumer, you might think more of the content you offer on top of platform while placing constraints on platform usability (that acts as a filter to which vendors you “buy” the platform from).
I don’t believe there is a clear cut answer, I’d be interested on your thoughts and how you approach this as a consumer or supplier of platforms. Let chat!
Picking the Right Skills & Tools (e.g., AI/ML Platform)
Once you understand the core value you are delivering via the platform and the type of platform you want to build, it’s time to make sure there is enough skill in the house to build it. Here, you might have to mix and mash skills. For example, you might need a mix of frontend, backend, and infrastructure, as well as one or more of DevOps, MLOps, SecOps, ... xOps 😁, whatever makes sense to get the job done (e.g., reduce cognitive load for developers).
To reduce the cognitive load of developers, the Platform team should cover the entire tech stack: Infrastructure/DevOps/SRE, but also frontend, backend, and security topics. ~ from Platform Engineering, Part 2: WHAT Are The Goals of a Platform Engineering Team? | by Benoit Hediard | Stories by Agorapulse | Medium
To make things a bit more realistic, let’s go with an example of an AI/ML Platform and the typical components involved. The closer the component is to the top, the more visible it is to the users.
Let’s break it down relative to the end-user (inspired by Wardley maps but without the X-axis):
Top Row (High Value to Users): This represents the User Interface Layer where direct user interactions happen. AI tooling, including chat interfaces (e.g., streamlit), Dev environments (e.g., Jupyter), and maybe an orchestration UI (e.g. Kubeflow)
Second Row (Medium Value to User): Data Processing and Workflow Management tools (e.g., Kubeflow pipelines, Airflow, DAGs,…) that are crucial for the operational aspects but less visible to the end user.
Third Row (Medium to Low Value to User): Model Monitoring and Operations tools (Promethus/Grafana, Evidently,…), which are essential for maintaining the platform's health and performance.
Bottom Row (Lowest Visibility): Infrastructure (AWS, GCP, Azure,…) and Platform Management along with DevOps and Collaboration Tools, which are foundational yet mostly invisible to the end user, another name for this layer can be the “commodity” layer.
I will not go more into the technical details here, but there are multiple venues to explore if you are interested specifically in AI/ML platforms and personas. One venue to explore is the Cloud Native AI working group, especially the most recent AI whitepaper. Another venue for exploring platform engineering is the platform engineering white paper.
The components will change depending on the type of platform you are building. You don’t need everything in a single platform. You need to curate and specialize based on feedback from your target platform users (remember specialization and Indian food 🥘). The left side of the figure above shows what a typical “generic” platform would be comprised of.
As such, you will need to ensure that you have the right skill set to build your target platform, for the personas you are targeting.
Conclusions & Reflections
So here are some parting recommendations:
Define Clear Objectives and APIs: Start by clearly defining your platform's purpose and the APIs it will offer (how will your platform be consumed). This clarity will enable your platform to provide specific, valuable functionalities that are easy for users to implement and integrate. This is another way to say, start with the end in mind (from the 7 habits).
Focus on User Needs: keep the end-user in mind during development. Understand their pain points, requirements, and usage patterns. This understanding will help in creating features that are not just innovative but also highly relevant and user-friendly.
Keep it lean and Evolve with time: Avoid adding unnecessary features/functions that do not add clear value. Overcomplication can deter users and increase the cognitive load, making the platform less appealing and harder to use. Always aim for simplicity and efficiency in design which itself will drive more straightforward execution.
Continuous Feedback and Improvement: Implement a robust feedback loop with your users to continuously improve the platform. Regular updates based on user feedback and emerging trends will keep the platform relevant and highly functional. I found this post by Abi Noda and Tim Cochran helpful as a guide for collecting feedback (via qualitative and quantitative metrics). I will also discuss feedback collection in upcoming posts.
Different people have different definitions of value. What’s important is that within your realm, among your people, you align on the core value you want to deliver. Here are some questions to ask to sanity check if you are on the right path:
Do I understand what problem I'm solving (the core job to be done)? Can I articulate it better than the customers themselves?
Is this tech the right fit for the use case at hand? It’s not about the fanciest tool, but one that gets the job done.
How well do the tools and systems I implement meet the evolving needs of your users? This usually translates to how modular is my architecture.
Am I adding value or just complexity? Sometimes, the simplest option is the right one, beware of overengineering (been there, done that)
Can I do better? In what ways can you further automate to reduce costs and cognitive load? Are you collecting enough feedback to know this?
That’s it! If you want to collaborate, co-write, or chat, reach out via subscriber chat or simply on LinkedIn. Looking forward to hearing from you!
