Last month at API Days Singapore, the FAIT team shared our take on how to architect an AI-first platform. Not bolt-on prompts. Not API wrappers. A real re-architecture around what AI actually is — and what it actually needs.
This article is part one of a two-part series on building sustainable platforms in the age of AI. In this first piece, we focus on the architecture: how to design systems that don’t just use AI, but are built around it. In part two, we’ll explore the other side of the coin: how to design for humans — the users, reviewers, and professionals who interact with these systems every day.
For the last year, we’ve been building FAIT — an AI-powered platform for automating the messy, unglamorous world of enterprise data integration. And in that journey, we’ve made plenty of architectural decisions that were counterintuitive at first, but critical in practice.
We distilled our experience into three lessons — not just for engineers or AI specialists, but for anyone serious about building platforms that will still work in five years — not just demo well today.
Lesson 1: Segment Your Architecture by Determinism
Not everything should go to AI. One of the most overlooked architectural decisions is simply: where to apply AI at all.
Three Task Types: Deterministic, Probabilistic, and Judgment-Based
In enterprise systems, every workflow contains a mix of logic, inference, and judgment tasks. We learned early on to split these tasks into three distinct categories:
- Deterministic: Logic-driven, repeatable, rule-based. If traditional programming is faster, cheaper, and guaranteed to be correct — use it. No shame in old tools for the right jobs.
- Probabilistic: Pattern-driven, ambiguous, data-rich. These are your AI candidates — when there are too many options to brute-force and too much fuzziness to code manually.
- Relationship- or Judgment-driven: The human zone. Tasks where trust, context, ethics, and forward-looking discretion matter more than raw speed or scale. This isn’t just UX — it’s where people consistently outperform machines.
This segmentation isn’t just a framework — it’s a design principle. And it becomes especially important as AI gets better. Because when everything could be done by AI, you need a clear compass for what should be.
We’ll explore the “human zone” more deeply in part two of this series — how to design human-centered systems that support human oversight, build user trust, and preserve learning rather than replacing it. But even at the architectural level, this third category is essential — and worth unpacking briefly here before we go deeper in part two.
Why Humans Matter When You Architect an AI-First Platform
AI systems today lack persistent organizational memory, evolving interpersonal context, and ethical foresight. They can’t track stakeholder dynamics, anticipate regulatory pushback, or explain decisions in stakeholder-specific terms. Humans can — and those are exactly the reasons human judgment must remain in the loop.
This isn’t speaker-circuit empathy or conference-stage performance. There’s real scholarly support for putting humans in the loop — not as sentiment, but as robust system design. Harvard Business Review frames this well:
“AI notoriously fails in capturing or responding to intangible human factors that go into real-life decision-making — the ethical, moral, and other human considerations that guide the course of business, life, and society at large.”
Stanford’s Institute for Human‑Centered AI (HAI) offers a contrast grounded in AI’s potential, championing AI as:
“…a tool for quickly recognizing patterns or predicting outcomes, which are then reviewed by experts. Keeping people in the loop can ensure that AI is working properly and fairly and also provides insights into human factors that machines don’t understand.“
As Fei-Fei Li from HAI puts it, this “is a win-win. AI is not taking away from the human element, but it’s an enabler to make human jobs faster and more efficient.”
Don’t Let AI Cannibalize the Next Generation
And there’s a longer-term cost if we forget that. If AI replaces all the “busywork,” junior professionals lose the very pathways that teach context, ownership, and judgment. That’s not just bad for morale — it’s bad for talent development. As one CTO put it, we may be “cannibalizing our future” by eliminating entry-level learning opportunities – which is not something any AI-first architecture should enable by default.
At FAIT, deterministic logic (like schema validation) runs separately from probabilistic AI inference (like field mapping or transformation logic). And humans get the final say on ambiguous mappings — not just to fix AI errors, but to learn by reviewing.
We’ll talk more in part 2 of the series about how this actually works. In short, you can think of it as judgment routing. And it’s one of the most scalable things you can do to architect an AI-first platform.
Lesson 2: Stay Model-Agnostic
The second lesson is simple: don’t marry your model.
LLMs are evolving fast. What’s best today may degrade tomorrow. What works for code might fail on compliance logic. We’ve seen Claude outperform GPT-4 in one task and underperform in another — and that’s without accounting for changes across time.
A study from Stanford and UC Berkeley found that GPT-4’s accuracy on coding queries dropped dramatically between March and June 2023, without warning or changelog. So even if your model is great today — you can’t count on it staying that way.
That’s why FAIT is built to be model-agnostic from the ground up. We route tasks to the model best suited for each job — Claude, GPT-4o, DeepSeek, Gemini, open-source, and others — and we track which ones perform best for which categories of logic.
This isn’t just a performance optimization — it’s a resilience strategy. If a vendor API breaks, or prices spike, or regulations shift (as they already have in some markets), we don’t get caught flat-footed.
For example, TrueFoundry, an LLM orchestration platform provider, highlights model routing and fallback as essential to uptime and integration flexibility — enabling failover across providers and seamless switching without code changes. That kind of modularity is a core principle when you architect an AI-first platform that can evolve with the ecosystem.
The upshot: LLMs are infrastructure. Treat them like interchangeable components, not magic partners.
Lesson 3: Test Like an AI Thinks
The third lesson may be the hardest for traditional software teams: testing AI isn’t like testing code.
In deterministic systems, testing is simple: same input → same output → test passes. But LLMs are probabilistic by nature. The same input might yield different — but equally valid — results. So “pass/fail” thinking breaks down.
In other words, unpredictability isn’t a bug — it’s a feature.
In a 2024 interview with McKinsey, Stanford HAI’s James Landay put it bluntly: “AI systems aren’t deterministic…where the same input always gives you the same output.” That unpredictability makes them “harder to design” — and, as he warns, “harder to protect against what they might do when they do something wrong.”
To architect and test an AI-first platform, you need new mental models. At FAIT, we developed FADM-1, a benchmark to evaluate:
- Field-level accuracy (Did the mapping work?)
- Logic success (Was the transformation valid?)
- Output variance (Is the model stable across multiple runs?)
It’s not just about correctness — it’s about confidence and stability. You’re not asking, “Did it get it right?” You’re asking, “How close does it get, how often — and how far off is it when it doesn’t?”
This is where most QA teams are struggling. According to Leapwork, just 16% of QA teams say they feel “very prepared” to test the systems they’re building — and that was before GenAI dialed up the complexity. In the age of AI, most still rely on deterministic test scripts — and many don’t realize how dangerous that is.
If you’re still writing tests expecting the same result every time, you’re not testing the world we live in now — you’re testing the one we already left behind.
Final Thoughts: You Can’t Retrofit AI
You can’t architect an AI-first platform by sprinkling ChatGPT on top of legacy systems.
You need a clean slate — one that reflects how AI actually behaves: flexible, contextual, and probabilistic. That’s what we’ve built with FAIT. And that’s where we think the future is going.
So if you’re designing for the next generation of software:
- Segment logic by judgment type — not by tool preference.
- Stay model-agnostic — loyalty is a liability.
- Rethink your testing strategy — AI doesn’t think in green checkmarks.
And most of all, don’t forget the human side. Keep people in the loop. Not just for compliance — but for growth. AI may be faster. But humans still do something it can’t — and never will: they care.
This is part one of a two-part series — follow us for Part 2: How to Design a Human-Centered Platform.
Curious how this applies to your architecture? Drop us a comment.
And if you’re wrestling with integration or data mapping, we’d love to show you what FAIT can do.
