This article is part one of a two-part series on building sustainable platforms in the age of AI. In this first piece, we focus on the architecture: how to design systems that don’t just use AI, but are built around it. In part two, we’ll explore the other side of the coin: how to design for humans — the users, reviewers, and professionals who interact with these systems every day.

For the last year, we’ve been building FAIT — an AI-powered platform for automating the messy, unglamorous world of enterprise data integration. And in that journey, we’ve made plenty of architectural decisions that were counterintuitive at first, but critical in practice.

We distilled our experience into three lessons — not just for engineers or AI specialists, but for anyone serious about building platforms that will still work in five years — not just demo well today.

Lesson 1: Segment Your Architecture by Determinism

Not everything should go to AI. One of the most overlooked architectural decisions is simply: where to apply AI at all.

Three Task Types: Deterministic, Probabilistic, and Judgment-Based

In enterprise systems, every workflow contains a mix of logic, inference, and judgment tasks. We learned early on to split these tasks into three distinct categories:

1. Deterministic: Logic-driven, repeatable, rule-based. If traditional programming is faster, cheaper, and guaranteed to be correct — use it. No shame in old tools for the right jobs.
2. Probabilistic: Pattern-driven, ambiguous, data-rich. These are your AI candidates — when there are too many options to brute-force and too much fuzziness to code manually.
3. Relationship- or Judgment-driven: The human zone. Tasks where trust, context, ethics, and forward-looking discretion matter more than raw speed or scale. This isn’t just UX — it’s where people consistently outperform machines.

This segmentation isn’t just a framework — it’s a design principle. And it becomes especially important as AI gets better. Because when everything could be done by AI, you need a clear compass for what should be.

We’ll explore the “human zone” more deeply in part two of this series — how to design human-centered systems that support human oversight, build user trust, and preserve learning rather than replacing it. But even at the architectural level, this third category is essential — and worth unpacking briefly here before we go deeper in part two.

Why Humans Matter When You Architect an AI-First Platform

AI systems today lack persistent organizational memory, evolving interpersonal context, and ethical foresight. They can’t track stakeholder dynamics, anticipate regulatory pushback, or explain decisions in stakeholder-specific terms. Humans can — and those are exactly the reasons human judgment must remain in the loop.

This isn’t speaker-circuit empathy or conference-stage performance. There’s real scholarly support for putting humans in the loop — not as sentiment, but as robust system design. Harvard Business Review frames this well:

“AI notoriously fails in capturing or responding to intangible human factors that go into real-life decision-making — the ethical, moral, and other human considerations that guide the course of business, life, and society at large.”

Stanford’s Institute for Human‑Centered AI (HAI) offers a contrast grounded in AI’s potential, championing AI as:

“…a tool for quickly recognizing patterns or predicting outcomes, which are then reviewed by experts. Keeping people in the loop can ensure that AI is working properly and fairly and also provides insights into human factors that machines don’t understand.“

As Fei-Fei Li from HAI puts it, this “is a win-win. AI is not taking away from the human element, but it’s an enabler to make human jobs faster and more efficient.”

Don’t Let AI Cannibalize the Next Generation

And there’s a longer-term cost if we forget that. If AI replaces all the “busywork,” junior professionals lose the very pathways that teach context, ownership, and judgment. That’s not just bad for morale — it’s bad for talent development. As one CTO put it, we may be “cannibalizing our future” by eliminating entry-level learning opportunities – which is not something any AI-first architecture should enable by default.

At FAIT, deterministic logic (like schema validation) runs separately from probabilistic AI inference (like field mapping or transformation logic). And humans get the final say on ambiguous mappings — not just to fix AI errors, but to learn by reviewing.

We’ll talk more in part 2 of the series about how this actually works. In short, you can think of it as judgment routing. And it’s one of the most scalable things you can do to architect an AI-first platform.

Lesson 2: Stay Model-Agnostic

The second lesson is simple: don’t marry your model.

LLMs are evolving fast. What’s best today may degrade tomorrow. What works for code might fail on compliance logic. We’ve seen Claude outperform GPT-4 in one task and underperform in another — and that’s without accounting for changes across time.

A study from Stanford and UC Berkeley found that GPT-4’s accuracy on coding queries dropped dramatically between March and June 2023, without warning or changelog. So even if your model is great today — you can’t count on it staying that way.

That’s why FAIT is built to be model-agnostic from the ground up. We route tasks to the model best suited for each job — Claude, GPT-4o, DeepSeek, Gemini, open-source, and others — and we track which ones perform best for which categories of logic.

This isn’t just a performance optimization — it’s a resilience strategy. If a vendor API breaks, or prices spike, or regulations shift (as they already have in some markets), we don’t get caught flat-footed.

For example, TrueFoundry, an LLM orchestration platform provider, highlights model routing and fallback as essential to uptime and integration flexibility — enabling failover across providers and seamless switching without code changes. That kind of modularity is a core principle when you architect an AI-first platform that can evolve with the ecosystem.

The upshot: LLMs are infrastructure. Treat them like interchangeable components, not magic partners.

Lesson 3: Test Like an AI Thinks

The third lesson may be the hardest for traditional software teams: testing AI isn’t like testing code.

In deterministic systems, testing is simple: same input → same output → test passes. But LLMs are probabilistic by nature. The same input might yield different — but equally valid — results. So “pass/fail” thinking breaks down.

In other words, unpredictability isn’t a bug — it’s a feature.

In a 2024 interview with McKinsey, Stanford HAI’s James Landay put it bluntly: “AI systems aren’t deterministic…where the same input always gives you the same output.” That unpredictability makes them “harder to design” — and, as he warns, “harder to protect against what they might do when they do something wrong.”

To architect and test an AI-first platform, you need new mental models. At FAIT, we developed FADM-1, a benchmark to evaluate:

• Field-level accuracy (Did the mapping work?)
• Logic success (Was the transformation valid?)
• Output variance (Is the model stable across multiple runs?)

It’s not just about correctness — it’s about confidence and stability. You’re not asking, “Did it get it right?” You’re asking, “How close does it get, how often — and how far off is it when it doesn’t?”

This is where most QA teams are struggling. According to Leapwork, just 16% of QA teams say they feel “very prepared” to test the systems they’re building — and that was before GenAI dialed up the complexity. In the age of AI, most still rely on deterministic test scripts — and many don’t realize how dangerous that is.

If you’re still writing tests expecting the same result every time, you’re not testing the world we live in now — you’re testing the one we already left behind.

Final Thoughts: You Can’t Retrofit AI

You can’t architect an AI-first platform by sprinkling ChatGPT on top of legacy systems.

You need a clean slate — one that reflects how AI actually behaves: flexible, contextual, and probabilistic. That’s what we’ve built with FAIT. And that’s where we think the future is going.

So if you’re designing for the next generation of software:

• Segment logic by judgment type — not by tool preference.
• Stay model-agnostic — loyalty is a liability.
• Rethink your testing strategy — AI doesn’t think in green checkmarks.

And most of all, don’t forget the human side. Keep people in the loop. Not just for compliance — but for growth. AI may be faster. But humans still do something it can’t — and never will: they care.

This is part one of a two-part series — follow us for Part 2: How to Design a Human-Centered Platform.
Curious how this applies to your architecture? Drop us a comment.
And if you’re wrestling with integration or data mapping, we’d love to show you what FAIT can do.

The post How to Architect an AI-First Platform appeared first on FAIT.

Nearly two-thirds of respondents said they are still planning or implementing the rewrite. Some were unsure of their detailed readiness status. Fewer than half of firms report being ready for production. Even among firms further along, 64.3% still rely on manual spreadsheets or basic scripts for data mapping and transformation.

Notably, none of the firms surveyed have yet achieved AI-based mapping in production — though many are moving quickly in that direction. Nearly 80% of respondents express interest in AI or are actively evaluating AI-based solutions for mapping and transformation. Moreover, around one-third are already in active evaluation, positioning themselves to benefit from early adoption.

“These results show both a gap in readiness and a tipping point in mindset,” said Aaron Hallmark, CEO of FAIT. “AI-Driven Mapping isn’t just a concept. It’s emerging as a competitive edge for firms looking to modernize and accelerate their compliance workflows.”

These findings come at a critical time for Asia-Pacific institutions preparing for the HKMA’s trade reporting rewrite, which takes effect on 29 September 2025. The update mandates stricter validation, the adoption of complex ISO 20022 message formats, standardized identifiers like UTI and UPI, and an expanded set of critical data elements (CDE). These changes demand exceptional data quality, integration speed, and architectural flexibility.​

FAIT conducted the survey in conjunction with its webinar on “AI and the HKMA Rewrite.” The session featured regulatory insights from Josephine Chung and compliance implementation lessons from Neil Fletcher. Finally, Hallmark concluded with a live demo of FAIT’s AI-driven mapping platform.

About FAIT

FAIT is an enterprise SaaS platform that uses generative AI to automate business analysis, integration logic, and deployment workflows. By combining AI-driven mapping with automated runtime execution, FAIT accelerates complex data integrations by orders of magnitude—reducing time, cost, and human error. Founded by veterans of enterprise financial technology, FAIT is headquartered in Singapore and serves clients across the APAC region and beyond. Learn more at fait.ai.

Media Contact:

FAIT Solutions
press@fait.ai

The post AI Momentum Surges for HKMA Rewrite as Firms Struggle with Manual Mapping appeared first on FAIT.

Together with the previously released FAIT Analyze, the platform now enables an automated end-to-end integration lifecycle—from data mapping through to production delivery. Clients can transform mapping logic into executable code, configure source and target system connections, and deploy into live environments within minutes—without the need for handoffs to engineering or DevOps teams.

“We’ve always believed that integration isn’t just about connectivity—it’s about comprehension,” said Aaron Hallmark, Co-Founder and CEO of FAIT. “With FAIT Build and Run, we’re giving financial institutions the tools to execute that vision: from intelligent mapping to controlled, auditable deployment workflows.”

The FAIT platform supports a range of advanced capabilities critical for regulated markets, including:

• AI-driven mapping that empowers business analysts to independently generate, preview, and validate transformation logic without dependency on technical teams
• Automated transformation packaging that produces output in industry-standard formats such as ISO 20022
• Fine-grained control over test and production environments, with deployment configuration in seconds
• Real-time execution monitoring through a built-in runtime status engine, allowing users to track each record from source ingestion through target delivery
• Support for flexible deployment modes to accommodate a wide range of integration maturity and connectivity models

Clients are already using FAIT Build and Run in production to automate complex regulatory workflows, including reporting to trade repositories under regimes aligned with CPMI-IOSCO guidelines. These deployments include support for fully auditable data lineage and runtime feedback, dramatically reducing both turnaround time and operational risk.

The launch comes at a critical time for Asia-Pacific institutions preparing for the Hong Kong Monetary Authority’s (HKMA) trade reporting rewrite, which takes effect on 29 September 2025. The update mandates stricter validation, the adoption of complex ISO 20022 message formats, standardized identifiers such as UTI and UPI, and an expanded set of critical data elements (CDE). These changes demand exceptional data quality, integration speed, and architectural flexibility.​

“What we’re seeing across the region is that compliance is becoming a test of integration agility,” Hallmark added. “With FAIT, our clients can go from rule changes to testable production deployments in minutes—not months.”

FAIT’s platform is already being used by asset managers and financial institutions in APAC to support a variety of integration use cases, from regulatory compliance to custodian connectivity and internal system modernization. In the coming months, FAIT will expand access to its platform with a self-service offering designed to accelerate onboarding and hands-on evaluation for smaller teams.

About FAIT

FAIT is an enterprise SaaS platform that uses generative AI to automate business analysis, integration logic, and deployment workflows. By combining AI-driven mapping with automated runtime execution, FAIT accelerates complex data integrations by orders of magnitude—reducing time, cost, and human error. Founded by veterans of enterprise financial technology, FAIT is headquartered in Singapore and serves clients across the APAC region and beyond. Learn more at fait.ai.

Media Contact:

FAIT Solutions
press@fait.ai

The post FAIT Launches AI-Driven Build and Run Modules Ahead of HKMA Rewrite appeared first on FAIT.

FAIT’s Co-Founder and CEO, Aaron Hallmark, spoke alongside Co-Founder and CTO Corey Manders in a session titled “How to Architect an AI-First Platform: Lessons from Building FAIT.” Their talk introduced a practical framework for integrating artificial intelligence into the heart of enterprise system design. The discussion illustrated how to transcend basic API connectivity to enable true semantic understanding across applications.

Drawing on his extensive experience in capital markets integration and his early studies at Stanford University under AI pioneer John McCarthy, Hallmark highlighted the limitations of traditional API-based approaches in achieving true system comprehension. Manders, a seasoned researcher and platform architect, elaborated on FAIT’s architectural strategies. He demonstrated how FAIT segments workflows by determinism, avoids model lock-in, and applies probabilistic benchmarks to test non-deterministic AI outputs.

“While APIs are critical infrastructure—the connectivity they address is no longer the bottleneck,” said Hallmark. “True integration requires comprehension, not just connectivity. That’s where AI-first architecture comes in—and why we created the ADI category.”

Further, Hallmark and Manders demonstrated how FAIT improves mapping efficiency, enabling clients to reduce integration timelines by orders of magnitude. Notably, the platform accelerates the upfront business analysis of data mapping by up to 400 times compared to traditional methods.

Looking ahead, the team also previewed FAIT’s upcoming roadmap. The company will soon publish its FADM-2 benchmark, covering chained logic and hierarchical formats. FAIT will also launch a “lite” version of its platform aimed at broader business analyst and developer access. Interested parties are encouraged to follow FAIT on LinkedIn to receive updates and early access opportunities.​

About FAIT

FAIT revolutionizes enterprise integration by leveraging advanced artificial intelligence to automate the business analysis of data mapping between systems. This approach significantly reduces deployment times and costs, enabling organizations to implement enterprise-grade technology solutions rapidly and effectively.

Media Contact:

FAIT Solutions
press@fait.ai

The post FAIT Presents ADI Architecture at API Days appeared first on FAIT.

The Wrong Question—And the Right Strategy

“Which model is the best?”

As an AI-first SaaS company, it’s one of the most common questions we get at FAIT.

Asking which is the best AI is a fair question—but it’s the wrong one.

When you’re building production-grade, AI-first applications—especially in fast-evolving domains like AI-Driven Integration (ADI)—the more important questions are: How easily can you switch between models at runtime? How do you choose the right model for each subtask, not just the whole workflow? And what happens when a model goes down, spikes in cost, or simply gets outpaced in the next release cycle?

Just in the last few months, Claude 3.7, DeepSeek-R1, and others have reshaped the leaderboard in different ways—and new contenders seem to arrive every week. Some models excel at analytical reasoning. Others are tuned for conversational safety. Some are fast and cheap but shallow; others are slower and more thorough. Some handle PDFs natively. Others don’t. The right model depends on the task—and that changes fast.

Amid this rapid change, it’s no wonder that we’ve seen top commercial AIs go offline for hours—a sharp reminder that asking which AI performs best is the wrong question when resilience matters just as much.

The Model Is Not the Strategy

And that raises an important point—our strategy doesn’t begin with building models from scratch. Given the pace of innovation and the billions backing today’s leading LLMs, it’s far more effective to harness the best of what’s already available. Thus, with the right engineering, orchestration, and design patterns layered on top, today’s commercial and open-source foundation models already deliver transformative results.

To put it another way, Andrew Ng famously said, “AI is the new electricity.” If that’s true, then model providers like OpenAI, Anthropic, and DeepSeek are like the electrical power grid—delivering raw power through massive, ever-improving foundation models. Their consumer-facing tools, like ChatGPT and Claude, are like basic lightbulbs: general-purpose applications that light up when plugged in.

In this case, that makes AI-first SaaS companies the appliance manufacturers—the ones designing how that power gets used. Some build simple tools; others engineer advanced, adaptive systems. At FAIT, we’re focused on the industrial end of that spectrum: applying model power with precision, control, and resilience. Continuing the analogy, think of our platform as a smart industrial lighting system for AI-Driven Integration—built not just to shine, but to orchestrate how, where, and when light is applied across a complex enterprise factory floor.

In that kind of environment, staying model-agnostic isn’t just smart—it’s essential.

From Strategy to Standard: Architecting Model Agility

Designing for Cognitive Granularity

Before you can choose the right model for each subtask, you need to understand the nature of the task itself. For us, that task is AI-Driven Integration (ADI)—a broad domain spanning everything from enterprise data architecture and mapping to validation, reconciliation, and governance. Our flagship component, FAIT Analyze, zeroes in on one of the most critical pieces: AI-Driven Mapping (ADM). Specifically, it automates the business analysis that interprets and translates meaning between source and target systems.

ADM isn’t just about schema alignment—it’s about semantic translation. Traditionally, human analysts would spend weeks—sometimes months—combing through spreadsheets, PDFs, and systems specs, writing logic by hand and validating every edge case. It’s a slow, expensive, and often inconsistent process. At FAIT, we broke this into a repeatable series of reasoning steps. Each of those steps maps to a model-level decision point—moments in the pipeline where different models can be chosen based on their strengths. That’s where architectural flexibility delivers real performance gains.

Our FAIT Analyze pipeline includes eight key decision points where specific models can be selected:

• Process source references
• Process target references
• Filter irrelevant targets
• Generate mapping logic & rationale
• Validate mapping logic
• Generate transformation code
• Refine logic via AI interaction
• Test logic on sample data

Our decision point architecture gives us the ability to match the right model to the right step, rather than forcing a single model to handle the entire pipeline. And those models behave differently: Claude is conservative and cautious. GPT-4o is fluent but literal. DeepSeek is erratic—but occasionally brilliant. That’s exactly why asking for the best AI is often the wrong question—and why subtask-level flexibility delivers real performance gains.

The Architecture of Abstraction

To make that possible, we built a generalized model abstraction layer that standardizes how our platform interacts with LLMs. Behind the scenes, that means decoupling prompts, outputs, and validation logic from any one model provider’s quirks or APIs. We also integrate guardrails and confidence checks throughout, catching low-confidence outputs before they cascade into errors. We validate structured outputs against internal schemas, allowing for consistency and quality regardless of the underlying model.

This design lets us orchestrate subtasks through a clean interface—without rewriting logic every time the underlying model changes. It’s complex engineering, but it enables seamless model swapping, resilient fallback strategies, and ongoing optimization. That flexibility lets FAIT deliver mapping results 400x faster than a human analyst, with 80–90% accuracy on real-world tasks. But when your platform depends on only one model, disruptions can cause those gains to disappear overnight. That’s why model agility isn’t just a performance feature—it’s a survival trait.

That’s our current operational baseline. But to push further, we need a way to measure…

Turning Strategy into Score: Introducing FADM-1

To systematically measure model performance, we created FADM-1—the first public benchmark to evaluate LLMs on real-world industrial use cases in enterprise systems integration, starting with AI-Driven Mapping (ADM). By aligning to benchmark best practices—clarity, measurability, fairness, and extensibility—FADM-1 establishes a strong foundation for evaluating real-world ADM performance.

Clarity

FADM-1 simulates a real-world integration scenario: mapping data from an HR source system to a government reporting target system. The inputs include structured CSVs and semi-structured PDFs; in order to produce meaningful mapping output, the AI must:

• Identify relevant source fields
• Interpret target field requirements
• Generate transformation logic
• Provide reasoning commentary
• Handle gaps and partial mappings
• Self-report mapping status and confidence

Measurability

Each model’s output is compared against a human-created golden source mapping, with results scored across six sub-metrics. But not all metrics are equal. FADM-1 places 70% of the total weight on transformation logic—the most complex and high-value part of the mapping process, and the hardest for models to get right. Source and target recognition are each weighted at 10%, while status and confidence each contribute 5%. Commentary is captured and scored, but not currently included in the final score.

Fairness

Scoring blends exact-match rules, semantic similarity, and domain-specific validation. We structure outputs as JSON and build in guardrails to ensure syntactic and referential integrity—for example, verifying that source fields used in logic actually exist. The benchmark infrastructure is agnostic to individual models, and no postprocessing is required to align formats. To account for differences in model capabilities—such as native PDF processing—we setup multiple versions of each scenario where applicable. For example, models with native PDF support (like Claude) were tested both with and without that feature enabled, ensuring results reflected real-world conditions while maintaining a level playing field.

Extensibility

FADM-1 isn’t just a one-off test—it’s a repeatable, extensible benchmark. Because the evaluation framework is scenario-agnostic, we can add new test cases simply by registering a new pair of source/target reference documents and a corresponding golden mapping. That makes FADM-1 adaptable across domains, formats, and levels of complexity—laying the groundwork for FADM-2, which will introduce measurement of XML addressing, logic chaining, ambiguity resolution, and new metrics like latency and token cost.

By turning our model strategy into a measurable score, we can track progress, compare models, and—most importantly—keep improving. Because model agility isn’t just about flexibility. It’s about performance, at scale.

Not All Intelligence Is Created Equal: What FADM-1 Reveals

Here’s how the models performed across all metrics:

Note: “(w/ PDF)” indicates the model was provided the original PDF file directly (native support). “(No PDF)” means the PDF content was first converted to plain text.

Why Performance Diverged: What the Scores Didn’t Show

Claude 3.5 with native PDF processing outperformed all other models across nearly every sub-metric, especially in transformation logic and commentary. Even when leveling the playing field by removing native PDF support, Claude 3.5 maintained strong performance, underscoring its internal consistency and reasoning capabilities.

Claude 3.7 was slightly less accurate, despite identical scores in source and target schema recognition. Commentary and transformation accuracy dipped slightly, suggesting possible differences in model tuning.

DeepSeek exceeded expectations in logic generation, outperforming GPT-4o and nearly matching Claude 3.7 in key areas. However, it was less stable across runs, showing occasional fallback behavior.

GPT-4o delivered the lowest logic accuracy—despite strong commentary and perfect field recognition. It frequently omitted required business mapping rules, defaulting to literal passthrough logic.

Symbolic Reasoning in Action: Why Lookup Logic Separates the Leaders

As an example, one of the clearest indicators of integration intelligence emerged in the mapping for the Occupation target field.

In this scenario, the source system used the field Role, which stored short internal codes like "DEV", "TST", and "MGR". The target system, however, required full-form occupational titles like "Developer", "Tester", and "Manager"—from a fixed list of accepted values. This meant the AI needed to construct a symbolic lookup table, translating each internal code into the appropriate external label.

The golden source human mapping specified this transformation explicitly:

Lookup(
Role    Occupation
DEV     Developer
TST     Tester
MGR     Manager
)

This is more than a simple copy or conditional—it’s a form of structured symbolic reasoning. The AI must infer that the Role field contains coded values, align those to the expected business labels, and express the mapping and transformation logic in a formal lookup structure.

Surprisingly, most models failed this task.

• Claude 3.5 (with PDF) consistently generated the correct mapping logic, including a valid lookup table with matching source-target pairs.
• Claude 3.5 (no PDF) performed almost as well, but occasionally defaulted to simplified or incomplete mappings.
• GPT-4o, despite its fluency, almost always defaulted to output = #Role#, ignoring the need for translation entirely.
• DeepSeek showed flashes of correct logic, but was inconsistent across runs.

To be sure, in the enterprise world, this isn’t a cosmetic error—mislabeling job roles, or any domain value for that matter, can trigger downstream reporting errors, regulatory violations, or automation failures. That’s why lookup logic is a high-signal test that we track closely in FADM-1—it separates models that understand schema from those that understand the business.

Beyond the Benchmark: What Comes Next

Getting to 80–90% accuracy didn’t come from prompting alone. It came from FAIT’s end-to-end architecture: a structured pipeline, an abstraction layer, and decision-level model control. But to move beyond that, we need to raise the bar—not just for models, but for the benchmarks that evaluate them.

FADM-2, our next benchmark iteration, will measure more demanding logic combinations, multi-layered field mappings, and hierarchical formats like XML. Models will need to generate transformation logic that combines formatting, conditionals, and lookups—sometimes within a single field.

We’ll also scale up the benchmark scenarios: more fields, more diverse mappings, and broader coverage of commercial, open-source, and small-scale models. All will be tested under uniform conditions using FAIT’s abstraction layer.

But we’re not stopping at accuracy. FADM-2 will introduce new performance dimensions—latency, token cost, and code quality—measured alongside mapping fidelity.

Over time, we may explore lightweight fine-tuning—or even proprietary models trained on FAIT’s integration data. But for now, the most scalable strategy remains the one FADM-1 already proves:

Architectural agility beats model lock-in—every time.

It’s yet another reminder that chasing the best AI is the wrong question when the target keeps moving. In a world where models improve weekly, architectural agility will always outpace any single-model bet.

Thoughts on the results? Surprises? Suggestions for what you’d like to see in FADM-2? Do you agree that “the best AI?” is the wrong question? Share your thoughts in the comments.

The post The Best AI? It’s the Wrong Question. appeared first on FAIT.