Executive Summary

Enterprises evaluating generative and agentic AI today are choosing between two fundamentally different infrastructure models: consuming AI capability as a managed service from a hyperscaler (Amazon Web Services, Microsoft Azure, or Google Cloud), or building and operating PrivateAI capability on infrastructure the organisation owns and controls, using VMware Cloud Foundation (VCF) 9 Private AI Services (PAIS). Neither model is universally superior. They optimise for different things: hyperscaler AI services optimise for speed of access to the broadest range of frontier models with minimal infrastructure ownership, while VCF 9 PAIS optimises for data sovereignty, predictable cost at scale, and full control of the AI stack on infrastructure the organisation already operates.

Key takeaways

Most large enterprises will end up running both models side by side — hyperscaler platforms for rapid experimentation and access to the newest frontier models, and VCF 9 PAIS for production workloads involving regulated, proprietary, or highly sensitive data. Industry research cited in this brief indicates that 56% of organisations are already running or planning production AI inference in private cloud environments, compared to 41% in public cloud — reflecting the growing weight given to data control and cost predictability at scale.

What is covered in this document

• The architectural building blocks of hyperscaler AI platforms (Amazon Bedrock, Azure AI Foundry, Google Vertex AI) versus VCF 9 Private AI Services.
• A side-by-side comparison across data sovereignty, cost model, latency, control, compliance, and time-to-value.
• A practical decision framework to help match specific AI use cases to the right platform.
• Why a hybrid approach is emerging as the default enterprise pattern in 2026.

• Two Models, One Goal: Framing the Decision

Both approaches exist to answer the same business question — “how do we put large language models and AI agents safely and cost-effectively to work on our data?” — but they answer it from opposite starting points.

Hyperscaler AI Solutions – Architecture & Characteristics

In 2026, three platforms dominate enterprise hyperscaler AI consumption: Amazon Bedrock, Microsoft Azure AI Foundry (formerly Azure AI Studio), and Google Vertex AI. All three have matured from simple “model hosting” services into full-stack platforms offering model access, agent orchestration, retrieval-augmented generation (RAG) pipelines, fine-tuning, and enterprise governance controls.

How it works

• Managed model gateway — a single API surface (Bedrock, AI Foundry, or Vertex AI Model Garden) fronts a catalog of foundation models. Amazon Bedrock offers the broadest catalog (30–40+ models across eight providers including Anthropic, Meta, Mistral, Cohere and Amazon’s own Nova line); Azure AI Foundry is anchored by exclusive access to OpenAI’s GPT family; Google Vertex AI is Gemini-first with the strongest native MLOps and AutoML tooling.
• Shared GPU/TPU capacity — compute is multi-tenant and abstracted. The provider decides hardware placement and generation; the customer consumes an API endpoint, not a physical GPU.
• Managed RAG and agent tooling — each platform provides built-in knowledge base, agent orchestration and guardrail services (e.g. Bedrock Agents and Knowledge Bases, Azure AI Foundry Agent Service, Vertex AI Search and Agent Builder).
• Consumption-based billing — pricing is per-token or per-hour of provisioned throughput, set by the provider, with limited negotiation outside large enterprise agreements.

Strengths

• Fastest path to production — no infrastructure to procure, provision, or patch.
• Access to the broadest and newest range of frontier models without capital investment or GPU allocation planning.
• Elastic scaling that absorbs unpredictable demand spikes without over-provisioning hardware.
• Deep integration with each provider’s existing identity, data and DevOps ecosystem (IAM, Entra ID, Google Cloud IAM).

Considerations

• Data and prompts generally traverse outside the customer’s network boundary, which raises data residency and sovereignty questions for regulated industries and cross-border data rules (relevant in South Africa under POPIA).
• Cost scales directly with usage and can become unpredictable and materially higher at sustained, high-volume production scale compared to owned infrastructure.
• Customers have limited visibility or control over the underlying GPU allocation, noisy-neighbour effects, or exact model-serving configuration.
• Long-term vendor and model-family lock-in risk, particularly where a specific model (e.g. GPT-5 on Azure) is only available through one provider.

On-Premises VCF 9 Private AI Services (PAIS) – Architecture & Characteristics

VMware Cloud Foundation 9 makes Private AI Services (delivered through VMware Private AI Foundation with NVIDIA) a native, standard part of the platform. It runs AI workloads — inference, fine-tuning, and RAG — directly on the customer’s own VCF-based private cloud, using the same GPU-enabled hosts, NSX networking, and vSAN storage that already run the organisation’s virtualised estate.

How it works

• VCF Automation exposes self-service blueprints for “AI Workstation” and “AI Kubernetes Cluster” deployments (including RAG-focused variants), automating VM creation, networking, storage, and DRS placement.
• Model Store provides a curated, version-controlled catalog of open-weight LLMs that can be pulled from the NVIDIA GPU Cloud or, in airgapped environments, from a local Harbor registry — enabling AI in fully disconnected environments.
• Model Runtime + API Gateway serves models through inference engines such as vLLM or NVIDIA NIM microservices running in Kubernetes pods, with the gateway handling authentication, authorisation and load balancing.
• Data Indexing & Retrieval supports RAG designs where a private vector database keeps proprietary data inside the enterprise boundary, with only the retrieved context passed to the model at query time.
• Two deployment paths — a Deep Learning VM path for single-VM inference and fine-tuning (fastest to validate GPU access), and a VKS Supervisor / GPU Operator path for scalable, multi-tenant Kubernetes-based model serving.
• Underlying VCF 9 SDDC — SDDC Manager for lifecycle management, vSphere/ESXi hosts with vGPU or GPU passthrough, NSX for zero-trust micro-segmentation between AI workloads, and vSAN for high-performance encrypted storage.

Strengths

• Data, prompts, and models never leave the customer’s sovereign boundary — a decisive factor for regulated data, IP-sensitive workloads, and POPIA-aligned data residency requirements.
• Full administrative control and visibility of GPU consumption, storage, networking and security posture, allowing SLAs to be tuned precisely for internal consumers.
• Predictable cost at scale — capital and licensing costs are largely fixed regardless of inference volume, which typically favours steady, high volume production workloads over time.
• Supports fully air-gapped and disconnected deployments for the most sensitive environments.
• Reuses existing VCF operational skills, tooling (SDDC Manager, NSX, vSAN) and hardware investment rather than requiring a new operating model.

Considerations

• Requires capital investment in GPU-capable hardware (NVIDIA H100/H200/Blackwell-class accelerators) and the relevant VCF, Private AI Foundation, and NVIDIA AI Enterprise / vGPU licensing.
• Access to the very newest frontier proprietary models (e.g. the latest closed models from OpenAI, Anthropic, Google) is constrained to what is available as open-weight or via the Model Store/NGC catalog — it is not a substitute for consuming a hyperscaler’s proprietary frontier model APIs.
• Requires in-house (or partner-delivered) skills in GPU operations, Kubernetes, and MLOps in addition to existing VCF administration skills.
• Scaling beyond installed GPU capacity requires a hardware procurement cycle rather than an instant API call.

Side-by-side Comparison

Decision Framework: Which model fits which workload

Rather than treating this as an either/or platform decision, the most useful lens is per-workload fit. The questions below help route a given AI use case to the right platform.

The Hybrid Reality

In practice, most enterprises are not choosing one model exclusively. A common and pragmatic pattern emerging in 2026 is:

• Hyperscaler AI for early-stage experimentation, proof-of-concept work, and any use case that specifically requires a proprietary frontier model.
• VCF 9 Private AI Services as the production home for AI workloads once they involve regulated data, reach steady high-volume usage, or need to meet strict data residency or auditability requirements.

This mirrors the broader trend already visible in the market: 56% of organisations surveyed in Broadcom’s Private Cloud Outlook 2026 preview are running or planning production AI inferencing in private cloud environments, versus 41% in public cloud — with private cloud specifically favoured once workloads move from experimentation into sustained production use.

Practical implication for customers

A hybrid model lets the organisation prototype quickly with hyperscaler AI services while building out VCF 9 Private AI Services on existing infrastructure for the production, data-sensitive workloads where control, cost predictability and compliance matter most.

• Recommendations & Next Steps

• Inventory current and planned AI use cases and classify each by data sensitivity, expected volume, and required model(s) using the decision framework in Section 6.
• Assess existing GPU-capable capacity within the current VCF estate (or plan a GPU-enabled workload domain) to scope a VCF 9 Private AI Services pilot for a representative production use case.
• Run a bounded hyperscaler pilot in parallel for a use case that benefits most from proprietary frontier model access or highly variable demand.
• Review data residency and compliance requirements (including POPIA considerations) against each candidate use case before committing workloads to either platform.
• Revisit this comparison quarterly — both hyperscaler AI platforms and VCF Private AI Services are evolving rapidly, and the right balance point will shift as new model and hardware generations become available.

Sources

• Broadcom / VMware Cloud Foundation Blog — “VCF 9.1: The Secure, Cost-Effective Private Cloud Platform for Production AI” (May 2026)
• Broadcom / VMware Cloud Foundation Blog — “Private AI Services: New in VMware Private AI Foundation with NVIDIA in VCF 9.0” (2025)
• Broadcom / VMware Cloud Foundation Blog — “VMware Private AI Foundation with NVIDIA: Unlock AI with VCF 9.0” (2025)
• Broadcom TechDocs — “Architectural Options in VMware Cloud Foundation” & “Requirements for Deploying VMware Private AI Foundation with NVIDIA” (2026)
• StorageReview.com — “VMware Cloud Foundation 9.1 Positions Private Cloud as the Home for Enterprise AI” (May 2026)
• Virtualization Review — “Private AI, Not Public Cloud: Broadcom’s Message With VMware Cloud Foundation 9.1” (May 2026)
• Independent hyperscaler AI platform comparisons (Bedrock / Azure AI Foundry / Vertex AI), published 2026