The AI Workload Should Shape the Infrastructure Plan

The request often arrives in a form that sounds more complete than it is.

“We need GPUs.”

There may already be a preferred system, a vendor quote, a target date, and a room someone believes can hold it. Facilities is asked about power and cooling. Network starts looking at connectivity. Procurement wants to know when the specification will be ready.

That can be productive. It can also cause the equipment and location to become the unofficial plan before the organization has defined the workload well enough to choose either one.

Training, fine-tuning, inference, research computing, medical imaging, fraud analysis, and general enterprise AI do not create one standard infrastructure requirement. A small pilot used by one team does not create the same operating decision as a shared production service. A workload that runs steadily has a different capacity and economic profile from one that arrives in occasional bursts. Data location, response time, availability, security, staffing, and growth can each change which deployment paths deserve serious consideration.

The first useful question is therefore not, “Where can we put the equipment?”

It is: What work must the complete service perform, under what conditions, and who is responsible for the outcome?

Once that requirement is clear enough, the organization can compare its existing data center, another internal site, colocation, cloud, managed infrastructure, or a hybrid approach against the same facts. The current facility may prove to be an excellent fit. It simply should not win by default because it was the first option discussed.

Begin with the outcome, users, and operating context

A workload name is a category, not a specification.

“LLM inference” could mean an internal assistant used occasionally by a small group, a customer-facing service with unpredictable peaks, or a controlled application processing sensitive records. “Model training” could mean periodic fine-tuning on a modest internal dataset or coordinated work across many systems and large data collections. Even two applications using the same model can have different latency, availability, privacy, and integration requirements.

Start with a plain-language service definition:

• What business, research, clinical, operational, or public-service outcome is expected?
• Who will use the system, and how many users, teams, applications, or external parties may depend on it?
• Is this an experiment, a time-limited pilot, a production application, a shared platform, or the first phase of a larger capability?
• What decisions or processes will depend on the output?
• What happens if the service is slow, unavailable, delayed, or produces an unusable result?
• When is the first meaningful use expected, and what demand could follow it?
• Which requirements are mandatory, which are preferences, and which are still being explored?

That operating context matters because an infrastructure decision is supporting a service—not merely installing compute. The intended purpose, users, data, and consequences establish which performance, control, resilience, and approval questions belong in the comparison.

If the use case is still undefined, that is a legitimate planning result. The organization may be ready to explore AI without being ready to commit to a facility change or a dedicated hardware platform. Labeling that distinction early protects both the project and the infrastructure team from being asked to solve a requirement that is still moving.

Break the “AI workload” into the work it actually performs

One AI service may create several different infrastructure jobs:

• Collecting, cleaning, labeling, transforming, or staging data
• Developing, evaluating, or testing models and applications
• Training, fine-tuning, or running other compute-intensive jobs
• Serving inference requests interactively or in batches
• Moving data among instruments, enterprise systems, storage, users, partners, and providers
• Storing active datasets, model weights, checkpoints, logs, outputs, backups, and archives
• Monitoring performance, security, capacity, cost, model behavior, and service health
• Supporting recovery, updates, retraining, retention, and eventual retirement

Those jobs do not automatically belong in the same place.

A hybrid design may separate development from production, temporary training capacity from steady inference, or sensitive data from a service that can run elsewhere. A colocation path may host customer-owned compute while storage, security, network, and operations remain divided among several parties. A managed platform may absorb some responsibilities while leaving data, application, access, integration, and oversight work with the organization.

Before comparing locations, draw the workload boundary. Show which components and data flows are included in the decision, which already exist elsewhere, and which dependencies must cross an organizational or provider boundary.

Otherwise, two teams can compare “the same” deployment while one is pricing only accelerator capacity and the other is planning the complete service.

Build one workload profile that every path must answer

The workload profile does not need false precision. It does need enough definition to prevent each option from being evaluated against a different assumption.

Demand and growth

• Initial users, applications, jobs, and expected business or research demand
• Average, peak, and burst behavior where those patterns are known
• Whether work is interactive, scheduled, queued, seasonal, project-based, or continuous
• The realistic first phase and plausible growth case
• The point at which a pilot architecture, equipment configuration, contract, or facility path may need to change

Compute and platform

• The type of work being performed and the software, framework, model, or application dependencies that matter
• Required or preferred accelerator, CPU, memory, interconnect, operating-system, orchestration, and compatibility characteristics
• Whether a specific configuration has been established, a general direction exists, or the requirement is still open
• Licensing, support, lifecycle, supply, and approved-platform conditions

This is where a vendor proposal can become valuable evidence. It should be translated into a complete configuration and checked against the workload—not treated as proof that the proposed system is the only way to deliver the outcome.

Data, security, and governance

• Data sources, current locations, owners, volumes, growth, and frequency of change
• Data that must enter, leave, or remain within a defined environment
• Sensitive, regulated, proprietary, licensed, export-controlled, contractual, or personally identifiable information
• Requirements for access, isolation, encryption, logging, retention, deletion, audit, model weights, and generated outputs
• Legal, privacy, security, risk, and data-governance reviews needed for the intended use

Sensitive data does not automatically dictate one deployment model. It does mean that each path must show how the applicable controls, responsibilities, evidence, and approvals will be satisfied.

Performance and service expectations

• Job-completion time, response latency, throughput, concurrency, and data-access expectations that matter to the outcome
• Network paths among users, data, compute, storage, applications, instruments, and external services
• Availability, maintenance, recovery, continuity, and data-protection requirements
• How performance and service behavior will be tested before production acceptance

“As fast as possible” and “near-zero downtime” may express a real concern, but they are difficult to design or compare. The workload owner and technical teams should translate them into conditions that can be evaluated.

Operations and lifecycle

• Platform administration, scheduling, monitoring, patching, updates, incident response, and user support
• Facilities, network, storage, security, application, model, and provider responsibilities
• Support coverage, escalation, maintenance access, spares, contracts, and change control
• Capacity and cost monitoring as usage changes
• Refresh, migration, expansion, portability, exit, and decommissioning expectations

The workload profile becomes the common test. If one option assumes continuous high utilization and another assumes occasional bursts, their cost and capacity results are not comparable. If one includes network, storage, data transfer, staffing, support, and recovery while another includes only compute, the lower figure does not yet represent the lower-cost path.

Use hard constraints to shape the shortlist—not slogans

The deployment discussion often begins with a position: cloud first, on-premises first, keep sensitive data inside, avoid capital spending, use what we already own, or move quickly with a managed provider.

Those positions may reflect real strategy or experience. They still need to be translated into requirements that can be tested.

Questions that may narrow or reshape the shortlist include:

• Does policy, contract, law, licensing, or data governance restrict where particular data, models, or services can be stored, processed, accessed, or administered?
• Must the workload remain close to users, instruments, applications, or data sources for practical performance or continuity reasons?
• Is the required hardware, software, regional capacity, network service, or support model available on the proposed timeline?
• Can the organization or provider support the required availability, recovery, maintenance, security, and incident-response posture?
• Is demand steady enough to justify dedicated capacity, variable enough to value elasticity, or too uncertain for that conclusion yet?
• Does the organization want to own and operate the platform, own the equipment in another facility, contract for a managed environment, consume a service, or divide those responsibilities?
• What must remain portable, and what operational, contractual, technical, or data dependencies could make a later change difficult?

A constraint should be supported by the applicable policy, contract, measurement, provider response, technical requirement, or accountable owner. “We cannot use cloud” may eventually prove accurate for a particular component. “The cloud can handle anything” may sound equally confident. Neither statement defines which workload stage, data set, control, service level, cost condition, or provider offering is actually being evaluated.

Compare realistic paths on the same basis

The goal is not to keep every theoretical option alive. It is to compare a credible shortlist before decisions become expensive to reverse.

Each path should answer the same categories:

1. Workload fit and expected service outcome
2. Data location, movement, control, and approval
3. Compute, storage, network, and application performance
4. Availability, recovery, maintenance, and operational responsibility
5. Initial implementation requirements and schedule dependencies
6. Capital, recurring, variable, staffing, support, and lifecycle cost boundaries
7. Growth, flexibility, portability, contractual commitment, and exit conditions
8. Evidence required before the option can advance

The answers will not all use the same units. A facility study, a provider service description, a cloud estimate, a security review, a network test, and an operating plan establish different facts. The comparison should preserve those differences while making assumptions and exclusions visible.

Separate first availability from the long-term path

The fastest way to begin useful work may not be the final production architecture. That is not necessarily a problem.

A temporary cloud environment, existing shared platform, limited internal pilot, hosted service, or short-term managed capacity may allow the organization to validate a use case while a longer-term path is evaluated. Conversely, an existing internal system may support an early pilot without proving that the same room, equipment, staffing model, or network can support broader production demand.

The project should state what the first phase is intended to prove:

• Value of the use case
• User adoption or workflow fit
• Model or application behavior
• Data availability and quality
• Performance under a defined test condition
• Integration, security, governance, or operating feasibility
• Demand and utilization assumptions

It should also state what the phase does not prove.

A successful pilot does not automatically validate production availability, full-scale cost, growth capacity, facility suitability, provider capacity, security approval, support coverage, or lifecycle ownership. It provides evidence for the next decision.

That distinction lets the organization move without pretending the entire future architecture has already been settled.

Compare the complete cost and schedule boundary

No path should be described as inexpensive, expensive, fast, or slow without defining the comparison boundary. For each realistic option, distinguish:

• Acquisition or consumption costs, contractual commitments, and what capacity or service is actually included
• Site, provider, connectivity, implementation, validation, and transition work required before use
• Ongoing staffing, support, software, data movement, utilities, maintenance, and lifecycle obligations
• Growth, portability, renewal, termination, migration, and exit costs

The schedule should likewise extend beyond delivery of compute. Provider capacity, approvals, contracts, site work, connectivity, data movement, integration, testing, commissioning, and operational acceptance may sit on different critical paths.

Use ranges and scenarios where demand is uncertain. Record which assumptions drive the result. A cost model based on steady production usage should not quietly become the business case for a workload that is still a sporadic pilot.

Bring the right evidence and owners into the comparison

No single team is likely to own the complete answer.

The project owner does not need to turn this into one enormous meeting. The practical job is to make sure each decision uses the same workload profile and that every material assumption has a source, owner, and date.

Use decision gates while the path can still change

Four gates can keep the comparison proportional and useful:

1. Before a platform or location becomes the working plan

• Is the outcome, user population, workload stage, data, operating context, first phase, and growth case defined well enough to plan?
• Which requirements are firm, and which are still assumptions?
• Which deployment paths are realistic enough to compare?

2. Before the shortlist is narrowed

• Has each option answered the same workload, control, service, cost, schedule, and operating questions?
• Are any paths being removed because of documented constraints rather than preference or habit?
• Are the remaining unknowns assigned to people who can obtain the evidence?

3. Before procurement, contract, or infrastructure work is committed

• Has the proposed configuration and complete service boundary been established?
• Have the site, provider, network, storage, security, data, cost, responsibility, and schedule claims needed for this decision been validated at the appropriate level?
• Does the approval cover a pilot, a production phase, future growth, or only part of that sequence?

4. Before production acceptance

• Has the service been tested against the agreed workload and operating conditions?
• Are monitoring, support, incident response, maintenance, recovery, security, cost, capacity, and lifecycle responsibilities active?
• Does leadership understand any limits, assumptions, temporary arrangements, or later decision points?

A gate can confirm the preferred path, approve a bounded first phase, require one more validation item, or keep an alternative open. It does not have to force a premature yes-or-no verdict.

What if the workload or path is still uncertain?

Uncertainty does not automatically require the project to stop. The response should match the missing decision.

Possible response categories include:

• Narrowing the first use case and defining what the pilot must establish
• Running a time-limited test using existing, cloud, hosted, shared, or managed capacity
• Obtaining configuration-specific equipment and provider information before selecting a location
• Comparing two or three realistic paths with one workload profile and common cost boundary
• Separating workload components whose data, performance, scale, or operating needs differ
• Validating the existing or alternate site while keeping an external path open
• Phasing the deployment so later growth depends on measured demand and completed infrastructure work
• Revising the configuration, service expectation, schedule, operating model, or budget boundary
• Delaying a major capital or contractual commitment until the use case and demand are defined well enough to support it

These are planning responses, not recommendations for a specific organization. The right next step depends on what is known, what is still assumed, and which commitment is approaching.

Five questions for the leadership conversation

Before the organization commits to a major AI infrastructure path, leadership should be able to ask:

1. What service or outcome are we supporting, for whom, under what operating conditions, and at what initial and growth scale?
2. Which workload components and data flows are included—and do they all need to run in the same environment?
3. Which on-premises, internal, colocation, cloud, managed, hybrid, or phased paths were compared against the same requirements?
4. What evidence supports the proposed path, and which facility, provider, security, cost, schedule, or operating assumptions remain unresolved?
5. What are we approving now: exploration, a pilot, a platform purchase, infrastructure work, a production service, or a longer-term operating commitment?

If those answers are incomplete, the finding is not that the organization is unready for AI. It is that the workload and path decision need greater definition before the most difficult-to-change commitments are made.

What a readiness assessment can—and cannot—tell you

A readiness assessment can organize the reported workload intent, scale, timing, data considerations, infrastructure position, and possible alternatives and identify where the path needs greater definition.

It cannot select or validate a platform, site, provider, cost, schedule, performance level, or control environment. Those conclusions require the actual workload, equipment, data, facility, provider, contract, policy, testing, and qualified review appropriate to the decision.

Its value is identifying that work while the workload, equipment, location, infrastructure path, budget, and schedule can still change.

This page provides readiness and decision-framing guidance. It does not select an AI platform or deployment path, verify a site or provider, confirm security or compliance, validate performance, establish cost or schedule, or provide a site-specific technical, financial, legal, procurement, or operational recommendation.