Q1. Could you start by giving us a brief overview of your professional background, particularly focusing on your expertise in the industry?

With more than 18 years of hands-on experience, I help organizations transform experimental tech projects into production ready platforms that are secure, reliable, and cost effective. My background spans enterprise infrastructure and automation, cloud native technologies, and platform engineering.
I started by working with Linux systems, enterprise automation, and large scale infrastructure. As the tech landscape changed, I dove into containers, Kubernetes, hybrid cloud, DevOps, and more recently, AI infrastructure. I’ve worked as a Red Hat engineer and supported organizations managing complex environments in highly regulated industries and across teams worldwide.

Today, a significant part of my work involves production AI infrastructure. This includes designing Kubernetes based GPU platforms, improving GPU utilization, introducing multi-tenancy, deploying model serving systems, implementing observability, and managing generative AI operating expenses. I also advise engineering leaders on the governance and security boundaries required for AI agents and automated systems.

I am a Docker Captain, a speaker at KubeCon + CloudNativeCon Europe 2026, and the author of eight technical books covering automation, Kubernetes, and cloud technologies, including Kubernetes Recipes. In addition to consulting and advisory work, I create practical educational content for engineers and technology leaders.

My perspective is therefore not limited to the model itself. I view AI as an end-to-end production system involving compute, data, networking, storage, security, software delivery, observability, and financial governance. A technically impressive model has limited business value if it cannot be deployed repeatedly, monitored effectively, or operated within predictable economic boundaries.

The common thread throughout my career has been automation with appropriate controls. Whether the subject is traditional infrastructure, Kubernetes, or autonomous AI agents, my objective is to make complex systems repeatable and scalable without jeopardizing reliability, security, or accountability.

 

Q2. How can infrastructure teams prevent runaway token and compute costs from eroding the financial margins of generative AI applications?

The initial step is to move beyond viewing AI costs as a generic line item on the monthly cloud bill. Teams need to break down expenses to understand the cost of each transaction, workflow, customer, and business outcome. Without this level of unit based insight, it becomes difficult to distinguish between workloads that deliver tangible value and those that are merely costly technical demonstrations.

Produce Operational and Financial Telemetry

Each model invocation should produce both operational and financial telemetry. Teams should track which model was used, input and output token counts, latency, retries, cache usage, tool calls, and the originating application or tenant. This level of detail enables organizations to determine key metrics such as the cost to complete a task, cost per active user, and cost per successful business transaction.

Model Routing

The next key control is model routing. Not every job needs the biggest or priciest model. A smart production platform should sort incoming requests—sending routine tasks like extraction, classification, or summarization to smaller models, and saving the more powerful (and expensive) models for situations that really demand advanced reasoning. Sometimes, this can mean using self hosted models, especially when their utilization and operational needs make sense for the business.

Context Management

Managing context is just as important. If retrieval systems aren’t kept in check, they tend to send too many documents or too much conversation history with every request. Teams should keep things efficient by limiting how much information gets retrieved, summarizing long histories, removing duplicate context, and setting clear token limits. Using tools like semantic and deterministic caching also helps, making sure that identical or very similar requests don’t keep using up valuable model capacity.

Safeguards on Agentic applications

Agentic applications need extra guardrails, since a single agent can kick off lots of model calls, retries, and tool executions from just one user request. It’s important to set clear limits for each workflow—maximum tokens, execution time, retries, tool calls, and spending. If any of these limits are hit, the workflow should either shut down safely or ask for a human to step in, instead of running endlessly.

When running self hosted systems, infrastructure teams should blend autoscaling with smart queueing, batching, and GPU aware scheduling. Automatically scaling up for every spike in traffic isn’t always cost effective. The real goal is to get as much useful work done as possible while keeping latency where it needs to be—not just to maximize the number of GPUs running at any given time.

Cost Governance

Finally, product teams need real visibility into cost controls. Tools like budgets, showback or chargeback, alerts, and per tenant quotas help shift AI costs from being unexpected surprises to clear design considerations. The most successful organizations treat cost as a key part of the acceptance criteria for every production AI feature.

 

Q3. What approaches effectively maximize GPU multi-tenancy and hardware utilization without introducing noisy neighbor performance bottlenecks?

There’s no one-size-fits-all way to share GPUs. To make multi-tenancy work well, start by sorting workloads based on things like memory needs, how sensitive they are to latency, how long they run, their fault isolation requirements, and how much you expect them to use the hardware.

Multi-Instance GPU technology

If you need steady performance and strong isolation, Multi Instance GPU (MIG) technology is often your best bet—provided your hardware supports it. MIG lets you split a physical GPU into separate instances, each with its own dedicated compute and memory. This makes it a great fit for situations where you have multiple tenants, inference services, or development environments that all need predictable, clearly defined boundaries.

Time slicing

Time slicing is a handy option for bursty or low usage workloads that don’t need strict isolation. It lets multiple pods share a GPU by taking turns running their tasks. But, unlike MIG, time slicing doesn’t separate memory or faults, so one workload can end up using too much memory or causing slowdowns for others. That’s why it’s important to use time-slicing thoughtfully: set quotas, limit concurrency, and be realistic about performance.

A mature platform typically offers several resource classes: full GPUs for large training or latency critical workloads, MIG profiles for isolated shared capacity, and time-sliced resources for experimentation or lightweight inference. Users request an appropriate service class instead of selecting arbitrary hardware.

Scheduling

Scheduling is just as important. A good scheduler needs to know the ins and outs of each GPU—its type, memory, topology, partition profile, and the priority of the workload. That’s where Kubernetes Dynamic Resource Allocation comes in, giving you a more flexible way to describe and hand out specialized devices, instead of treating every GPU like it’s exactly the same.

Noisy neighbor issues aren’t just limited to the GPU—they can crop up elsewhere too. Things like shared storage, CPU bottlenecks, memory pressure, or overloaded networks can all make GPU performance unpredictable. That’s why platform teams need end-to-end visibility, tracking everything from GPU usage and memory, to queue times, CPU, storage speeds, network health, and how fast applications are running.

Finally, good multi - tenancy always comes with strong governance. Setting per tenant quotas, using priority classes, having admission policies, reserving resources, and implementing chargeback all help prevent wasteful behavior. The goal isn’t to run every GPU flat out all the time—instead, it’s about getting the most valuable work done while still ensuring reliable performance for the business critical jobs that matter most.

 

Q4. What architectural boundaries are required to keep autonomous multi agent systems from overriding core enterprise security and compliance guardrails?

The most important architectural rule is this: the model should never have the last word on security. An AI agent can suggest an action, but it’s up to external, deterministic systems to actually check and enforce authorization.

Every agent should have its own unique identity and only the permissions it needs for its specific tasks. Sharing admin credentials, using overly broad service accounts, or relying on long lived API keys can be risky—if something goes wrong, or if there’s a prompt-injection attack, the impact could spread far beyond the original job. That’s why access should always be temporary, easy to audit, and as limited as possible.

The next boundary is around tools. Agents shouldn’t get free rein over things like shells, databases, cloud consoles, or internal APIs. Instead, tools should be provided through managed interfaces with clear schemas, input checks, allowlists, and policy enforcement. For example, if you have a database agent, it should go through an approved query service—not get direct, unrestricted access to your production database.

It’s also important to keep planning and execution separate. One part of the system might come up with a plan, but another should double check each action against company policy before anything actually happens. For big impact changes—like updating network rules, deleting resources, moving money, changing identity permissions, or publishing regulated information—there should always be a human sign off or a separate, trusted process for approval.

Even autonomous systems need clear boundaries. Every agent or multi agent workflow should have built-in limits—like how long they can run, how many loops they can make, how many tokens they use, how much they spend, and how many tool calls they’re allowed. These safety checks prevent a task that looks fine on the surface from spiraling into an endless loop or triggering a cascade of unintended activity.

Also, always treat communication between agents as untrusted. Multi agent systems come with risks like identity spoofing, memory poisoning, tool misuse, or spreading bad instructions from one agent to another. That’s why organizations need strong runtime controls, reliable identities, and solid safeguards wherever agents can take real world actions.

Organizations also need immutable audit trails showing what an agent received, decided, requested, and executed. Sensitive prompt content should be protected, but security teams still need enough evidence to reconstruct an incident.

The right approach is to give agents some freedom, but within strict limits. Agents can act on their own as long as they stay inside a well defined safety zone—while identity, authorization, policy checks, and approval steps are still managed by your standard security systems.

 

Q5. What structural changes are necessary to ensure automated AI systems meet strict regulatory frameworks without stalling delivery velocity?

Compliance shouldn’t just be a last minute box to check before release—it needs to be built right into the platform. If every AI project has to figure out the rules, create its own documentation, and work out controls with security teams from scratch, things slow down, get messy, and consistency goes out the window.

Establish a risk based classification process

The first big shift is to implement a risk based classification process. Not every AI app comes with the same level of risk—a tool that just summarizes internal docs shouldn’t go through the same approval steps as something that affects hiring, credit, healthcare, or critical infrastructure. By sorting systems based on what they do, how sensitive the data is, how much autonomy they have, and their potential impact, you can apply the right level of control where it’s needed most.

Compliance-as-code

The next step is to move toward compliance-as-code. This means building required controls right into your platform using reusable policies, deployment templates, and automated checks. With this approach, you can automatically verify things like approved model sources, encryption, data residency, access controls, logging, vulnerability status, retention settings, and whether all the necessary documentation is in place—before any workload goes live.
Organizations also need an evidence pipeline. Model versions, prompts, datasets, evaluation results, approvals, and deployment history should be recorded automatically. This provides traceability without requiring engineering teams to reconstruct evidence after a release manually. A model registry alone is insufficient; the organization needs lineage connecting the deployed application to its model, data, configuration, evaluations, and responsible owner.

Testing needs to go beyond the standard checks used in traditional software. For AI delivery, it’s important to regularly evaluate quality, run security tests, check for prompt injection vulnerabilities, and include bias or fairness reviews where they’re relevant. Privacy testing and regression tests that reflect real world production scenarios should also be part of the process. Clear thresholds should be set—so you know when a release can move forward automatically, when it needs a closer look, or when it should be stopped altogether.

But even with automation, people still need to be accountable. It’s up to humans to set what level of risk is acceptable, approve high impact use cases, review any unclear failures, and decide if a system is still right for its intended purpose.

Finally, responsibility cannot sit exclusively with a central compliance team. Successful organizations use a federated model: a central group defines standards and reusable controls, while product teams remain accountable for their systems. This creates consistent governance without turning the central team into a delivery bottleneck.

The objective is not to choose between speed and compliance. A well designed platform makes the compliant path the fastest and easiest path to production.

 

Q6. If you had to share one key executive takeaway for industry leaders and investors to ensure their AI infrastructure drives profitability, what would it be and why?

Think of AI infrastructure as a shared business platform with clear, measurable costs—not just a series of separate experiments or a bunch of GPU purchases.

Many organizations kick off their AI journey by trying out different proof-of-concept projects, letting each team choose its own models, tools, and cloud services. While this can seem like progress at first, it usually leads to duplicated systems, messy oversight, and hard-to-predict costs. And even if these early projects work out, it’s often hard to scale them because there’s no standard, reliable process for moving ideas from testing to full-scale production.

A shared AI platform shifts the way organizations think about costs and resources. It offers everyone consistent access to models, GPU power, data services, deployment workflows, monitoring tools, security policies, and evaluation resources. With this setup, product teams can concentrate on solving business challenges instead of rebuilding infrastructure every time they start something new. At the same time, executives get a clear view of usage, costs, risks, and results across all their AI projects.

The main thing to measure isn’t how many GPUs you have, how many models you’ve launched, or even how busy your GPUs are. What really matters is how much useful business value you’re getting for each dollar spent and each limited resource used. For example, if a GPU is running at full capacity but is mostly processing unnecessary data or low-value trials, that’s not actually efficient. On the other hand, a well managed platform that delivers real results and good margins—even if the hardware isn’t maxed out all the time—can be much more efficient.

That’s why it’s important for leaders to make sure every project defines its basic economics. How much does it cost to complete a single transaction? How does that cost change as demand increases? What business benefits does each transaction bring? And which combination of models and infrastructure gives you the quality you need at the lowest sustainable cost?

When deciding where to invest, prioritize tools and systems that help everyone across the company—like shared workflows, model routing, GPU scheduling, evaluation tools, monitoring, security measures, and financial tracking. These things might not have the same appeal as the latest hardware, but they’re what ensure your expensive equipment is put to good use.

The winners in enterprise AI will not necessarily be the organizations with the largest models or the most GPUs. They will be the organizations that convert compute into reliable, governed, and repeatable business value more efficiently than their competitors.