In the current AI-as-a-Service ecosystem, one deceptively simple yet complex question is: who used the GPU? When a platform supports multiple operating models —Kubernetes pods with a full GPU, MIG instances, time-sliced shares, virtual machines with vGPU or passthrough— the answer is far from trivial. The same accelerator may be occupied by several tenants, show high activity without generating useful work, or remain apparently idle while holding critical memory that prevents another model from starting. This ambiguity makes a single utilization graph insufficient for per-tenant accountability.
To build a defensible showback or chargeback system, it is necessary to separate four measurements that are often conflated: allocation, utilization, productive work, and financial consumption. Allocation answers who controlled or reserved capacity during a time window. Utilization measures what the hardware was doing in sampled intervals. Productive work reflects service outcomes —successful requests, generated tokens, completed training steps. Financial consumption applies an approved cost policy to those data. Mixing them leads to incorrect reports and team disputes.
At Q2BSTUDIO, we understand that per-tenant telemetry is not just a technical problem but a cornerstone for enterprise AI platform governance. Our experience in developing custom software and integrating cloud services (AWS/Azure) has taught us that evidence must be built from the orchestration layer all the way to the financial ledger, preserving traceability even when attribution is approximate —as occurs with time-slicing or shared model server processes.
The first pillar is allocation: a control-plane fact. For an exclusive pod, the unit is one GPU assigned for three hours. For MIG, the profile must be preserved (e.g., 1g.10gb). For vGPU, the profile and VM. For time-slicing, the replica policy is declared, not an isolated fraction. The general equation is: allocated GPU-equivalent hours = integral of assigned capacity fraction over elapsed time. This fraction comes from an approved inventory, not from a momentary utilization percentage.
Utilization is sampled device behavior. DCGM and DCGM Exporter provide metrics like SM active ratio, used memory, power, temperature. They are essential for detecting idle reservations, saturation, bottlenecks, or thermal throttling. But utilization is not a universal billing unit. A tenant may be financially responsible for a reserved GPU even when utilization is low, just as a reserved VM costs even if its CPU is idle. A derived measure like 'active equivalent GPU hours' can be useful for analytics, but should not automatically become an invoice without organizational decision.
Productive work is application-specific. For inference, it can be successful requests, input and output tokens, generated images. For training, completed steps or processed samples. The same GPU activity can produce vastly different amounts of useful work depending on model size, precision, batch size, latency target. That is why model-server metrics —tokens per second, time to first token, queue depth— must accompany GPU telemetry.
Financial consumption applies policy to evidence. A defensible chargeback model makes explicit: tenant charge = guaranteed capacity charge + borrowed allocation charge + variable service or energy charge + shared overhead - approved credits and adjustments. Each term must be explainable; a finance reviewer must be able to reproduce the monthly result; the application owner must understand why an idle reservation still costs money.
To achieve this, a stable attribution chain across layers is needed: business (tenant ID, department), orchestration (cluster, namespace UID, pod UID), device (GPU UUID, MIG profile), service (endpoint ID, model, version), and request (trace ID). Immutable identifiers and time validity are key. When a pod is deleted or a namespace changes owner, the old relationship must remain queryable for month-end close. We recommend maintaining an independent allocation event store from the live orchestration system.
Prometheus labels should be limited to bounded dimensions (cluster, namespace, model, GPU class, tenant). Request IDs or process details go to logs or traces, not metrics. The reference architecture separates collection (DCGM Exporter, Kubernetes metrics, model-server telemetry), identity enrichment (via Prometheus recording rules), operational analytics, and the financial ledger. A practical pattern is an identity exporter that emits a gauge per active workload with allowed labels, then recording rules join normalized GPU metrics with those identities.
MIG attribution is stronger than time-slicing because an isolated instance identifier exists. However, capacity fraction must come from the approved profile, not inferred from memory usage. In time-slicing, the physical device metric is replicated among replicas; equal division is a modeled assumption, not exact measurement. Therefore, we recommend marking such data with an explicit confidence level. For vGPU, host evidence (GPU UUID, profile, VM) and guest evidence (DCGM metrics inside the VM) are needed, without summing both.
Shared model servers —a single process serving multiple tenants— require request-level accounting. Tenant cost is based on weighted tokens or requests, not on splitting the process metric. A tenant service share is calculated: tenant weighted productive units / total endpoint weighted productive units. The weighting model must be versioned and tested.
Cybersecurity also plays a role: GPU telemetry and allocations must be protected against unauthorized inter-tenant access. At Q2BSTUDIO we integrate security practices in every layer, from metric authentication to dashboard isolation. Additionally, integration with cloud services like AWS or Azure allows secure scaling of telemetry infrastructure, and using BI tools like Power BI facilitates custom showback reports for each department.
AI agents —autonomous systems running inferences or training— add another layer of complexity, as they may generate ephemeral workloads hindering attribution. Our approach assigns an agent identifier to each session and correlates it with GPU allocation via orchestration labels.
In summary, building per-tenant telemetry for AIaaS requires separating concepts, maintaining a stable evidence chain, and accepting that some attributions will be approximate but documented. The goal is not the flashiest dashboard, but an operating record that survives pod deletion, process churn, tenant disputes, and month-end close. At Q2BSTUDIO we help organizations design and implement this architecture, combining custom software development, cloud, cybersecurity, artificial intelligence, and business intelligence so that the question 'who used the GPU?' has a defensible answer.
