Reservation Is Not Utilization: A Kubernetes Resource Optimization Case Study for Analytical Workloads
Shared Kubernetes platforms rarely run one workload class with one resource profile. They host infrastructure services, platform backends, batch jobs with strict service-level agreements (SLAs), and interactive analytical workloads, whose demand is difficult to predict before execution. Fitting these workloads into one environment is difficult because scheduler reservations, runtime limits, quota policy, and workload priority can optimize for different goals. In the on-premises case study described here, peak operational views showed approximately 95% of platform resources consumed or blocked, creating pressure to expand hardware, yet Prometheus measurements showed actual utilization closer to 15%, with most of the capacity being reserved rather than used.
After selected critical workloads were temporarily isolated on dedicated-node capacity, a custom framework was developed that exposes live request-versus-usage views enriched with essential metadata describing projects and workloads, while historizing pod and node metrics in Iceberg with Spark. The resulting visibility supported quota tightening and an 8~GiB default memory profile, but those changes provided only temporary relief as platform use and individual quota exceptions grew.
The later shift from treating resource profiles primarily as scheduler reservations toward lower requests with limits as runtime ceilings reduced average reserved memory from a renewed pre-Burstable level of roughly 900-1000~GiB to about 400~GiB, or about 55-60% in the observed environment. The transferable conclusion is that mixed Kubernetes platforms need workload-specific resource semantics and evidence-driven rightsizing: lower requests can improve bin-packing and reduce blocked capacity, but they must be balanced against overcommit, OOM behavior, strict SLAs, admission-control boundaries, and product support.
Book Now