Resource details

Every deployment on the platform should have resource requests and limits set based on its actual usage — and revisited when traffic patterns change or new features are added. The platform provides defaults so new deployments have something to run with, but these are generic values — not tuned to your application. This page explains how to find the right values for your workload and how to keep them accurate over time.

The platform defaults are not tuned for your application. Check your actual usage on the deployment dashboard for your instance (see How to check your resource usage below) and set values accordingly.

The current GAP resource request and limit defaults are the following.

Two hard constraints regardless of what you set:

1. Limits must be greater than or equal to requests.
2. CPU limits must not be set below 100m — values lower than this cause unreliable throttling behavior on the cluster. CPU requests have no such lower bound; for mostly-idle applications a request of 10m is perfectly reasonable.

Resource requests directly determine what we pay for. Kubernetes reserves the requested amount on a node regardless of whether the application actually uses it. Requests that are higher than actual usage waste cluster capacity and affect everyone on the cluster.

Three concrete consequences:

1. Cost: Reserved node capacity that goes unused is still paid for.
2. Broken autoscaling: HPA scales based on actual usage / request. A request that is much higher than real usage makes HPA see artificially low utilization — it will not scale up even when your application is under real load.
3. Wasted node headroom: The cluster autoscaler provisions nodes to satisfy requests, not actual usage. Inflated requests cause more nodes to run than the workload warrants.

For a deeper explanation of how requests, limits, CPU throttling, and OOM kills work at the Kubernetes level, see the Kubernetes concepts deep dive.

Each GAP instance has its own deployment dashboard under the Custom dashboards section of Google Cloud Monitoring for that instance’s project. You can find the dashboards by navigating to Cloud Monitoring → Dashboards and selecting the relevant project, then filtering to custom dashboards. Direct links:

• EU staging (ems-gap-stage)
• EU production (ems-gap-production)
• US staging (ems-gap-s-us1-01)
• US production (ems-gap-p-us1-01)

This is the primary tool for inspecting your deployment’s resource usage. For a full guide to all dashboard panels see Monitoring your deployment.

Whichever dashboard you use:

1. Filter to your own namespace and deployment. Both dashboards can technically show data for many deployments at once, but the charts become unreadable that way. Set the namespace and deployment filters to your specific application. Note that the Cloud Monitoring filter dropdowns enumerate all values across the cluster before rendering — they are not as responsive as the Grafana variable selectors you may be used to. Type your value directly rather than scrolling the list.
2. Look at the usage-to-request percentage panels. A container consistently below 30% of its request is a strong candidate for reduction. Consistently above 100% means your request is too low.
3. Compare staging and production separately. Staging handles far less traffic — set lower values there using per-environment overrides. Many applications are effectively idle on staging; it can make sense to set very low requests and a single replica there.
4. Use at least 2–3 days of data for a representative picture. An hour may catch an idle window or an isolated spike. For applications with weekly traffic patterns (e.g. business-hours load that drops on weekends), a full week gives the most reliable baseline. Cloud Monitoring handles longer time ranges well — 1–2 weeks loads quickly, so there is no reason to limit yourself to a short window when right-sizing.

Rule of thumb for steady workloads: set your CPU request to P95 observed usage × 1.2.

P95 (95th percentile) means the value your application is at or below 95% of the time — it excludes the top 5% of spikes and gives a practical picture of sustained load.

For mostly-idle workloads the right request is much lower — closer to actual P99 usage or even a flat low value. See the workload patterns below for guidance by type.

Example: P95 CPU usage is 80m on a steady service → a request of around 100m is appropriate.

For a full panel-by-panel guide see Monitoring your deployment. The panels most relevant for right-sizing resource requests and limits:

CPU usage vs request (%) / Memory usage vs request (%)

These are the first panels to check. They show actual usage as a percentage of your configured request, summed across all app containers (istio-proxy excluded). Two reference lines are drawn:

• too low (30%) — your request is more than 3× your actual usage. You are reserving significantly more capacity than you need.
• too high (120%) — you are using more than your request. For CPU this means throttling risk; for memory it means approaching OOM territory.

The ideal range is roughly 50–90%. Below 30% is a clear signal to reduce the request; regularly above 100% means it needs to go up.

CPU — Used / Requests / Limits / Memory — Used / Requests / Limits

These plot all three values on the same axis. A healthy deployment shows used sitting comfortably below requests, with limits providing a ceiling above any peaks. If used regularly approaches limits, you are at risk of CPU throttling or memory OOM kills.

CPU Usage per container / Memory Usage per container

These break out usage per individual pod and container, labelled pod-container. Use these to check specific containers — for example to verify a sidecar is not consuming more than expected, or to spot a single misbehaving pod among otherwise healthy replicas. The Pod and Container filters at the top of the dashboard let you narrow down to a specific container when investigating.

CPU p95 / p99 scorecards

These show the 95th and 99th percentile CPU usage averaged across replicas over the last 24 hours. They are a convenient reference when applying the P95 × 1.2 sizing rule — the number is calculated for you.

CPU Usage spikes / CPU throttling

Check these if you suspect your CPU limit is too low. Spikes shows rolling max usage per container, which surfaces short bursts that would otherwise average out. Throttling shows what percentage of CPU scheduling periods were throttled — anything consistently above 10–20% warrants raising the CPU limit.

The Cloud Monitoring deployment dashboard intentionally excludes istio-proxy from all resource panels so that your app container numbers stay clean. For the vast majority of deployments the proxy’s default allocation is fine and you never need to look at it.

If your deployment has unusually high traffic volume or you are seeing unexpected latency that cannot be explained from your application metrics alone, it is worth checking whether the istio-proxy itself is resource-constrained. The Deployment Istio dashboard (available under Custom dashboards in Cloud Monitoring for each cluster’s project) shows the same panels for the proxy specifically — usage vs request ratio, used/requests/limits, and Istio-level request, latency, and error metrics.

If the proxy’s CPU usage vs request ratio is consistently high or you see throttling there, you can override the proxy’s default resource allocation per deployment using pod annotations in your gap.yaml. See Managing resources for Istio proxy for the exact configuration and an important warning about setting all four annotations together.

Set requests to match typical sustained load, not the theoretical maximum. Set limits to give your workload enough headroom to handle realistic peaks — the right amount depends on your workload type, see the patterns below.

Key guidelines:

• Never copy production values to staging. Staging workloads are lighter — always set staging and production values separately using per-environment overrides.
• Check every container. Sidecar containers (proxies, log shippers) often need very little and are commonly overlooked.
• Iterate. Adjust requests, deploy, observe for a few days, refine as needed.
• CPU limit minimum is 100m. Below this, throttling behavior becomes unreliable. Requests can go lower for mostly-idle apps.
• For Node.js: Do not set CPU limit higher than 1250m unless you are explicitly using multi-threading (throng, clustering). A Node.js process cannot use more than this.

If you use CPU-based autoscaling (HPA), your resource request is the denominator in the utilization calculation:

HPA utilization % = actual CPU usage / CPU request

An inflated request makes HPA think your pods are under low load and prevents scale-out even when your app is genuinely struggling:

• Request: 1000m, Actual usage: 200m → HPA sees 20% → no scale-up
• Request: 250m, Actual usage: 200m → HPA sees 80% → scale-up triggered correctly

Right-sizing requests is a correctness requirement for autoscaling to work, not just a cost concern.

Never run a production deployment with minReplicas: 1 (or a static replicas: 1).

With a single replica:

• A pod restart (crash, OOM kill, node eviction, rolling deploy) causes a brief period of zero available pods — requests fail or queue up until the pod is healthy again.
• The node the pod runs on becomes a single point of failure. If the node is evicted or goes unhealthy, your service is fully down until Kubernetes reschedules the pod elsewhere.

Two replicas means one pod can restart or be evicted while the other continues serving traffic. This is the minimum for any production service that needs to stay available during normal cluster operations (rolling updates, node maintenance, autoscaler scale-downs).

Staging is the exception. One replica is fine on staging — it handles low traffic, brief unavailability during deploys is acceptable, and the resource saving is meaningful.

# gap/gap.yaml — production
deployments:
  web:
    replicas: 2   # minimum for production HA

# gap/staging-defaults/gap.yaml — staging override
deployments:
  web:
    replicas: 1   # acceptable on staging

Characteristics: Roughly constant CPU usage throughout the day. Typical for always-on API services with relatively stable traffic.

resources:
  requests:
    cpu: "96m"      # P95 usage × 1.2, e.g. P95 ≈ 80m → 96m
    memory: "256Mi"
  limits:
    cpu: "225m"     # P99 usage × 1.5, e.g. P99 ≈ 150m → 225m
    memory: "512Mi"

Characteristics: Near-zero CPU most of the time, but periods of meaningful load — whether brief spikes (webhook callbacks, event-driven triggers) or longer sustained bursts (business-hours traffic, batch import waves). Both sub-types share the same resource strategy; the difference is how you set maxReplicas.

The most common pattern here is setting the request to cover peak load (e.g. 1000m for something that idles at 10m), reserving a near-full core around the clock for something that rarely needs it.

Strategy: Keep the request low to reflect idle reality. Keep the limit high to absorb bursts on existing pods while HPA scales up. With a low request, any real traffic will push utilization past the HPA threshold quickly — set targetAverageUtilization to 50% as a reasonable starting point and adjust based on observed scaling behaviour.

One important implication: with a low request, HPA will tend to scale aggressively toward maxReplicas whenever real traffic arrives, because utilization will be high relative to the request even under moderate load. This means maxReplicas acts as much as a cost cap as a performance ceiling — set it to a value you are comfortable running at sustained load, and monitor not just whether it is hit but whether those replicas are actually needed. If your application handles the load without errors or latency degradation at 4 pods, there is no reason to allow 8.

resources:
  requests:
    cpu: "50m"      # idle baseline
    memory: "128Mi"
  limits:
    cpu: "1000m"    # absorbs bursts while new pods start
    memory: "512Mi"

HPA config:

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 6      # raise this if traffic is sustained and needs real distribution
  metrics:
  - type: Resource
    name: cpu
    targetAverageUtilization: 50

Tuning maxReplicas:

The right maxReplicas depends entirely on how your load behaves — there is no universal value.

• Very short spikes (under a minute): HPA will likely not finish scaling before the burst ends given its 30–90 second reaction delay. The more effective lever here is a generous CPU limit — let existing pods absorb the burst rather than trying to spin up new ones. Adding replicas only helps if the burst genuinely needs more than one pod to handle it.
• Bursts of a few minutes: HPA can react in time if the burst is sustained long enough. maxReplicas: 3–4 is a reasonable safety net for most cases.
• Sustained load: Raise maxReplicas to reflect how many pods you actually need at peak. If a single pod handles ~50 req/s comfortably and you expect peaks of 300 req/s, maxReplicas: 8 gives you room. Monitor whether those replicas are ever actually reached — if your highest observed replica count is consistently 3, maxReplicas: 8 is unused headroom and can be lowered.

HPA has an inherent reaction delay of 30–90 seconds from when load increases to when a new pod is ready. For very short spikes this means HPA is mostly a safety net — the burst is often over before the new pod starts. For sustained load it works well. If your app has very latency-sensitive short spikes, consider keeping minReplicas higher rather than relying on HPA to react fast enough.

Characteristics: High memory consumption relative to CPU. Typical for JVM services, in-memory caches, applications loading large datasets.

Why memory needs more care than CPU

Unlike CPU — where exceeding the limit causes throttling (the process slows down but keeps running) — exceeding the memory limit causes an immediate OOM kill. The container is terminated and restarted with no warning. This makes memory limits a correctness concern, not just a performance one.

Request and limit strategy

For memory-heavy workloads, set the request close to your P95 actual usage. Unlike CPU, memory usage for these workloads tends to be stable rather than spiky — the application allocates what it needs and holds it. A large gap between request and limit does not buy you useful flexibility; it just means the pod could grow into that gap and get OOM killed unexpectedly.

A reasonable starting point: request = P95 × 1.2, limit = P99 × 1.5. Monitor OOMKilled events and raise the limit if they occur.

JVM applications

JVM memory usage extends beyond the heap: metaspace, code cache, thread stacks, and off-heap allocations all consume memory outside the heap. -Xmx is the JVM flag that sets the maximum heap size — it controls how much memory the JVM can allocate for your application’s objects. Setting -Xmx equal to the container memory limit is a common mistake — the JVM will OOM kill itself as non-heap usage pushes total consumption over the limit.

A safe starting point: set -Xmx to ~75% of the container memory limit, leaving the remaining 25% for non-heap overhead. For a 1Gi container limit:

-Xmx 768m   # 75% of 1024Mi, leaving ~256Mi for JVM overhead

The right percentage varies by application — a service with many threads or heavy use of direct buffers may need more headroom. Monitor the container’s actual memory usage against the limit and adjust accordingly.

If you are seeing OOMKilled restarts

Raise the memory limit by at least 25% and redeploy. If OOMKills continue, check whether memory usage is growing over time (possible leak) or is consistently near the limit (limit is simply too low). For JVM services, also check whether -Xmx is set appropriately relative to the new limit.

resources:
  requests:
    cpu: "100m"
    memory: "768Mi"   # P95 actual footprint
  limits:
    cpu: "1000m"
    memory: "1Gi"     # -Xmx 768m + ~256Mi for JVM overhead

Characteristics: Runs periodically, bursts CPU while active, then exits.

Frequency matters

Not all cron jobs are equal. A job that runs once a night has near-zero impact on cluster capacity between runs — its requests only matter during the brief window it is active. A job that fires every 1–2 minutes is effectively always running: by the time one pod exits, the next is already starting. For high-frequency jobs, right-sizing requests is just as important as it is for long-running deployments.

If your job runs more frequently than every 5 minutes, treat it like a deployment when setting resource values — check actual usage over several days and set requests based on what the job consistently consumes while active.

Request and limit strategy

Set the request to reflect what the job actually consumes while running, not its peak burst. Since the pod only exists during the run, there is less idle waste from a slightly higher request than there would be for a long-running service — but this is not a reason to over-provision. Inflated requests on high-frequency jobs accumulate the same scheduling waste as any other workload.

Set the CPU limit generously to allow the job to complete faster when the node has spare capacity — a higher limit costs nothing if the node is not fully loaded.

resources:
  requests:
    cpu: "200m"       # what the job actually consumes while running
    memory: "256Mi"
  limits:
    cpu: "2000m"      # allow burst for faster completion
    memory: "512Mi"

• Kubernetes resource concepts deep dive — CPU throttling, OOM kills, requests vs limits explained in detail
• Auto scaling documentation
• Official Kubernetes resource management docs