Per-token price is the easy part to estimate. What actually derails a Vertex AI budget is regional quotas, idle serving, and unplanned traffic spikes.
Vertex AI doesn't have a single pricing model: it mixes three different schemes depending on the service you use. Generative models (Gemini, Imagen, embeddings) are charged per token or per output unit (generated image, second of video). Training (AutoML, Custom Training, Pipelines) is charged per node-hour or per accelerator-hour (GPU/TPU) consumed during the job. Endpoint serving (deployed models, Vector Search) is charged per hour of active node, regardless of whether it receives traffic or not.
That last point is what derails the most budgets: an endpoint deployed with `min_replica_count=1` bills 24/7 even if it gets one query a day. For sporadic traffic, batch prediction or deploying on-demand with `min_replica_count=0` (where the service allows it) avoids that fixed cost.
Gemini pricing is quoted per million input and output tokens, with different rates between Pro and Flash, and a significant discount for tokens served from context caching versus fresh input tokens. To estimate a use case's real cost, you need to model three numbers separately: average input tokens per query (including the system prompt and any RAG-injected context), average output tokens, and projected monthly query volume. Ignoring the system prompt's size is the most common mistake: a 2,000-token system prompt repeated on every call, with no context caching, can end up costing more than the user's actual question.
Vertex AI applies "queries per minute" (QPM) and "tokens per minute" (TPM) quotas that vary by project, by region, and by specific model. A new project typically starts with conservative quotas that can become a bottleneck when scaling to production with no advance warning: a campaign with a traffic spike can start hitting 429 errors (resource exhausted) if the quota wasn't requested ahead of time.
Quota is managed from the IAM & Admin > Quotas console, and increases require a request to Google Cloud that can take hours to days depending on the size of the requested increase. Practical recommendation: request the quota increase during the load-testing phase, not the week before launch.
For predictable, high-volume traffic, Provisioned Throughput lets you reserve dedicated Gemini capacity at a fixed monthly cost, instead of relying on the shared pay-as-you-go pool. This removes the risk of rate limiting from other customers' spikes on shared infrastructure and gives predictable cost, but requires a minimum capacity commitment: it makes sense when projected monthly volume exceeds a certain threshold where the fixed cost is lower than the equivalent variable cost, not for low or highly irregular traffic loads.
In AutoML, the `budget_milli_node_hours` parameter sets a spending ceiling, but marginal returns decay quickly past a certain point: doubling the budget doesn't double the improvement in the validation metric. In Custom Training, using checkpointing and resumption avoids restarting a long job from scratch when it fails partway through -- something that, without that practice, can silently double an experiment's compute cost.
Cloud Billing lets you configure budgets with threshold alerts (50%, 90%, 100% of projected spend) at the project level or for a specific Vertex AI SKU, and export billing detail to BigQuery for granular analysis by service, model, or even resource label if endpoints and jobs are consistently tagged by team or internal project. Without that disciplined tagging from day one, reconstructing later "how much did each team spend on Vertex AI" is practically impossible.
Vertex AI offers Committed Use Discounts applicable to training compute and some serving types, with discounts growing based on commitment term (one or three years). These discounts only make sense once a stable, measured consumption pattern already exists for at least two or three months; committing before having that historical data means betting on a usage volume the project hasn't yet proven it needs.
Carlos Montiel is an enterprise AI solutions architect. He implements LLMs, Agents, RAG, and orchestrators for companies across Guatemala and Latin America. Reach out for a consultation.
Contact Carlos Montiel