The OpenAI API is where the marketing ends and the real engineering begins: tokens, rate limits, cost per million tokens, and architecture decisions you don't see in the ChatGPT interface. This is the guide a technical team needs before writing the first line of code.
Every request to the API requires an API key generated from the OpenAI dashboard, sent as a Bearer token, and optionally an organization/project header if you manage multiple teams billed separately. The right practice is one key per service/environment (dev, staging, production), never a key shared across all projects, so you can revoke and audit separately.
For years, Chat Completions (`/v1/chat/completions`) was the standard endpoint, and it's still supported and widely used in production. OpenAI introduced the Responses API as the recommended endpoint going forward, unifying capabilities that previously required separate calls (built-in tools like web search, code execution, and conversation-state handling via `previous_response_id` instead of resending the whole history on every call).
For new projects, the recommendation is to start with the Responses API unless you have existing dependencies (frameworks, third-party SDKs) that only support Chat Completions — both coexist and will keep working, but new capabilities land in Responses first.
The model family includes variants optimized for different cost/latency/quality trade-offs: the "mini" models (like gpt-4o-mini) for high-volume, low-complexity tasks (classification, simple extraction), the reasoning models (the o-series: o1, o3, o4-mini) for problems that require long reasoning chains before answering (math, complex code, planning), and the flagship models (gpt-4o, gpt-4.1, and successors) for general balance.
A common and costly mistake is using the most powerful model for everything. In a pipeline with a high volume of simple requests, using a mini model with good prompt engineering can cut cost 10-20x with no perceptible quality loss for that specific task.
For any user-facing interface, token streaming (Server-Sent Events) is practically mandatory: without it, the user waits in silence until the full response is ready, which on long responses can feel like a broken app.
Limits are measured in RPM (requests per minute), TPM (tokens per minute), and sometimes RPD (requests per day), and scale with the account's accumulated spend tier. A production system needs to handle the 429 error with exponential backoff and jitter, not immediate retries that worsen throttling.
Cost is billed separately for input and output tokens (output usually costs several times more than input), and varies by model. Prompt caching significantly reduces cost when you reuse the same long prefix (an extensive system prompt, context documents) across successive calls — it's worth structuring prompts so the static part comes first and the variable part comes last, maximizing the cache hit.
Set explicit per-request timeouts, log the `usage` of every response (input/output tokens) for real per-feature cost monitoring, use low `temperature` (0-0.3) for deterministic tasks and higher for creative generation, and never expose the API key directly to the client/frontend — everything should go through your own backend acting as a proxy with its own end-user authentication.
Carlos Montiel is an enterprise AI solutions architect. He implements LLMs, Agents, RAG, and orchestrators for companies across Guatemala and Latin America. Reach out for a consultation.
Contact Carlos Montiel