API Integration Tools for AI Agents

Designing API Tools

The fundamental design decision for API tools is the level of abstraction. A low-level approach provides a generic HTTP request tool that accepts a URL, method, headers, and body. The model constructs the entire API request, giving it maximum flexibility but requiring it to know API-specific details like authentication headers, request formats, and endpoint URLs. This approach works for developers and power users but produces unreliable results for agents because models must generate complex, provider-specific request structures from descriptions alone.

A high-level approach wraps each API operation in a dedicated tool with specific parameters. Instead of a generic HTTP tool, the agent has "search_web", "send_email", "get_weather", and "create_calendar_event" tools. Each tool handles the HTTP details internally: constructing the correct URL, adding authentication headers, formatting the request body, and parsing the response. The model only needs to provide the semantic parameters (search query, email recipient, city name, event details), and the tool handles the rest.

The high-level approach is recommended for production systems because it eliminates entire categories of errors. The model does not need to know API URLs, authentication mechanisms, or request body formats. It cannot accidentally expose API keys in arguments. It cannot construct malformed requests. The tool implementation handles all API-specific details, and the model interacts with a clean, semantic interface.

Authentication Management

API tools must manage authentication credentials securely, never exposing them to the model or including them in conversation logs. Credentials should be stored in environment variables, secrets managers, or credential vaults, and injected into API requests by the tool implementation, not by the model. If the model generates a tool call argument that looks like an API key, the validation layer should reject it immediately.

OAuth tokens require special handling because they expire and need periodic refresh. The tool implementation should manage the full OAuth lifecycle: storing refresh tokens securely, detecting expired access tokens, refreshing them automatically, and retrying the failed request with the new token. This complexity should be entirely invisible to the model, which should never see or handle OAuth tokens directly.

Per-user credentials enable agents to act on behalf of specific users when calling external APIs. A customer support agent might use the customer OAuth token to access their email or calendar. The tool implementation retrieves the appropriate credential for the current user, adds it to the request, and removes it from the result before returning it to the model. This ensures that credentials flow through the system without being exposed in the conversation or logs.

Rate Limiting and Throttling

External APIs impose rate limits that restrict the number of requests within a time window. API tools must respect these limits to avoid being blocked or charged overage fees. Rate limiting should be implemented at the tool level, tracking request counts per API and per time window, and queuing or rejecting requests that would exceed the limit.

Different APIs have different rate limiting schemes. Some use simple request-per-minute limits. Others use token bucket algorithms that allow bursts. Some rate limit by endpoint, and others rate limit by API key. The tool implementation must understand and respect the specific rate limiting scheme of each API it integrates with.

When a rate limit is hit, the tool should return a clear error message to the model rather than silently waiting. The message should indicate when the limit will reset and suggest alternative approaches: "Weather API rate limit reached (60 requests per minute). Limit resets in 45 seconds. Consider using cached weather data or asking the user to wait." This lets the model make an informed decision about whether to wait, use cached data, or inform the user.

Response Parsing and Formatting

Raw API responses are often too large, too complex, or too noisy for direct consumption by the model. Response parsing extracts the relevant data from the raw response and formats it for model consumption. A weather API that returns 50 fields of meteorological data should be parsed down to the 5 to 10 fields that are relevant to user queries: temperature, condition, humidity, wind speed, and forecast.

Error responses from external APIs should be translated into consistent, informative messages for the model. HTTP status codes, provider-specific error formats, and raw error messages vary widely across APIs. The tool should normalize these into a consistent format that the model can interpret: status (success or error), error code (if applicable), human-readable message, and suggested next action.

Large response bodies should be summarized or paginated before being returned to the model. A search API that returns 100 results should be limited to the top 10 with metadata indicating that more results are available. A data API that returns megabytes of JSON should be filtered to include only the requested fields. Context window space is a precious resource, and every unnecessary token in a tool result is a token unavailable for reasoning.

Webhook and Async Patterns

Some API operations are long-running and return results asynchronously through webhooks or polling. Report generation, batch processing, and complex analyses might take minutes or hours to complete. API tools for these operations should return immediately with a job identifier, and the agent should have a separate tool to check the job status and retrieve results when they are ready.

Webhook-based APIs push results to a callback URL when processing is complete. The tool implementation should register a webhook endpoint, associate incoming results with the originating agent session, and make the results available through a polling tool. The model does not need to know about webhooks, it simply calls the submit tool and then the check-status tool until the result is ready.