How to Create Custom Tools for AI Agents

Define the Tool Interface

Start by defining what the tool does and what the model needs to provide. Write out the function signature as if you were designing an API endpoint: what parameters does it accept, which are required, what types do they use, and what does it return. The interface should be as simple as possible while covering the use case completely.

Keep the parameter count low. Tools with 1 to 4 parameters have the highest accuracy rates. Tools with more than 6 parameters see significant drops in argument accuracy. If you need more parameters, consider splitting the tool into multiple simpler tools or using a two-step process where the first call determines which parameters are needed and the second call provides them.

Choose parameter types that constrain the model output. Enums are better than free-form strings for fixed sets of values. Integers with min/max constraints are better than unconstrained numbers. Boolean parameters are better than string parameters that accept "yes", "no", "true", "false", and variations. Every constraint you add to the schema is a potential error you prevent.

Implement the Execution Logic

The execution function should follow a clear structure: validate inputs, perform the operation, format the result, handle errors. Start with input validation that goes beyond the JSON Schema checks. Schema validation ensures correct types, but your code should verify semantic correctness: is the date in the valid range, does the referenced ID exist, is the operation permitted for the current user.

Wrap external calls (API requests, database queries, file operations) in try-catch blocks with specific error handling for common failure modes. Network timeouts, rate limit responses, authentication failures, and resource-not-found errors each require different handling. Generic catch-all error handlers lose important context about what went wrong.

Format results as clean JSON with descriptive field names. The model reads these results and must understand them without additional context. A result with fields like {"t": 22, "c": "PC"} is much harder for the model to interpret than {"temperature_celsius": 22, "condition": "partly cloudy"}. Clear field names in results pay for themselves in improved model response quality.

Write Comprehensive Descriptions

The tool description should answer: what does this tool do, when should the model use it, what does it return, and when should the model NOT use it. Cover all four aspects explicitly. Models that know when not to use a tool make fewer incorrect invocations, which saves tokens and improves user experience.

Parameter descriptions should include format examples. Instead of "the user email address" write "the user email address in standard format, for example user@domain.com". Instead of "the date to search from" write "the start date in YYYY-MM-DD format, for example 2026-01-15, must be within the last 365 days". Examples dramatically improve argument accuracy.

Add Validation and Error Handling

Implement multiple layers of validation. The first layer is JSON Schema validation that checks types, required fields, and basic constraints. The second layer is semantic validation that checks business rules, referential integrity, and cross-parameter consistency. The third layer is runtime validation within the execution function that checks preconditions before performing the operation.

Error messages returned to the model should be specific and actionable. "Invalid parameter" is useless. "The 'start_date' parameter value '2026-13-01' is not a valid date because month 13 does not exist. Please provide a date in YYYY-MM-DD format with a valid month (01-12)." gives the model everything it needs to fix the error on the next attempt.

Test with Real Model Interactions

Unit testing the execution function verifies that it handles inputs correctly, but it does not test the most critical aspect: whether the model can use the tool effectively. Testing with real model interactions involves sending conversations that should trigger the tool and verifying that the model selects the tool, generates correct arguments, and interprets the result appropriately.

Create a test suite of 10 to 20 representative user messages that should trigger the tool. For each message, verify that the model calls the tool (not a different tool or no tool), generates arguments that match expectations, and produces a correct final response based on the tool result. Also test edge cases: messages that are similar to but should not trigger the tool, messages that should trigger the tool with unusual parameter values, and messages where the tool should be one of several tools called.

Deploy and Monitor

In production, instrument the tool with logging that captures every invocation: timestamp, calling agent, arguments, result, latency, and any errors. Use these logs to calculate key metrics: invocation frequency (how often is the tool used), argument accuracy (how often are arguments valid on the first attempt), execution success rate (how often does the function complete without errors), and result usefulness (does the model use the result effectively in its response).

Set up alerts for anomalous patterns: sudden spikes in invocation frequency (might indicate a prompt injection attack or a bug causing repeated calls), drops in argument accuracy (might indicate a model update changed behavior), or increases in execution errors (might indicate an external dependency failure). Early detection of these patterns prevents them from affecting users.