Build a Python API Response Validator With Error Logging

by ggbond0821 in Circuits > Computers

12 Views, 0 Favorites, 0 Comments

Build a Python API Response Validator With Error Logging

1700.png

Getting an HTTP 200 response feels reassuring, but it only means that the server successfully handled the HTTP request. The JSON body could still be empty, malformed, or missing the field your program needs.

I ran into this while experimenting with public APIs. A short script that directly accessed response.json()["data"][0]["price"] worked during the first test, but it was far too easy to break.

In this project, we will build a small Python tool that:

  1. Sends a request with explicit timeouts
  2. Checks the HTTP status
  3. Confirms that the response is valid JSON
  4. Handles more than one top-level response structure
  5. Validates the symbol and price fields
  6. Preserves price precision with Decimal
  7. Records successful checks and errors in a log file

I use a public futures ticker endpoint as the test data source, but the same structure can be adapted to weather, transport, inventory, or other JSON APIs.

Supplies

You will need:

  1. A computer running Windows, macOS, or Linux
  2. Python 3.9 or newer
  3. A text editor, such as Visual Studio Code
  4. An internet connection
  5. The Python Requests package
  6. A terminal or command prompt

No API key is required for the public endpoint used in this project.

Create the Project and Install Requests

Create a new folder named api-validator, open a terminal inside it, and create a virtual environment:

python -m venv .venv

On Windows, activate it with:

.venv\Scripts\activate

On macOS or Linux, use:

source .venv/bin/activate

Install Requests:

python -m pip install requests

Finally, create a file named:

api_validator.py

Using a virtual environment keeps this project’s packages separate from other Python projects on the computer.

Add the Configuration and Logger

Open api_validator.py and add the following code:

from datetime import datetime, timezone
from decimal import Decimal, InvalidOperation
import logging

import requests


API_URL = "https://api.bydfi.com/api/v1/fapi/market/ticker/price"
SYMBOL = "BTC-USDT"

logging.basicConfig(
filename="api_validator.log",
level=logging.INFO,
format="%(asctime)s | %(levelname)s | %(message)s",
)

logger = logging.getLogger(__name__)

The logger will create a file named api_validator.log in the project folder.

The endpoint used here is part of BYDFi’s public market-data API. Notice the fapi section of the path: it identifies this as a futures endpoint. The returned value should not automatically be described as a spot-market price.

The symbol is stored separately so it can be changed without editing the validation functions.

Request JSON Safely

Below the configuration, add this function:

def request_json(url: str, params: dict):
try:
response = requests.get(
url,
params=params,
timeout=(3, 5),
)
response.raise_for_status()
return response.json()

except requests.exceptions.Timeout as exc:
raise RuntimeError("The API request timed out") from exc

except requests.exceptions.HTTPError as exc:
status = (
exc.response.status_code
if exc.response is not None
else "unknown"
)
raise RuntimeError(
f"The API returned HTTP status {status}"
) from exc

except requests.exceptions.JSONDecodeError as exc:
raise RuntimeError(
"The response body was not valid JSON"
) from exc

except requests.exceptions.RequestException as exc:
raise RuntimeError(
f"The network request failed: {exc}"
) from exc

The timeout tuple contains two separate values:

  1. Three seconds for establishing the connection
  2. Five seconds for waiting between received data

It does not guarantee that the entire operation will always finish within eight seconds.

raise_for_status() converts unsuccessful HTTP responses into exceptions, while the JSON exception prevents an HTML error page or empty response from being processed as data.

Normalize the Response Structure

Some APIs return a list directly. Others place the list inside an object containing fields such as code, message, and data.

Add this function below request_json():

def extract_records(payload) -> list:
if isinstance(payload, list):
records = payload

elif isinstance(payload, dict):
application_code = payload.get("code")

if (
application_code is not None
and application_code != 200
):
message = payload.get(
"message",
"Unknown application error",
)
raise ValueError(
f"API application error: {message}"
)

records = payload.get("data")

else:
raise ValueError(
"Unexpected top-level JSON structure"
)

if not isinstance(records, list):
raise ValueError(
"The response does not contain a record list"
)

if not records:
raise ValueError(
"The response contains no records"
)

return records

This function separates HTTP success from application-level success. Even when the server returns HTTP 200, the payload may still report an error or contain no usable records.

Accepting two known top-level structures also makes the program less dependent on a single response example. It does not mean that every possible response should be accepted.

Validate the Price Record

Now add a function that validates the first record:

def validate_price_record(
record: dict,
expected_symbol: str,
) -> tuple[str, Decimal]:
if not isinstance(record, dict):
raise ValueError(
"The price record must be a JSON object"
)

returned_symbol = record.get("symbol")
raw_price = record.get("price")

if not isinstance(returned_symbol, str):
raise ValueError(
"The record contains an invalid symbol"
)

if returned_symbol != expected_symbol:
raise ValueError(
f"Expected {expected_symbol}, "
f"but received {returned_symbol}"
)

if not isinstance(raw_price, str):
raise ValueError(
"The price must be encoded as a string"
)

try:
price = Decimal(raw_price)
except InvalidOperation as exc:
raise ValueError(
"The price is not a valid decimal value"
) from exc

if not price.is_finite() or price <= 0:
raise ValueError(
"The price must be a positive finite value"
)

return returned_symbol, price

Financial APIs frequently encode prices as strings. Using Decimal preserves the decimal value more predictably than immediately converting it to a binary floating-point number.

The function also checks that the returned symbol matches the requested symbol. This prevents the application from silently displaying a valid price for the wrong instrument.

Connect the Functions

Add the main function at the bottom of the file:

def main() -> None:
checked_at = datetime.now(timezone.utc).isoformat()

try:
payload = request_json(
API_URL,
{"symbol": SYMBOL},
)

records = extract_records(payload)

symbol, price = validate_price_record(
records[0],
SYMBOL,
)

except (RuntimeError, ValueError) as exc:
logger.error(
"Validation failed at %s | %s",
checked_at,
exc,
)
print(f"Validation failed: {exc}")
return

logger.info(
"Validation succeeded at %s | symbol=%s",
checked_at,
symbol,
)

print("Validation succeeded")
print(f"Symbol: {symbol}")
print(f"Price: {price}")
print(f"Checked at: {checked_at}")


if __name__ == "__main__":
main()

The current value is printed to the terminal, but it is deliberately not written to the log. For many applications, the log only needs to show whether validation succeeded and provide enough information to investigate failures.

Run the program with:

python api_validator.py

A successful run should print:

Validation succeeded
Symbol: BTC-USDT
Price: [current value]
Checked at: [current UTC timestamp]

The exact price will naturally change over time.

Test the Error Handling

A validator is not very useful if we only test the successful path.

Try changing:

SYMBOL = "BTC-USDT"

to an invalid value and run the program again. Depending on the API response, the script should report an HTTP, application, empty-record, or symbol-validation error instead of crashing with an unclear traceback.

You can also temporarily change the expected symbol passed to validate_price_record() to confirm that the mismatch check works.

After testing, open:

api_validator.log

You should see entries similar to:

INFO | Validation succeeded at ...
ERROR | Validation failed at ...

Restore the correct symbol after finishing the tests.

Avoid repeatedly sending requests in a fast loop. API limits may differ between public market-data endpoints and authenticated trading operations, so polling behavior should always be based on the current documentation for the specific endpoint.

Ideas for Improving the Project

This is a small validator, but it provides a useful foundation. Possible improvements include:

  1. Adding limited retries with exponential backoff
  2. Saving validated records to CSV
  3. Validating every record instead of only the first
  4. Defining the expected schema in a separate configuration file
  5. Adding automated tests with saved sample responses
  6. Reading the endpoint and symbol from command-line arguments
  7. Rotating log files so they do not grow indefinitely

The important lesson is that receiving JSON is not the same as receiving trustworthy application data. A reliable integration checks the transport status, response structure, business status, field types, and numerical validity before using a value.