Calling a REST API in Python Without Getting It Wrong

Calling a REST API in Python is straightforward to begin but surprisingly easy to get wrong in ways that only surface under real production conditions. This article is a practical, mistake-centered guide that walks developers through the decisions and patterns where real bugs are made: authentication, error handling, retries, session reuse, and response parsing. Each section identifies what the wrong approach looks like, why it fails silently or unpredictably, and what the correct implementation looks like with working code. The goal is to help developers move from code that runs in a demo to code that holds up against slow endpoints, flaky uptime, expired tokens, and malformed payloads.

Making a REST API call in Python: easy to start, hard to get right

Five lines of code is all it takes to make a REST API call in Python: import requests, pass a URL, read the response. That simplicity is useful for getting started, but it hides a dozen silent failure modes that only appear under real conditions: slow endpoints, flaky uptime, auth expiry, malformed payloads.

This guide is for developers at any level who have made at least one API call in Python and want to move from "it runs" to "it holds up." Each section ahead covers one category of mistake, framed as a decision point: the wrong path that feels fine until it isn't, and the correct path that costs you two extra lines of code.

Choosing between requests, HTTPX, and aiohttp

There are three realistic choices in 2026, and picking one is less about which is best and more about which job you are doing.

requests is synchronous, has the broadest third-party ecosystem, and is the right tool for scripts, prototypes, and low-concurrency workloads, covering most REST API consumption needs. HTTPX gives you the same mental model as requests but with both sync and async support, plus HTTP/2. It is the library under the hood of the OpenAI and Anthropic Python SDKs, and it powers FastAPI's TestClient, so if async or HTTP/2 matters to you, this upgrade path is worth taking. aiohttp finishes large batches of requests noticeably faster than HTTPX in benchmark testing, though how much faster depends on your hardware and network conditions.

Reaching for requests by default is a sound engineering instinct for the overwhelming majority of use cases. The mistakes covered in this guide are written against requests, since that is where most developers start, and HTTPX is flagged wherever its behavior diverges. This section is about consuming APIs, not building them, so no framework recommendations are made here.

Skipping raise_for_status() and silent 4xx/5xx errors as the hardest bugs to trace

requests does not raise an exception when a server responds with a 404 or a 500. You get a response object back, status code and all, and your code keeps running as if nothing happened.

The consequence is that downstream code treats an error payload as if it were real data, and any crash that results appears somewhere completely unrelated to the API call that caused it. The root cause becomes difficult to find because the failure site and the error site are far apart in the call stack.

The fix costs one line: call response.raise_for_status() before touching .json() or anything else on the response. Always, every time.

resp = requests.get(url, timeout=5)
resp.raise_for_status() # do this before anything else
data = resp.json()

Skipping that line means the bug you are debugging next week will not look anything like the bug you actually have.

Catching exceptions correctly: the difference between HTTPError, ConnectionError, & bare Exception

Catching all exceptions with a bare except Exception swallows real bugs in your own code and makes them indistinguishable from API failures.

Learning the requests exception hierarchy by name matters, because each exception tells you something different about what failed. HTTPError comes from raise_for_status() catching a 4xx or 5xx. ConnectionError means the network itself failed: DNS did not resolve, the host was unreachable, or something failed at the transport level.

Write separate except blocks for each, and log something specific in each one. A single catch-all with a generic message provides no actionable information when diagnosing failures.

try:
 resp = session.get(url, timeout=5)
 resp.raise_for_status()
except requests.exceptions.ConnectionError:
 log.warning("network unreachable, worth a retry")
except requests.exceptions.HTTPError as e:
 log.error(f"API returned {e.response.status_code}")

That distinction also matters for retry logic. A ConnectionError might resolve itself if you try again after a short delay. A 401 will not resolve itself regardless of how many retries you attempt.

Why every request needs an explicit timeout

By default, requests waits forever. No timeout is set unless you set one, which means a request to a server that never responds will hang indefinitely, tying up the thread or, in async code, blocking the event loop.

The fix is to pass timeout as a tuple: timeout=(connect_timeout, read_timeout). The connect timeout covers how long you will wait for the initial TCP handshake. The read timeout covers how long you will wait for data once the connection is open. A single float sets both to the same value, which is simpler but less precise, so production code generally benefits from the tuple form.

Realistic values: connect timeouts in the low single digits of seconds are common, since a handshake either succeeds quickly or something is wrong. Read timeouts depend heavily on what the endpoint does. An endpoint returning a cached value should respond in well under a second. An endpoint running a heavy report might reasonably take longer. Match the timeout value to the actual behavior of the endpoint rather than copying a value from a tutorial.

resp = requests.get(url, timeout=(3, 10))

When the clock runs out, requests.exceptions.Timeout is raised, and it fits cleanly into the exception handling pattern described in the section above.

Using json= instead of data= for JSON payloads and the silent 400 the wrong choice gives you

Passing data=json.dumps(payload) sends the correct bytes over the wire, but it sets the Content-Type header to application/x-www-form-urlencoded rather than application/json. Most REST APIs read that header, assume they are receiving form data, and return a 400 with an error message that gives you no useful diagnostic information.

Use json=payload instead. The requests library handles serialization and sets Content-Type: application/json automatically.

# wrong: silent 400, wrong Content-Type
requests.post(url, data=json.dumps(payload))

# right: correct header, correct serialization
requests.post(url, json=payload)

What makes this bug difficult to catch is that everything appears normal on the surface. The request sends. A status code comes back. The error is typically a 400, and the server's error message usually says something about a missing field rather than an incorrect Content-Type, which directs your attention to the payload when the actual problem is the header.

A related issue: do not assume a 200 status means the response body contains JSON. Some APIs return an HTML error page or plain text even with a 200 status code, particularly behind proxies or load balancers experiencing problems. Call response.json() inside a try/except that catches ValueError or json.JSONDecodeError.

What Session() does that repeated get() calls do not

Calling requests.get() or requests.post() in a loop opens a brand-new TCP connection for each call and closes it immediately after. That produces a large number of unnecessary handshakes when you are making dozens or hundreds of calls to the same host.

Session() pools connections, reusing the same TCP connection to a given host across multiple calls. It also lets you set headers once (authentication tokens, for instance) rather than attaching them to every individual call. And it persists cookies automatically, which is necessary for any API using session-based authentication.

Use it as a context manager so connections are released cleanly when you are done:

with requests.Session() as session:
 session.headers.update({"Authorization": "Bearer TOKEN"})
 resp = session.get(url, timeout=(3, 10))

This is also the correct place to mount retry logic, since an HTTPAdapter attaches directly to a Session. The two patterns reinforce each other, as the next section shows.

Retry logic: what to retry, what never to retry, & avoiding hammering a failing service

Retrying everything is a common instinct and an incorrect one. Retrying a 401 or a 404 accomplishes nothing because those errors do not resolve themselves with additional attempts.

What is safe to retry: connection errors, 429 (Too Many Requests), and 5xx server errors. Exponential backoff doubles the wait between retries (1s, 2s, 4s, 8s), and jitter adds a small random offset so multiple clients do not retry in lockstep after a shared outage. urllib3.util.Retry exposes both via backoff_factor and, in urllib3 2.0+, backoff_jitter.

What is never safe to retry blindly: any 4xx besides 429, and POST requests in general, because POST is not idempotent and a retry can silently create a duplicate record. If you need to retry a POST, do so only for connection errors, or use an Idempotency-Key header if the API supports one. GET, HEAD, OPTIONS, PUT, and DELETE are all idempotent, so retrying them carries none of that risk.

There are two solid approaches for implementing retry logic. urllib3.util.Retry, mounted onto a Session via HTTPAdapter, comes built into the ecosystem with no extra dependency and honors Retry-After headers natively. It is the right choice when you want one retry policy applied consistently across every request on a session. tenacity is the alternative when you need more control: retrying only on specific exceptions, checking a predicate against the response body, or working inside async code.

Cap retries at 3 to 5 attempts to avoid adding pressure to an already-overloaded service. If the server sends back a Retry-After header, honor it precisely.

from requests.adapters import HTTPAdapter
from urllib3.util import Retry

retry = Retry(total=4, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504])
session.mount("https://", HTTPAdapter(max_retries=retry))

Authentication patterns that hold up in production

Start with the non-negotiable: HTTPS, always. No authentication scheme is safe over plain HTTP.

For Bearer token auth, set the header once on the Session object via headers.update({"Authorization": "Bearer TOKEN"}). Rebuilding that header on every individual call is unnecessary work and an easy place to introduce errors. Hardcoding credentials directly into source code is a separate and costly mistake. Use environment variables or a secrets manager and never allow a token to be committed to version control.

API key authentication has its own quiet failure mode: some APIs expect the key in a query parameter, others expect it as a header, and the documentation is the only reliable way to determine which. Getting it wrong produces a 401 that looks identical to an expired token, which makes the actual problem harder to identify.

OAuth tokens expire, and code that does not account for expiry fails silently after a period of inactivity. Everything works correctly for hours and then quietly stops. Catch 401 responses, trigger a token refresh, and only re-raise if the refresh itself fails. If you are building an API rather than consuming one, never combine Access-Control-Allow-Origin: * with credentials, as that combination undermines the purpose of scoping access.

Putting the patterns together in a minimal production-grade client

Stacking every pattern above together produces a client with the following characteristics: a Session for connection pooling, a timeout on every call, raise_for_status() before touching any response data, json= for POST bodies, retry logic mounted through HTTPAdapter with exponential backoff, and exception handling that is specific at each call site rather than one broad catch-all.

import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

def make_client(base_url: str, token: str) -> requests.Session:
 session = requests.Session()
 session.headers.update({"Authorization": f"Bearer {token}"})
 retry = Retry(
 total=4,
 backoff_factor=1,
 status_forcelist=[429, 500, 502, 503, 504],
 allowed_methods=["GET", "HEAD", "OPTIONS", "PUT", "DELETE"],
 )
 session.mount("https://", HTTPAdapter(max_retries=retry))
 return session

with make_client("https://api.example.com", token="...") as session:
 try:
 resp = session.get("/orders/42", timeout=(3, 10))
 resp.raise_for_status()
 data = resp.json()
 except requests.exceptions.HTTPError as e:
 log.error(f"API error: {e.response.status_code}")
 except requests.exceptions.ConnectionError:
 log.warning("network issue, consider retry")

This is a wrapper intended to be dropped into an existing project and adjusted to fit. It deliberately omits OAuth refresh logic and async support, both of which require their own implementation. The Session-as-context-manager pattern is the default throughout because it guarantees connections are released and nothing leaks.

Async workloads & choosing between HTTPX and aiohttp when requests is not enough

Everything above assumes one request at a time. That assumption breaks when you are firing off dozens or hundreds of outbound calls concurrently: fan-out patterns, batch processing jobs, or ML inference pipelines calling a model API for every row in a dataset. Synchronous requests blocks the thread between each call, and at that volume the blocking overhead accumulates quickly.

HTTPX is the natural migration path because it mirrors the requests API closely enough that most of the patterns above translate directly, while adding async support and HTTP/2. It is the library behind the OpenAI and Anthropic Python SDKs and powers FastAPI's TestClient. aiohttp outperforms HTTPX on large batches of concurrent calls in benchmark conditions, making it the better choice when throughput is the primary concern.

The failure modes do not change significantly in async code. A missing timeout does not block a thread; it blocks the event loop, which is worse because it can stall every other coroutine running alongside it. The same discipline applies regardless of which library is in your import statement: set a timeout, check the status before trusting the body, catch the specific exception that actually occurred, and retry only what is safe to retry.

FastAPI is the framework context where async API clients appear most often. It is the highest-ranked Python web framework in the JetBrains State of Python 2025 survey, with 35% Django usage and 20% DRF usage providing context for the ecosystem's overall size.