API Returning 500 Errors? How to Debug REST API Issues Systematically

TL;DR

When your REST API returns 500 errors, follow this order: reproduce the issue reliably, isolate the failing component, and trace the request through your stack using logs and curl. Most 500 errors come from unhandled exceptions, database connection failures, or misconfigured environment variables. Check your server logs first — the answer is almost always there.

Prerequisites

• Terminal access to your server (SSH or direct)
• curl installed (available on all major OS)
• Access to server logs (application and web server)
• Optional: Postman or HTTPie
• Basic understanding of HTTP and JSON

Step 1: The Systematic Approach — Reproduce, Isolate, Trace

Reproduce

Before debugging anything, you need a reliable way to trigger the error. Document the exact request that fails:

# Save the failing request so you can replay it
curl -v -X POST https://api.example.com/v1/users \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"name": "Test", "email": "test@example.com"}' \
  2>&1 | tee /tmp/api-debug.log

Ask yourself: Does it fail every time? Only with certain data? Only at certain times? Intermittent failures often point to resource exhaustion or race conditions.

Isolate

Narrow down which component is failing. A typical REST API stack has multiple layers:

Client → DNS → Load Balancer → Reverse Proxy (Nginx) → App Server → Database

Test each layer independently. Can you reach Nginx? Can the app server connect to the database? Use targeted checks:

# Test if Nginx is responding at all
curl -I https://api.example.com/health

# Test the app server directly, bypassing the reverse proxy
curl -I http://localhost:3000/health

# Test database connectivity from the app server
mysql -u appuser -p -h db-host -e "SELECT 1;"
# or for PostgreSQL
psql -U appuser -h db-host -c "SELECT 1;"

Trace

Follow the request through every layer. Enable verbose logging temporarily and use request IDs if your application supports them:

# Add a custom request ID to trace through logs
curl -v -X GET https://api.example.com/v1/orders \
  -H "X-Request-ID: debug-12345" \
  -H "Authorization: Bearer YOUR_TOKEN"

Then search your logs for that ID across all services.

Step 2: Reading Server Logs

Logs are your primary debugging tool. Where they live depends on your setup:

systemd Services (journalctl)

# View logs for a specific service, most recent first
journalctl -u my-api-service -n 100 --no-pager

# Follow logs in real-time (like tail -f)
journalctl -u my-api-service -f

# Filter by time range
journalctl -u my-api-service --since "2024-01-15 14:00" --until "2024-01-15 15:00"

# Show only errors
journalctl -u my-api-service -p err -n 50

Docker Containers

# View last 200 lines of a container's logs
docker logs --tail 200 my-api-container

# Follow logs in real-time with timestamps
docker logs -f --timestamps my-api-container

# For docker-compose setups
docker-compose logs -f --tail=100 api

PM2 (Node.js)

# View logs for a specific app
pm2 logs my-api --lines 100

# View only error logs
pm2 logs my-api --err --lines 50

# Flush and start fresh
pm2 flush my-api

Traditional Log Files

# Nginx error log
tail -f /var/log/nginx/error.log

# Nginx access log — filter for 500 errors
grep ' 500 ' /var/log/nginx/access.log | tail -20

# Application logs (common locations)
tail -f /var/log/myapp/error.log
tail -f /var/log/syslog | grep myapp

Step 3: Testing with curl

curl is the most versatile tool for API debugging. Master these patterns:

Basic Requests with Verbose Output

# GET request with full headers shown (-v for verbose)
curl -v https://api.example.com/v1/users

# Only show response headers
curl -I https://api.example.com/v1/users

# Show response headers AND body
curl -i https://api.example.com/v1/users

POST Requests with JSON Data

# POST with JSON body
curl -X POST https://api.example.com/v1/users \
  -H "Content-Type: application/json" \
  -d '{"name": "Jane Doe", "email": "jane@example.com", "role": "admin"}'

# POST with data from a file
curl -X POST https://api.example.com/v1/import \
  -H "Content-Type: application/json" \
  -d @payload.json

# PUT request to update a resource
curl -X PUT https://api.example.com/v1/users/42 \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer eyJhbGciOi..." \
  -d '{"name": "Jane Smith"}'

Debugging Connection Issues

# Show timing information
curl -o /dev/null -s -w "DNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTLS: %{time_appconnect}s\nFirstByte: %{time_starttransfer}s\nTotal: %{time_total}s\nHTTP Code: %{http_code}\n" https://api.example.com/v1/health

# Resolve to a specific IP (bypass DNS)
curl --resolve api.example.com:443:10.0.0.5 https://api.example.com/v1/health

# Set a custom timeout (5 seconds)
curl --max-time 5 --connect-timeout 3 https://api.example.com/v1/slow-endpoint

Step 4: Understanding HTTP Status Codes

Each status code tells a specific story. Here is what they mean in practice:

400 Bad Request

Meaning: The server cannot process the request due to malformed syntax or invalid data.

Common causes: Missing required fields, wrong data types, invalid JSON syntax, exceeding field length limits.

# This will likely return 400 — invalid JSON
curl -X POST https://api.example.com/v1/users \
  -H "Content-Type: application/json" \
  -d '{name: "missing quotes on key"}'

401 Unauthorized

Meaning: Authentication is missing or invalid. The server does not know who you are.

Common causes: Missing Authorization header, expired token, wrong API key.

403 Forbidden

Meaning: Authentication succeeded, but you lack permission for this action. The server knows who you are but you are not allowed.

Common causes: Insufficient role/scope, IP allowlist restrictions, resource owned by another user.

404 Not Found

Meaning: The requested resource does not exist at this URL.

Common causes: Typo in URL, resource was deleted, wrong API version prefix, missing trailing slash.

500 Internal Server Error

Meaning: The server encountered an unexpected condition. This is always a server-side bug.

Common causes: Unhandled exceptions, null pointer errors, missing environment variables, database connection failures, out-of-memory conditions.

502 Bad Gateway

Meaning: The reverse proxy (Nginx, Apache) received an invalid response from the upstream application server.

Common causes: App server crashed, app server not running, wrong upstream port in proxy config.

503 Service Unavailable

Meaning: The server is temporarily unable to handle requests.

Common causes: Server is starting up, maintenance mode, overloaded, dependency (database) is down.

504 Gateway Timeout

Meaning: The reverse proxy timed out waiting for the upstream server to respond.

Common causes: Slow database queries, external API calls hanging, insufficient proxy timeout settings.

Step 5: Debugging CORS Errors

Cross-Origin Resource Sharing (CORS) errors only occur in browsers. If your API works in curl but fails in the browser with "CORS policy" errors, the fix is on the server side.

How to Identify CORS Issues

# Simulate a CORS preflight request
curl -v -X OPTIONS https://api.example.com/v1/users \
  -H "Origin: https://app.example.com" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: Content-Type, Authorization"

# Check the response for these headers:
# Access-Control-Allow-Origin: https://app.example.com
# Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
# Access-Control-Allow-Headers: Content-Type, Authorization

Nginx CORS Fix

# /etc/nginx/conf.d/api.conf
server {
    listen 443 ssl;
    server_name api.example.com;

    location /v1/ {
        # CORS headers
        add_header Access-Control-Allow-Origin "https://app.example.com" always;
        add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS" always;
        add_header Access-Control-Allow-Headers "Content-Type, Authorization, X-Request-ID" always;
        add_header Access-Control-Max-Age 86400 always;

        # Handle preflight requests
        if ($request_method = OPTIONS) {
            return 204;
        }

        proxy_pass http://localhost:3000;
    }
}

Express.js CORS Fix

const cors = require('cors');

app.use(cors({
  origin: ['https://app.example.com', 'https://staging.example.com'],
  methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
  allowedHeaders: ['Content-Type', 'Authorization', 'X-Request-ID'],
  credentials: true,
  maxAge: 86400
}));

Important: Never use Access-Control-Allow-Origin: * together with credentials: true. Browsers will reject this combination. Always specify the exact origin.

Step 6: Timeout Issues

Timeouts can occur at every layer. Identify where the timeout happens:

# Measure response time breakdown
curl -w "\n---Timing---\nDNS:        %{time_namelookup}s\nConnect:    %{time_connect}s\nTLS:        %{time_appconnect}s\nFirstByte:  %{time_starttransfer}s\nTotal:      %{time_total}s\n" \
  -o /dev/null -s https://api.example.com/v1/reports/generate

If time_starttransfer is high but time_connect is low, the server is slow to process. If time_connect is high, there is a network or DNS issue.

Common Timeout Configurations

# Nginx — increase proxy timeout for slow endpoints
location /v1/reports/ {
    proxy_pass http://localhost:3000;
    proxy_read_timeout 120s;
    proxy_connect_timeout 10s;
    proxy_send_timeout 60s;
}

# Node.js — increase server timeout
const server = app.listen(3000);
server.setTimeout(120000); // 120 seconds

Step 7: JWT and OAuth Debugging

Authentication issues are among the most common API problems. Here is how to inspect tokens:

Inspecting a JWT Token

A JWT has three parts separated by dots: header.payload.signature. You can decode the first two parts without the secret:

# Decode a JWT payload in the terminal (no secret needed)
echo 'eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTUxNjIzOTAyMiwiZXhwIjoxNTE2MjQyNjIyfQ.signature' \
  | cut -d'.' -f2 \
  | base64 -d 2>/dev/null | python3 -m json.tool

Check for these common issues:

• exp — Is the token expired? Compare with current Unix time: date +%s
• iss — Does the issuer match what the server expects?
• aud — Does the audience claim match your API?
• scope/roles — Does the token carry the permissions you need?

You can also paste tokens into jwt.io for a visual breakdown (never paste production tokens with sensitive data).

Testing OAuth Flows

# Request a new token using client credentials grant
curl -X POST https://auth.example.com/oauth/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials" \
  -d "client_id=YOUR_CLIENT_ID" \
  -d "client_secret=YOUR_CLIENT_SECRET" \
  -d "scope=read write"

# Use the returned access_token
curl -H "Authorization: Bearer ACCESS_TOKEN_HERE" \
  https://api.example.com/v1/protected-resource

Step 8: Postman and HTTPie

HTTPie — A Human-Friendly curl Alternative

# Install
brew install httpie    # macOS
apt install httpie     # Ubuntu/Debian

# GET request (automatically pretty-prints JSON)
http GET https://api.example.com/v1/users Authorization:"Bearer TOKEN"

# POST with JSON (no need to specify Content-Type)
http POST https://api.example.com/v1/users name="Jane" email="jane@example.com"

# Show both request and response headers
http -v POST https://api.example.com/v1/users name="Jane"

# Download a response to file
http --download https://api.example.com/v1/export

Postman Tips

• Use Environments to switch between dev/staging/production without changing requests
• Store tokens in environment variables: set {{base_url}} and {{token}}
• Use the Console (View → Show Postman Console) to see raw request/response data including redirects
• Use Pre-request Scripts to auto-refresh expired OAuth tokens
• Use Tests tab to validate response structure automatically

Troubleshooting Quick Reference

Symptom	Likely Cause	First Step
500 on all endpoints	App crash, missing env var, DB down	Check application logs
500 on one endpoint	Unhandled exception in that route	Check logs for stack trace
502 Bad Gateway	App server not running	systemctl status myapp
504 Gateway Timeout	Slow query or external call	Check curl timing, DB slow log
CORS error in browser	Missing response headers	Test with curl OPTIONS request
401 with valid token	Token expired or wrong audience	Decode JWT, check exp claim
Works in Postman, fails in browser	CORS or cookie issue	Check preflight response
Intermittent 500s	Resource exhaustion, race condition	Monitor memory/CPU, check connection pool

Prevention and Best Practices

• Structured logging: Use JSON logs with request IDs, timestamps, and context. Tools like ELK or Loki make searching trivial.
• Health check endpoints: Implement /health and /ready endpoints that verify database connectivity and dependency availability.
• Error handling middleware: Catch all unhandled exceptions at the framework level. Never leak stack traces to clients in production — log them server-side, return a generic error with a request ID.
• Monitoring and alerting: Set up alerts for 5xx rate spikes. A sudden increase in 500 errors after a deployment means you should roll back first, debug later.
• Request validation: Validate input at the API boundary using schema validation (e.g., Joi, Zod, JSON Schema). Return clear 400 errors with descriptions of what is wrong.
• Timeouts everywhere: Set explicit timeouts on database queries, external HTTP calls, and proxy configurations. Never rely on defaults — they are often too high or infinite.
• Retry with backoff: When calling external services, implement exponential backoff. Never retry on 4xx errors — only on 5xx and network errors.