Why Python Web Apps Need Bot Detection
Python-powered web applications (Flask, FastAPI, Django, Bottle) are prime targets for bot attacks. Your API endpoints collect user data, handle authentication, process payments, and enable content creation. Without bot detection, attackers can:
- Scrape your data at scale using Python's
requestslibrary or Scrapy - Credential-stuff login endpoints with stolen username/password combinations
- Brute-force account enumeration APIs
- Spam your forms, comments, and user-generated content platforms
- Abuse your free tier or API rate limits by automating requests
- Perform fake account registration to resell on dark markets
Python's simplicity makes it the attacker's first choice for automation. A single Python script using requests + threading can hit your API with thousands of requests per second. This guide shows you how to detect and block bot traffic in Python web applications—with real Flask and FastAPI code, a free tier (1,000 daily verifications), and zero CAPTCHA friction.
The Challenge: Detecting Bots in Python
Unlike JavaScript frameworks (React, Vue) that run in browsers and can collect device fingerprints, Python web applications receive API requests from anywhere—browsers, mobile apps, desktop clients, bots, and everything in between. You can't see the user's device or behavioral signals. You only have:
- HTTP request headers (User-Agent, Referer, Accept-Language, etc.)
- Client IP address
- Request timing and patterns
- Request body content
That's not enough. A sophisticated bot can spoof headers, use residential proxies, and mimic human timing. You need a service that can verify requests server-to-server, scoring bot likelihood based on multiple independent signals. That's where device.ai comes in.
Device.AI: Server-Side Bot Verification
Device.AI's verification API accepts a request from your backend and returns a bot score (0.0 = certain bot, 1.0 = certain human). Device.AI analyzes:
- IP reputation and geolocation
- Request headers and fingerprints
- Request patterns and timing
- Known bot signatures
- Behavioral anomalies
You decide what to do with the score: block, rate-limit, challenge, or allow. Device.AI's free tier gives you 1,000 daily verifications at no cost—enough for small to mid-size applications.
Step 1: Get Your Free API Key (1 minute)
Visit device.ai/v1/keys and generate a free API key. No signup required. No credit card. Your first 1,000 daily verifications are free.
You'll get a key like this:
dv_live_abc123def456ghi789jklStore it in your environment variables:
# .env or export DEVICE_AI_API_KEY
DEVICE_AI_API_KEY=dv_live_abc123def456ghi789jklStep 2: Flask Integration (Middleware Pattern)
For Flask, create a middleware that verifies every incoming request before it reaches your application logic:
# bot_detection.py
import os
import requests
from functools import wraps
from flask import request, jsonify, g
DEVICE_AI_API_KEY = os.getenv('DEVICE_AI_API_KEY')
DEVICE_AI_VERIFY_URL = 'https://device.ai/v1/verify'
class BotDetectionMiddleware:
"""
Flask middleware to detect bots on every incoming request.
"""
def __init__(self, app, threshold=0.5, protected_paths=None):
"""
Initialize bot detection middleware.
Args:
app: Flask application instance
threshold: Risk score threshold (0.0=bot, 1.0=human). Default 0.5 blocks scores below this.
protected_paths: List of URL paths to protect. If None, protects all paths.
"""
self.app = app
self.threshold = threshold
self.protected_paths = protected_paths or []
# Register before_request handler
app.before_request(self.check_bot)
def should_verify(self, path):
"""
Determine if this request should be bot-verified.
Args:
path: Request path
Returns:
Boolean: True if request should be verified
"""
if not self.protected_paths:
# No protection list means protect everything
return True
return any(path.startswith(p) for p in self.protected_paths)
def check_bot(self):
"""
Verify incoming request before processing.
"""
if request.method == 'OPTIONS':
# Skip preflight requests
return
if not self.should_verify(request.path):
# Path not in protected list
return
try:
# Call device.ai verification API
response = requests.post(
DEVICE_AI_VERIFY_URL,
json={
'ip': request.remote_addr,
'user_agent': request.headers.get('User-Agent', ''),
'headers': dict(request.headers),
},
headers={'Authorization': f'Bearer {DEVICE_AI_API_KEY}'},
timeout=5
)
if response.status_code == 200:
result = response.json()
bot_score = result.get('score', 0.5)
# Store result in Flask's g object for access in route handlers
g.device_ai_score = bot_score
g.device_ai_result = result
# Block if score below threshold (low score = bot)
if bot_score < self.threshold:
return jsonify({
'error': 'Request blocked: bot detected',
'score': bot_score,
}), 403
else:
# API error: fail open (allow the request, log incident)
print(f'Device.AI verification failed: {response.status_code}')
g.device_ai_score = 0.5 # Neutral score
except requests.Timeout:
# Timeout: fail open
print('Device.AI verification timeout')
g.device_ai_score = 0.5
except Exception as e:
# Any other error: fail open
print(f'Device.AI verification error: {e}')
g.device_ai_score = 0.5
def requires_human(threshold=0.5):
"""
Decorator for route handlers that require human verification.
Usage:
@app.route('/login', methods=['POST'])
@requires_human(threshold=0.6)
def login():
# Only humans reach here
...
"""
def decorator(f):
@wraps(f)
def decorated_function(*args, **kwargs):
try:
response = requests.post(
DEVICE_AI_VERIFY_URL,
json={
'ip': request.remote_addr,
'user_agent': request.headers.get('User-Agent', ''),
'headers': dict(request.headers),
},
headers={'Authorization': f'Bearer {DEVICE_AI_API_KEY}'},
timeout=5
)
if response.status_code == 200:
result = response.json()
bot_score = result.get('score', 0.5)
if bot_score < threshold:
return jsonify({
'error': 'Bot detected. Request blocked.',
'score': bot_score,
}), 403
g.device_ai_score = bot_score
g.device_ai_result = result
except Exception as e:
print(f'Verification error: {e}')
# Fail open
return f(*args, **kwargs)
return decorated_function
return decorator
Now integrate the middleware into your Flask app:
# app.py
from flask import Flask, jsonify, request, g
from bot_detection import BotDetectionMiddleware, requires_human
app = Flask(__name__)
# Option 1: Protect all routes with middleware
BotDetectionMiddleware(app, threshold=0.5)
# Option 2: Protect specific routes only
# BotDetectionMiddleware(app, threshold=0.5, protected_paths=['/api/login', '/api/signup', '/api/checkout'])
@app.route('/api/login', methods=['POST'])
def login():
"""
Login endpoint. Protected by middleware—only humans reach here.
"""
email = request.json.get('email')
password = request.json.get('password')
# Bot score available in g.device_ai_score
print(f'Login attempt from {request.remote_addr}, bot score: {g.device_ai_score}')
# Your login logic here
return jsonify({'success': True, 'token': 'abc123'})
@app.route('/api/sensitive-action', methods=['POST'])
@requires_human(threshold=0.6)
def sensitive_action():
"""
Sensitive endpoint with stricter verification.
"""
# Only humans with bot score > 0.6 reach here
return jsonify({'success': True})
if __name__ == '__main__':
app.run(debug=True)
Step 3: FastAPI Integration (Dependency Injection Pattern)
For FastAPI, create a dependency that verifies requests using the dependency injection pattern:
# bot_detection.py
import os
import httpx
from typing import Optional
from fastapi import Depends, HTTPException, Request
DEVICE_AI_API_KEY = os.getenv('DEVICE_AI_API_KEY')
DEVICE_AI_VERIFY_URL = 'https://device.ai/v1/verify'
class DeviceAIResult:
"""
Bot detection result from device.ai
"""
def __init__(self, score: float, confidence: float, is_bot: bool):
self.score = score
self.confidence = confidence
self.is_bot = is_bot
async def verify_human(
request: Request,
threshold: float = 0.5
) -> DeviceAIResult:
"""
FastAPI dependency: verify that the request comes from a human.
Usage:
@app.post('/api/login')
async def login(device: DeviceAIResult = Depends(verify_human)):
if device.is_bot:
raise HTTPException(status_code=403, detail='Bot detected')
# Process login
Args:
request: FastAPI request object (auto-injected)
threshold: Bot score threshold (0.0=bot, 1.0=human)
Returns:
DeviceAIResult: Bot detection result
Raises:
HTTPException: If request is identified as a bot
"""
try:
client_ip = request.client.host
user_agent = request.headers.get('user-agent', '')
# Call device.ai verification API
async with httpx.AsyncClient() as client:
response = await client.post(
DEVICE_AI_VERIFY_URL,
json={
'ip': client_ip,
'user_agent': user_agent,
'headers': dict(request.headers),
},
headers={'Authorization': f'Bearer {DEVICE_AI_API_KEY}'},
timeout=5.0
)
if response.status_code == 200:
result = response.json()
bot_score = result.get('score', 0.5)
confidence = result.get('confidence', 0.0)
device_result = DeviceAIResult(
score=bot_score,
confidence=confidence,
is_bot=bot_score < threshold
)
if device_result.is_bot:
raise HTTPException(
status_code=403,
detail=f'Bot detected (score: {bot_score:.2f})'
)
return device_result
else:
# API error: fail open
print(f'Device.AI verification failed: {response.status_code}')
return DeviceAIResult(score=0.5, confidence=0.0, is_bot=False)
except httpx.TimeoutException:
# Timeout: fail open
print('Device.AI verification timeout')
return DeviceAIResult(score=0.5, confidence=0.0, is_bot=False)
except Exception as e:
# Any other error: fail open
print(f'Device.AI verification error: {e}')
return DeviceAIResult(score=0.5, confidence=0.0, is_bot=False)
async def verify_human_strict(request: Request) -> DeviceAIResult:
"""
Strict verification dependency (threshold=0.6).
Use for sensitive endpoints like checkout or password change.
"""
return await verify_human(request, threshold=0.6)
Now use the dependency in your FastAPI routes:
# main.py
from fastapi import FastAPI, Depends, HTTPException
from pydantic import BaseModel
from bot_detection import verify_human, verify_human_strict, DeviceAIResult
app = FastAPI()
class LoginRequest(BaseModel):
email: str
password: str
@app.post('/api/login')
async def login(
req: LoginRequest,
device: DeviceAIResult = Depends(verify_human)
):
"""
Login endpoint. Protected by human verification.
device.ai score is available in the device parameter.
"""
print(f'Login from {req.email}, bot score: {device.score}')
return {'success': True, 'token': 'abc123'}
@app.post('/api/checkout')
async def checkout(
device: DeviceAIResult = Depends(verify_human_strict)
):
"""
Checkout endpoint. Uses stricter verification (threshold=0.6).
"""
print(f'Checkout request, bot score: {device.score}')
return {'success': True, 'order_id': 'order_12345'}
@app.post('/api/public')
async def public_endpoint():
"""
Public endpoint. No bot verification required.
"""
return {'message': 'Hello, world!'}
Step 4: Production Best Practices
4.1 Async Verification (FastAPI)
The FastAPI example above uses async/await with httpx.AsyncClient for non-blocking I/O. This is critical for production—blocking verification calls can bottleneck your API.
For Flask with async support, use aiohttp:
import aiohttp
async def verify_async(ip, user_agent):
async with aiohttp.ClientSession() as session:
async with session.post(
DEVICE_AI_VERIFY_URL,
json={'ip': ip, 'user_agent': user_agent},
headers={'Authorization': f'Bearer {DEVICE_AI_API_KEY}'},
timeout=aiohttp.ClientTimeout(total=5)
) as response:
return await response.json()
4.2 Caching Results
Bot detection scores can be cached per IP address for 5-60 minutes to reduce API calls. This is safe because:
- Bot behavior is consistent over time (once a bot, likely always a bot from that IP)
- Caching reduces verification costs
- Latency is improved (cache hits are instant)
from functools import lru_cache
import time
# Simple in-memory cache with TTL
scores_cache = {} # {ip: (score, timestamp)}
CACHE_TTL = 300 # 5 minutes
def get_cached_score(ip):
if ip in scores_cache:
score, timestamp = scores_cache[ip]
if time.time() - timestamp < CACHE_TTL:
return score # Cache hit
return None # Cache miss
def set_cached_score(ip, score):
scores_cache[ip] = (score, time.time())
# Use in verification:
if cached_score := get_cached_score(client_ip):
bot_score = cached_score # Use cached score
else:
bot_score = verify_and_cache(client_ip) # Fresh verification
4.3 Timeout Handling
Device.ai verification should never timeout your critical paths. Use short timeouts and fail open:
try:
response = requests.post(
DEVICE_AI_VERIFY_URL,
json=payload,
headers=headers,
timeout=2 # 2-second timeout
)
except requests.Timeout:
# Fail open: allow request, log for monitoring
print(f'Bot verification timeout for {client_ip}')
return {'score': 0.5, 'confidence': 0.0} # Neutral score
4.4 Rate Limiting on 429 (Quota Exceeded)
When you hit the free tier limit (1,000 daily verifications), device.ai returns HTTP 429. Handle it gracefully:
if response.status_code == 429:
# Quota exceeded. Escalate to stricter rate limiting.
print('Device.ai quota exceeded. Applying local rate limit.')
# Option 1: Allow more requests locally (trusted IPs only)
if is_trusted_ip(client_ip):
return {'score': 0.8, 'is_bot': False}
# Option 2: Block all requests until quota resets
return {'score': 0.0, 'is_bot': True}
# Option 3: Upgrade to paid plan
# See https://device.ai/upgrade?source=blog-python-bot-detection
Step 5: Monitoring and Logging
Track bot detection metrics to understand your traffic:
import logging
from collections import defaultdict
logger = logging.getLogger('bot_detection')
metrics = defaultdict(
lambda: {'bot': 0, 'human': 0, 'neutral': 0}
)
def record_verification(client_ip, bot_score, endpoint):
"""
Record bot detection result for monitoring.
"""
if bot_score < 0.3:
classification = 'bot'
elif bot_score > 0.6:
classification = 'human'
else:
classification = 'neutral'
metrics[endpoint][classification] += 1
logger.info(
f'Bot verification: {endpoint} | IP: {client_ip} | Score: {bot_score} | Class: {classification}'
)
# Send to monitoring service (Datadog, New Relic, etc.)
# monitor.increment('bot_detection.requests', tags=[
# f'endpoint:{endpoint}',
# f'classification:{classification}'
# ])
def get_bot_rate(endpoint):
"""
Calculate bot block rate for an endpoint.
"""
stats = metrics[endpoint]
total = stats['bot'] + stats['human'] + stats['neutral']
if total == 0:
return 0
return stats['bot'] / total * 100
Step 6: Testing Bot Detection
Test that your bot detection actually catches bots. Use Puppeteer or Selenium to simulate automated requests:
# test_bot_detection.py
import requests
from requests_selenium import requests
import time
# Test 1: Normal HTTP request (looks like a bot)
response = requests.post('http://localhost:5000/api/login', json={
'email': 'test@example.com',
'password': 'password123'
})
print(f'Normal request: {response.status_code}')
# Test 2: Request with basic auth (looks more human)
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
}
response = requests.post('http://localhost:5000/api/login', json={
'email': 'test@example.com',
'password': 'password123'
}, headers=headers)
print(f'Request with user-agent: {response.status_code}')
# Test 3: Puppeteer automated request (should be detected as bot)
# Use: node -e "const p = require('puppeteer'); ..."
print('Puppeteer automated requests should return 403')
Step 7: Handling False Positives
If legitimate users are being blocked, adjust your threshold or add allowlists:
# Allowlist specific IPs (corporate networks, mobile carriers)
ALLOWLIST = [
'192.168.1.0/24', # Corporate network
'10.0.0.0/8', # Internal network
]
def should_skip_verification(client_ip):
for network in ALLOWLIST:
if ipaddress.ip_address(client_ip) in ipaddress.ip_network(network):
return True
return False
# Use in middleware:
if should_skip_verification(client_ip):
return # Skip verification for allowlisted IPs
Pricing and Free Tier
| Plan | Daily Verifications | Cost | Best For |
|---|---|---|---|
| Free | 1,000 | $0 | Development, small apps |
| Pay-as-you-go | Unlimited | $0.001 per verification | Production, scaling |
| Pro | Custom | $19/month + usage | High volume, enterprise |
The free tier (1,000 daily verifications) is perfect for testing. When you're ready to scale, upgrade at device.ai/upgrade.
Common Patterns and Examples
Pattern 1: Protect Login and Signup Only
# Flask: protect specific paths
BotDetectionMiddleware(
app,
threshold=0.5,
protected_paths=['/api/login', '/api/signup']
)
Pattern 2: Stricter Verification for Sensitive Actions
# FastAPI: different thresholds for different endpoints
@app.post('/api/login')
async def login(device: DeviceAIResult = Depends(verify_human)): # threshold=0.5
...
@app.post('/api/checkout')
async def checkout(device: DeviceAIResult = Depends(verify_human_strict)): # threshold=0.6
...
Pattern 3: Custom Verification with Fallback
async def verify_with_challenge(
request: Request,
threshold: float = 0.5
) -> DeviceAIResult:
device = await verify_human(request, threshold)
if 0.3 <= device.score <= 0.6:
# Uncertain: require additional verification
# Option 1: Send email confirmation
# Option 2: Require SMS code
# Option 3: Show CAPTCHA
raise HTTPException(
status_code=403,
detail='Please complete verification'
)
return device
Frequently Asked Questions
Can I Use Device.AI with Django?
Yes. Create a middleware decorator or use Django's middleware system. The verification logic is the same—just call device.ai and handle the response.
What About WebSocket Connections?
Verify on the initial HTTP handshake before upgrading to WebSocket. Once a connection is established, you've already verified the user.
Can I Cache Verification Results?
Yes—cache per IP address for 5-60 minutes. Bot behavior is consistent, so caching is safe and reduces API calls.
What If Device.AI is Down?
Fail open: return a neutral score and allow the request. Log the incident for monitoring. This ensures your app stays online even if device.ai is temporarily unavailable.
How Much Does Device.AI Cost at Scale?
$0.001 per verification. For 1 million daily checks, that's $30/month. Upgrade anytime at device.ai/upgrade.
Next Steps
- Get your free API key at device.ai/v1/keys (1 minute, no signup)
- Copy the Flask or FastAPI middleware code above
- Integrate into your app (5 minutes)
- Test with Puppeteer or curl
- Monitor bot detection rates
- Upgrade to paid when you hit the free tier limit
Resources
- Device.AI API Documentation — Full API reference and integration guides
- Bot Detection for React — Framework Guide #2 for React apps
- Bot Detection for Next.js — Framework Guide #1 for Next.js apps
- What Is Bot Detection? — Understand bot detection concepts and methodology
Questions? Contact support@device.ai or check our docs.