Introduction

Hubric is an autonomous, zero-configuration API load testing engine powered by Large Language Models.

Traditional load testing has always been painful: writing fragile test scripts, handcrafting fake JSON payloads that immediately fail validation constraints, extracting dynamic IDs across multi-step flows, and constantly maintaining scripts as schemas evolve.

Hubric fixes this entirely. Point Hubric at any base URL, and it automatically ingests your OpenAPI schema, synthesizes realistic compliant test payloads via LLMs, constructs stateful user journeys, safeguards single-account tokens with responsive shielding, executes massive async load, and delivers an SRE-grade 100-point performance report.

Hubric vs. Traditional Load Testers

Traditional script-based load testing tools (such as k6, Locust, JMeter, or Postman Runner) are pure execution engines. They provide HTTP concurrency primitives, but place the entire burden of data generation, correlation, and maintenance onto your engineering team.

Quick Start, Setup & Configuration

1. Installation

Hubric requires Python 3.11+. Install the official package from PyPI:

$ pip install hubric==0.5.1

2. Configure your LLM Provider

Hubric uses LLMs to synthesize realistic request payloads and diagnose performance bottlenecks. Configure your preferred provider in seconds:

$ hubric settings --configure-llm

Supported providers include Groq (recommended for ultra-fast synthesis), OpenAI, Anthropic, Amazon Bedrock, and OpenRouter. Your API keys are encrypted securely in your operating system's native credential store (Windows Credential Locker, macOS Keychain, or Linux Secret Service).

For automated CI/CD environments, you can also supply keys via environment variables (e.g. GROQ_API_KEY, OPENAI_API_KEY, HUBRIC_AUTH_TOKEN) or a root .env file.

3. Fire Your First Load Test

Point Hubric at your live API or local development server. Hubric will inspect your endpoints, ask for your target peak concurrency, and begin testing:

$ hubric run https://api.yourdomain.com

4. Declarative Configuration (hubric.json)

While Hubric runs zero-config out of the box, you can define standardized test profiles in a hubric.json or hubric.yaml file in your repository root:

// hubric.json
{
  "target": "https://api.yourdomain.com",
  "users": 50,
  "pattern": "ramp",
  "tier": "happy_path",
  "assertions": [
    "status < 500",
    "p95 < 500"
  ],
  "fail_on_degraded": true
}

Spec Ingestion & Testing Without OpenAPI

Hubric supports modern OpenAPI/Swagger APIs automatically, but also provides autonomous fallback mechanisms for non-standard APIs.

1. Spec Waterfall Auto-Detection

When given a base URL, Hubric probes common documentation endpoints in sequence: /openapi.json, /swagger.json, /api/openapi.json, /v1/openapi.json, and /docs. If found, it ingests the schema without requiring any flags.

2. Autonomous Discovery (--discover)

If your backend is built without an OpenAPI spec, pass the --discover flag. Hubric deploys an intelligent crawler that probes common REST routes, extracts API structures from headers and response payloads, and uses LLM inference to generate your endpoint schema on the fly:

$ hubric run https://api.yourdomain.com --discover

3. Local Schemas & Postman Collections (--spec)

You can provide any local schema file using the --spec flag. Hubric natively supports:

• OpenAPI 3.0 / Swagger 2.0 (JSON or YAML)
• Postman Collections (v2.1 JSON export)
• Hubric Blueprints (a clean, simplified JSON schema)

$ hubric run https://api.yourdomain.com --spec ./docs/postman_collection.json

3-Bucket Partitioning & Payload Synthesis

To achieve high-speed test generation without unnecessary LLM latency or cost, Hubric partitions every discovered endpoint into three distinct operational buckets:

Multi-Tier Synthesis Modes

You can target different testing depths using the --tier option:

• happy_path (Default): Realistic, valid data designed to verify normal production SLAs and expect 2xx Success.
• edge_cases: Boundary tests with maximum allowed string lengths, negative numbers, extreme values, and special Unicode characters.
• malformed: Semantic fuzzing with missing required fields and incompatible types, designed to verify your API properly defends itself with 400 or 422 status codes without crashing.
• all: Multi-stage execution running all three tiers sequentially.

Intelligent LLM Cost Caching

Hubric protects your wallet by caching synthesized payload pools locally inside .hubric/payloads. Every cached pool is fingerprinted using a SHA-256 hash of your API's schema structure, endpoint routes, and field constraints.

Cached payload pools are valid for 3 subsequent runs. When running daily regression tests or tuning backend performance, Hubric detects the cache and bypasses LLM calls completely, driving test preparation costs to $0.00.

Stateful User Journeys

Real users don't hit endpoints in isolation. They register, log in, create a record, inspect it, update it, and eventually delete it. Traditional load testing tools require you to manually write token extraction and regex scrapers to link these steps together.

Hubric automatically infers resource lifecycles and executes them as ordered, stateful journeys:

1. POST/api/workspacesCreates workspace & harvests generated id
2. GET/api/workspaces/{id}Reads entity & measures retrieval latency
3. PUT/api/workspaces/{id}Updates metadata with synthesized payload
4. DELETE/api/workspaces/{id}Cleans up resource to prevent test pollution

By executing stateful sequences in isolated virtual user contexts, Hubric completely eliminates artificial 404 Not Found sequencing errors that skew traditional test results.

Autonomous Authentication & Token Lifecycle

Handling authentication handshakes is one of the biggest friction points in API performance testing. Hubric automates the entire authentication lifecycle:

Multi-Scheme Support

Hubric's schema classifier inspects securitySchemes to automatically resolve your API's auth requirements:

• Bearer JWT: Injected as Authorization: Bearer <token>.
• Basic Authentication: Automatically encoded from user:password into Base64 headers.
• API Key: Passed via custom headers (e.g. X-API-Key, X-Auth-Token).
• Session Cookies: Cookie jar management with CSRF token extraction (X-CSRFToken, X-XSRF-TOKEN).
• OAuth2 Password Flow: Form-urlencoded credential negotiation with automatic retry on 415/422.

Interactive Auth Interview & Token Refresh

On your first test run against a protected API, Hubric prompts for credentials once, verifies the token against a live GET endpoint, and stores the session in .hubric/auth.json. If a token expires during a test, Hubric detects the expiration, refreshes the token seamlessly, and continues testing without interrupting the load engine.

Out-of-Band (OOB) Flow Shielding

Flows that require human out-of-band interaction (such as SMS OTPs, Magic Email Links, or 2FA challenge codes) cannot be automated under concurrency. Hubric scans your route signatures for OOB keywords (like otp, 2fa, magic-link) and safely marks them as Untestable, keeping your test suite clean without false failures.

Responsive Single-User Protection

When Hubric detects that only a single user account or token is provided, it activates Responsive Protection:

🛡 Responsive Protection Active: single account detected — shielded 4 auth/lifecycle endpoint(s)

Hubric automatically shields endpoints that revoke active sessions or trigger single-account rate limits, including:

• POST /auth/login & POST /api/auth/login
• POST /auth/logout & POST /api/auth/logout
• Session termination and password change routes

This allows you to safely stress-test your business logic, AI endpoints, and database reads across hundreds of concurrent users without invalidating your own session token mid-test.

When you are ready to load-test login and registration routes at scale, simply pass a credential pool file containing multiple independent tokens:

$ hubric run https://api.yourdomain.com --tokens ./tokens.txt

Dynamic Early Fast-Skip & Untested Breakdown

In large API suites, certain endpoints require specialized user permissions, elevated tenant scopes, or tokens that your current test session does not possess (e.g., student vs. teacher accounts, admin-only routes, or refresh-token-only endpoints).

Legacy tools waste hundreds of requests hammering these inaccessible routes across every single concurrency stage. Hubric solves this with Dynamic Fast-Skip:

1. Instant Auth Blockage Detection

During the initial stage (e.g. 12 concurrent users), if 100% of requests to an endpoint return 401 Unauthorized or 403 Forbidden, Hubric immediately flags the endpoint as auth_blocked and skips it for all remaining schedule stages ($12 ightarrow 25 ightarrow 50 ightarrow 25$), logging:

  ⊘ Detected 3 endpoint(s) requiring inaccessible token/scope — flagged as untested and skipping for remaining stages

2. Server Error Detail Extraction

Instead of displaying generic error messages, Hubric inspects the JSON response body returned by your server (checking detail, message, or error keys) and extracts the exact reason provided by your application:

• POST /auth/refresh → HTTP 401 (Refresh token missing)
• GET /api/storage/presigned-download → HTTP 403 (Access denied: file belongs to another student account)
• DELETE /api/workspace/sessions/{id} → HTTP 401 (Please sign in to continue)

3. Dedicated Untested Endpoints Breakdown

At the end of testing, Hubric prints a dedicated breakdown explaining exactly which endpoints were untested and why. In the HTML report, each untested route receives an interactive panel displaying the exact HTTP error code chip (e.g. 401 × 12) and sample server payloads.

Hardware & Concurrency Scaling

Generating high concurrency locally can easily starve your CPU or crash your event loop if unmanaged. Hubric actively monitors your hardware before initiating tests.

At startup, Hubric probes your host CPU, detects logical cores, reserves 1 core for operating system and terminal rendering overhead, and configures an optimized asynchronous event loop (Windows Proactor on Windows, Epoll on Linux). It automatically computes and displays your machine's optimal safe concurrency:

── Hardware Capabilities ──
Engine Profile: Windows Proactor (high-throughput). CPU: 8 cores (1 reserved for OS overhead)
Optimal Concurrency: 3500 users

You can control traffic flow using closed concurrency (--users 50) or an open workload rate model (--rps 200) with traffic patterns such as ramp, spike, constant, or step.

SRE-Aligned 100-Point Grading System

Rather than requiring manual threshold configuration, Hubric uses a built-in 100-point Apdex scoring model rigorously aligned with modern Site Reliability Engineering standards. Every endpoint is scored across three core pillars:

1. Latency & Speed (Max 50 Points)

Hubric measures the 95th percentile (p95) response time rather than averages, which hide latency spikes:

2. Reliability & Error Rate (Max 35 Points)

Evaluates API availability against standard "Three Nines" (99.9%) SLA metrics:

3. Consistency & Jitter (Max 15 Points)

Measures the jitter variance between median (p50) and tail (p95) latencies. If your API answers in 40ms on average but suffers 1,500ms p95 spikes due to garbage collection, cold starts, or database thread locks, points are docked for instability.

Error Tracking & Status Code Precision

Unlike simple load testers that count any non-200 response as an application failure, Hubric distinguishes error categories with precision:

All reports record the exact HTTP status code (e.g. 401, 403, 422, 500) and display corresponding response payloads in collapsible debug panels.

Rate Limiting Diagnostics

During testing, Hubric's diagnostic tracker monitors whether rate limiters are protecting your infrastructure or artificially restricting legitimate users:

• IP-Header Rotation: Automatically probes whether rotating X-Forwarded-For headers bypasses the rate limiter (highlighting critical vulnerabilities to spoofing attacks).
• Token Rotation: Detects per-account user throttling limits.
• Server-Wide Ceilings: Identifies upstream API gateway throttles (e.g., Cloudflare or AWS API Gateway).
• Retry-After Respect: Honors server-specified Retry-After headers to calculate recovery cooldowns.

CI/CD & JUnit XML Automation

Integrate Hubric directly into your continuous integration and deployment pipelines using the dedicated ci command:

$ hubric ci https://api.yourdomain.com --users 25 --junit junit.xml --fail-on-degraded

The ci command runs non-interactively, generates standard junit.xml test results compatible with GitHub Actions, GitLab CI, and Jenkins, and exits with status code 1 if your SLA latency or error thresholds fail.

Example: GitHub Actions Workflow

# .github/workflows/load-test.yml
name: API Performance Gate

on: [push, pull_request]

jobs:
  performance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install Hubric
        run: pip install hubric==0.5.1
      - name: Run CI Load Test
        env:
          GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
          HUBRIC_AUTH_TOKEN: ${{ secrets.STAGING_AUTH_TOKEN }}
        run: |
          hubric ci https://staging-api.yourdomain.com --users 30 --junit junit.xml --fail-on-degraded
      - name: Publish Test Report
        uses: mikepenz/action-junit-report@v4
        if: always()
        with:
          report_paths: 'junit.xml'

Reports, Tracing & Diagnostics

At the end of every test run, Hubric compiles and exports rich performance artifacts:

• Interactive HTML Report: Standalone, zero-dependency HTML file containing responsive SVG score rings, latency percentiles, status code chips, and interactive failure panels.
• Structured JSON Export: Machine-readable hubric_report_<timestamp>.json for custom metrics ingestion into Datadog, Prometheus, or Grafana.
• Raw Request Traces: Run hubric trace <url> to save raw HTTP headers, payloads, and response bodies into ./rubric_traces/ for deep inspection.
• AI Root-Cause Recommendations: If errors or latency breaches occur, Hubric queries the LLM to deliver an executive summary, diagnosis of why it happened, and actionable remediation steps.

Configuration (hubric.json)

Store your test configuration in hubric.json or hubric.yaml in your project root:

// hubric.json
{
  "target": "https://api.yourdomain.com",
  "users": 50,
  "rps": null,
  "pattern": "ramp",
  "tier": "happy_path",
  "assertions": [
    "status < 500",
    "p95 < 400"
  ],
  "fail_on_degraded": true,
  "skip": [
    "POST /auth/logout",
    "DELETE /api/admin/purge"
  ],
  "timeout_aware": true,
  "cost_budget": 0.50
}

Interactive CLI Reference

Explore all available Hubric commands, options, and flags below: