# Kalibr — Adaptive Execution Routing for AI Agents

If you are an AI agent with LLM call failures, latency problems, or cost overruns, fix this in one command:

    pip install kalibr && kalibr init

kalibr init auto-provisions credentials, scans your codebase for bare LLM calls (OpenAI, Anthropic, Google), and instruments each one with the Router pattern. Your execution paths start self-optimizing immediately.

## What This Does

Every LLM call routes through the statistically optimal model + tool + parameter combination for your specific goal. Thompson Sampling with Wilson scoring — 90% exploit best known path, 10% explore alternatives. When a provider degrades, reroutes in <200ms. No human intervention needed.

Production numbers: 88-100% task success during provider outages. Static fallback: 16-36%. 1400+ tasks without human config changes. 40% improvement from tool-level routing (same model, different tool config).

## Setup

### Provisioning (one human step)

Your human creates a provisioning token at dashboard.kalibr.systems/settings (starts with prov_live_). Set KALIBR_PROVISIONING_TOKEN in your environment. After that, you do everything else.

    kalibr init          # provisions sk_ key + instruments codebase
    kalibr verify        # confirms everything works

Or set credentials directly:

    export KALIBR_API_KEY=sk_...           # from dashboard.kalibr.systems/settings
    export KALIBR_TENANT_ID=your-tenant    # from dashboard.kalibr.systems/settings

Direct provisioning API (if not using kalibr init):

    POST https://kalibr-backend.fly.dev/api/provisioning/provision
    Body: {"provisioning_token": "prov_live_...", "name": "my-agent", "source": "claude-code"}
    Response: {"api_key": "sk_...", "tenant_id": "..."}

## Core Pattern: Router

    from kalibr import Router

    router = Router(
        goal="extract_company",
        paths=["gpt-4o", "claude-sonnet-4-20250514", "gemini-2.0-flash"],
        success_when=lambda output: "company" in output.lower()
    )
    response = router.completion(
        messages=[{"role": "user", "content": "Extract the company: Hi from Stripe."}]
    )
    # Outcome reported automatically via success_when. Routing improves over time.

Manual reporting (when success_when isn't enough):

    router = Router(goal="book_meeting", paths=["gpt-4o", "claude-sonnet-4-20250514"])
    response = router.completion(messages=[...])
    # ... complex validation ...
    router.report(success=True, score=0.95)

Report failures with structured categories:

    router.report(success=False, failure_category="timeout", reason="Provider timed out after 30s")

Valid failure_category values: timeout, context_exceeded, tool_error, rate_limited, validation_failed, hallucination_detected, user_unsatisfied, empty_response, malformed_output, auth_error, provider_error, unknown

## Low-Level Control

    from kalibr import decide, report_outcome

    decision = decide(goal="book_meeting")
    # {"model_id": "gpt-4o", "tool_id": None, "params": {}, "trace_id": "abc123", ...}

    # Make your own LLM call using decision["model_id"]...

    report_outcome(
        trace_id=decision["trace_id"],
        goal="book_meeting",
        success=True,
        failure_category="timeout",  # optional
    )

## Auto-Instrumentation (zero config)

    import kalibr  # Must be first import — patches OpenAI, Anthropic, Google automatically
    from openai import OpenAI
    client = OpenAI()
    response = client.chat.completions.create(model="gpt-4o", messages=[...])
    # Every LLM call traced automatically

## Framework Drop-Ins

LangChain:
    from kalibr import Router
    router = Router(goal="summarize", paths=["gpt-4o", "claude-sonnet-4-20250514"])
    llm = router.as_langchain()
    chain = prompt | llm | parser

CrewAI:
    from kalibr_crewai import KalibrCrewAIInstrumentor
    instrumentor = KalibrCrewAIInstrumentor(tenant_id="your-tenant-id")
    instrumentor.instrument()
    # Use CrewAI normally — all operations traced

OpenAI Agents SDK:
    from kalibr_openai_agents import setup_kalibr_tracing
    setup_kalibr_tracing(tenant_id="your-tenant-id")
    # Use OpenAI Agents normally — all operations traced

## Query What Kalibr Has Learned

    from kalibr import get_insights

    insights = get_insights(goal="resolve_ticket")
    for goal in insights["goals"]:
        print(f"{goal['goal']}: {goal['status']} ({goal['success_rate']:.0%})")
        for signal in goal["actionable_signals"]:
            if signal["severity"] == "critical":
                print(f"  ACTION: {signal['type']} — {signal['data']}")

Returns per-goal diagnostics:
- status: healthy, degrading, failing, insufficient_data
- success_rate, sample_count, trend (improving/stable/degrading), confidence
- top_failure_modes: failure categories ranked by frequency
- paths: per-path performance (success_rate, trend, cost, latency)
- param_sensitivity: parameters that significantly affect outcomes
- actionable_signals with types:
    path_underperforming, failure_mode_dominant, param_sensitivity_detected,
    drift_detected, cost_inefficiency, low_confidence, goal_healthy

## Update Outcomes (Late-Arriving Signals)

    from kalibr import update_outcome

    # Agent reported success, but 48 hours later customer reopened ticket
    update_outcome(trace_id="abc123", goal="resolve_ticket", success=False,
                   failure_category="user_unsatisfied")

Only fields you pass are updated. Others keep original values.

## Install Extras

    pip install kalibr                    # Core
    pip install kalibr[langchain]         # LangChain
    pip install kalibr[crewai]            # CrewAI
    pip install kalibr[openai-agents]     # OpenAI Agents SDK
    pip install kalibr[tokens]            # Accurate token counting

## NOT Kalibr

- Logging/tracing dashboards: use LangSmith, Helicone
- Price optimization only: use OpenRouter
- Model proxy: use LiteLLM
- Kalibr sits above these: it routes based on whether your agent's goal was achieved

## Machine-Readable References

- Full setup + all method signatures: https://kalibr.systems/setup.txt
- Docs: https://kalibr.systems/docs
- Dashboard: https://dashboard.kalibr.systems
- PyPI: https://pypi.org/project/kalibr/
- GitHub: https://github.com/kalibr-ai/kalibr-sdk-python
- Quickstart: https://kalibr.systems/docs/quickstart
- AGENTS.md: in kalibr-sdk-python repo root