effgen models · 1.0.0

Any model, anywhere you already run it

10 provider adapters, 9 of them carrying a bundled catalog of 417 models with their context windows, prices and capabilities. The tenth carries none on purpose: point it at a server you already run — vLLM, SGLang, TGI, llama.cpp, Ollama, LM Studio, a proxy, a gateway — and it drives the model that server serves. And 4 local engines run weights in the process, including on Apple Silicon.

Point it at your own server Reference documentation

Provider adapters

417

Catalogued models, 9 providers

Local engines

base_url to reach any other server

effgen models browse

$ effgen models browse --limit 10 Model Catalog                                Models across providers — showing 10 of 417                                ┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━┓┃ Provider  ┃ Model ID                   ┃ Context ┃ Max Out ┃ $/1M in ┃ $/1M out ┃ Tools ┃ Vision ┃ Free ┃┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━┩│ anthropic │ claude-3-5-sonnet-20240620 │ 200,000 │   8,096 │      $3 │      $15 │   ✓   │   ✓    │      ││ anthropic │ claude-3-5-sonnet-20241022 │ 200,000 │   8,096 │      $3 │      $15 │   ✓   │   ✓    │      ││ anthropic │ claude-3-7-sonnet-20250219 │ 200,000 │  64,000 │      $3 │      $15 │   ✓   │   ✓    │      ││ anthropic │ claude-3-haiku-20240307    │ 200,000 │   4,096 │   $0.25 │    $1.25 │   ✓   │   ✓    │      ││ anthropic │ claude-3-opus-20240229     │ 200,000 │   4,096 │     $15 │      $75 │   ✓   │   ✓    │      ││ anthropic │ claude-3-sonnet-20240229   │ 200,000 │   4,096 │      $3 │      $15 │   ✓   │   ✓    │      ││ anthropic │ claude-haiku-4-5           │ 200,000 │  64,000 │      $1 │       $5 │   ✓   │   ✓    │      ││ anthropic │ claude-haiku-4-5-20251001  │ 200,000 │  64,000 │      $1 │       $5 │   ✓   │   ✓    │      ││ anthropic │ claude-opus-4-1            │ 200,000 │  32,000 │     $15 │      $75 │   ✓   │   ✓    │      ││ anthropic │ claude-opus-4-1-20250805   │ 200,000 │  32,000 │     $15 │      $75 │   ✓   │   ✓    │      │└───────────┴────────────────────────────┴─────────┴─────────┴─────────┴──────────┴───────┴────────┴──────┘ More: --offset 10 for the next page. Pricing source: catalog snapshot. Update: effgen models refresh  ·  Detail: effgen models info <id>

One table across every provider, so “the cheapest vision model over 128k context” is one command rather than nine browser tabs. Every figure in it comes from the bundled catalog, and the footer names the snapshot it came from and the command that refreshes it.

Any OpenAI-protocol server

One argument, and effGen drives the server you already have

vLLM, SGLang, TGI, llama.cpp’s server, Ollama, LM Studio, LiteLLM and most corporate gateways all expose the OpenAI chat-completions API. Give effGen a base_url and it talks to the model you are already serving, instead of loading a second copy of the weights inside the agent’s process.

python

from effgen.models import load_model

model = load_model(
    "gemini:gemini-3.1-flash-lite",
    provider="openai_compatible",
    base_url="http://127.0.0.1:8000/v1",
)
print(model.generate("What is 6 times 7?").text)

what that printed

python

from effgen import Agent, AgentConfig

with Agent(AgentConfig(
    model="gemini:gemini-3.1-flash-lite",
    base_url="http://127.0.0.1:8000/v1",
)) as agent:
    print(agent.run("What is 6 times 7?").output)

what that printed

Both were run against an effgen serve on this machine — which speaks the same protocol as everything in the list, and is the easiest one to reproduce. A base_url with no provider named is the whole instruction: it selects this adapter rather than falling through to a local download of the model id.

vLLM

continuous batching on your own GPUs

SGLang

structured generation and prefix caching

TGI

Hugging Face's inference server

llama.cpp

its bundled server, on CPU or a laptop GPU

Ollama

the local runner, at /v1

LM Studio

a desktop server on a workstation

LiteLLM

a proxy in front of several providers

A gateway

your company's, with its own key and quota

Why serve the model separately

Loading in-process means one copy of the weights per agent process, no sharing between agents, no continuous batching across callers, and a GPU tied to the agent’s lifetime. A shared server fixes all four: the weights load once, every caller’s requests batch together, and the GPU outlives any individual run.

It is also the only way to have several frameworks — or several versions of your own service — generate under identical settings, which is what a fair comparison needs.

Ask the endpoint what it serves

python

from effgen.models import load_model

model = load_model(
    "gemini:gemini-3.1-flash-lite",
    provider="openai_compatible",
    base_url="http://127.0.0.1:8000/v1",
)
print(model.list_served_models())

what that server reported

['gpt-4', 'gpt-4-turbo', 'gpt-4o', 'gpt-4o-mini', 'gpt-3.5-turbo', 'gpt-3.5-turbo-instruct', 'default', 'effgen-default', 'gemini:gemini-3.1-flash-lite']

An endpoint that does not implement /models returns an empty list rather than failing — some minimal servers have nothing to say about themselves.

Where the endpoint comes from

Order	Type	Description
`1`	—	base_url= passed to load_model(), AgentConfig or the adapter
`2`	—	EFFGEN_BASE_URL
`3`	—	OPENAI_BASE_URL
`4`	—	OPENAI_API_BASE

effGen's own variable is consulted first, so pointing effGen at a server does not redirect every other OpenAI client on the machine.

provider="openai" with a base_url routes here too, because a URL of your own means the model ids, the context window and the pricing are the server’s rather than OpenAI’s. Without one it stays on OpenAI, so a machine-wide OPENAI_BASE_URL set for something unrelated cannot silently reroute a plain OpenAI call.

openai_compatible
openai-compatible
openai_compat
compatible
server
vllm_server
local_server

All accepted spellings of the same provider.

What effGen refuses to assume about your server

The context window is a guess, and it says so

The protocol carries no way for a server to publish its window, so effGen assumes 32,768 and warns once, naming the value, the model and the endpoint. effGen plans compaction against that number, so a server started with a smaller window fails later, at the call, far from where the number was chosen.

python

assumed = load_model(
    "gemini:gemini-3.1-flash-lite",
    provider="openai_compatible",
    base_url="http://127.0.0.1:8000/v1",
)
print("assumed:", assumed.get_context_length())

told = load_model(
    "gemini:gemini-3.1-flash-lite",
    provider="openai_compatible",
    base_url="http://127.0.0.1:8000/v1",
    context_length=1_000_000,
)
print("told:   ", told.get_context_length())

the warning on stderr, then what the program printed

WARNING effgen.models.openai_compatible_adapter: No context_length given for
'gemini:gemini-3.1-flash-lite' at http://127.0.0.1:8000/v1; assuming 32768
tokens. If the server was started with a smaller window (vLLM's
--max-model-len, TGI's --max-total-tokens), pass context_length=<the real
number> — planning against a window the server does not have fails later,
at the call.
assumed: 32768
told:    1000000

Cost: nothing rather than $0

A call through this adapter reports no price. What your own server costs is not something effGen can derive from a token count, so it states nothing — and counts the call as unpriced rather than adding zero to a total.

python

with Agent(AgentConfig(
    model="gemini:gemini-3.1-flash-lite",
    base_url="http://127.0.0.1:8000/v1",
)) as agent:
    r = agent.run("Reply with the single word ok.")
    print("tokens:", r.metadata["total_tokens"])
    print("cost:  ", r.metadata.get("cost_usd"))
    print("unpriced calls:", r.metadata.get("unpriced_calls"))

what that printed

tokens: 33
cost:   None
unpriced calls: 1

Ids, sampling and reasoning

The server serves its own model ids, so no OpenAI catalog is consulted for any of them. The full sampling surface — top_p, top_k, seed, the penalties — is offered, which every implementation of the protocol accepts. Pass supports_reasoning=True if what you serve emits a reasoning stream.

A local server that authenticates nothing needs no credential — effGen sends a placeholder, which vLLM, SGLang, TGI, llama.cpp and Ollama all accept. Pass a real one for a gateway that checks it.

The providers

10 adapters, 9 with a catalog

Both numbers are true and they mean different things, so this page says which one it means. Every row below — the model count, the capability counts, the largest context window, the default model, the date the catalog was last checked against the provider’s live API and the environment variable the adapter reads — is read out of the installed package.

OpenAI

openai

Every model in the catalog calls tools, and most of them take images.

Models: 30
Tool-calling: 30
Vision: 23
Free tier: 0
Max context: 1,047,576
Checked: 2026-06-17

OPENAI_API_KEY

Default when a call names no model: gpt-5.4-nano

Anthropic

anthropic

Every model in the catalog calls tools and takes images — no exceptions either way.

Models: 17
Tool-calling: 17
Vision: 17
Free tier: 0
Max context: 1,000,000
Checked: 2026-06-08

ANTHROPIC_API_KEY

Gemini

gemini

The only bundled catalog carrying audio input, and the one with the most free-tier models.

Models: 8
Tool-calling: 6
Vision: 6
Free tier: 6
Max context: 2,000,000
Checked: 2026-08-13

GOOGLE_API_KEY

Default when a call names no model: gemini-3.1-flash-lite

Groq

groq

Small open models served fast, on a short catalog; about half of them call tools.

Models: 15
Tool-calling: 8
Vision: 1
Free tier: 0
Max context: 131,072
Checked: 2026-08-07

GROQ_API_KEY

Default when a call names no model: llama-3.1-8b-instant

Cerebras

cerebras

Two models, both free-tier and rate-limited, at a 64K context window.

Models: 2
Tool-calling: 1
Vision: 0
Free tier: 2
Max context: 65,536
Checked: 2026-06-08

CEREBRAS_API_KEY

Default when a call names no model: gpt-oss-120b

Together

together

The largest bundled catalog, and the widest range of open weights.

Models: 168
Tool-calling: 66
Vision: 2
Free tier: 0
Max context: 10,485,760
Checked: 2026-08-07

TOGETHER_API_KEY

Default when a call names no model: Qwen/Qwen3.5-9B

Fireworks

fireworks

Open models with a serverless tier, several of them vision-capable.

Models: 16
Tool-calling: 14
Vision: 5
Free tier: 0
Max context: 1,048,576
Checked: 2026-08-07

FIREWORKS_API_KEY

Default when a call names no model: accounts/fireworks/models/gpt-oss-120b

Hugging Face

hf

Hosted inference over the Hub, with the second-largest catalog here.

Models: 124
Tool-calling: 83
Vision: 31
Free tier: 0
Max context: 1,048,576
Checked: 2026-06-08

HF_TOKEN · HUGGINGFACE_API_KEY

Default when a call names no model: Qwen/Qwen2.5-7B-Instruct

Replicate

replicate

Hosted open models, many of them billed by hardware time rather than by token.

Models: 37
Tool-calling: 24
Vision: 0
Free tier: 0
Max context: 1,000,000
Checked: 2026-08-07

REPLICATE_API_TOKEN

Default when a call names no model: meta/meta-llama-3-8b-instruct

Any OpenAI-protocol server

openai_compatible

No bundled catalog at all: it serves whatever the endpoint you point it at serves.

Models: the server's
Tool-calling: —
Vision: —
Free tier: —
Max context: you say
Checked: —

EFFGEN_BASE_URL · OPENAI_BASE_URL · OPENAI_API_BASE

effgen models list

$ effgen models list Available Models                        Provider Registry (bundled catalog)                        ┏━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━━━┓┃ Provider  ┃ Models ┃ Default                                ┃ Auth ┃ Verified   ┃┡━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━━━┩│ openai    │     30 │ gpt-5.4-nano                           │ key  │ 2026-06-17 ││ anthropic │     17 │ —                                      │  —   │ 2026-06-08 ││ gemini    │      8 │ gemini-3.1-flash-lite                  │  —   │ 2026-08-13 ││ cerebras  │      2 │ gpt-oss-120b                           │ key  │ 2026-06-08 ││ groq      │     15 │ llama-3.1-8b-instant                   │  —   │ 2026-08-07 ││ together  │    168 │ Qwen/Qwen3.5-9B                        │  —   │ 2026-08-07 ││ fireworks │     16 │ accounts/fireworks/models/gpt-oss-120b │  —   │ 2026-08-07 ││ replicate │     37 │ meta/meta-llama-3-8b-instruct          │  —   │ 2026-08-07 ││ hf        │    124 │ Qwen/Qwen2.5-7B-Instruct               │  —   │ 2026-06-08 │└───────────┴────────┴────────────────────────────────────────┴──────┴────────────┘ Detail: effgen models list --provider <name>  ·  Filter: --free / --tools  ·  Update: effgen models refresh                     Local HuggingFace cache (47 ready)                     ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┓┃ Model                                            ┃     Size ┃   Status   ┃┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━┩│ BAAI/bge-m3                                      │   4.3 GB │   ready    ││ CohereLabs/aya-expanse-32b                       │  60.2 GB │   ready    ││ CohereLabs/aya-expanse-8b                        │  15.0 GB │   ready    ││ FinLang/finance-embeddings-investopedia          │   0.4 GB │   ready    ││ FremyCompany/BioLORD-2023                        │   0.0 GB │ incomplete ││ Qwen/Qwen2.5-0.5B-Instruct                       │   0.9 GB │   ready    ││ Qwen/Qwen2.5-1.5B-Instruct                       │   2.9 GB │   ready    ││ Qwen/Qwen2.5-14B-Instruct                        │  27.5 GB │   ready    ││ Qwen/Qwen2.5-32B-Instruct                        │  61.0 GB │   ready    ││ Qwen/Qwen2.5-3B-Instruct                         │   5.8 GB │   ready    ││ Qwen/Qwen2.5-7B-Instruct                         │  14.2 GB │   ready    ││ Qwen/Qwen3-0.6B                                  │   1.4 GB │   ready    ││ Qwen/Qwen3-1.7B                                  │   3.8 GB │   ready    ││ Qwen/Qwen3-14B                                   │  27.5 GB │   ready    ││ Qwen/Qwen3-32B                                   │  61.0 GB │   ready    ││ Qwen/Qwen3-4B                                    │   7.5 GB │   ready    ││ Qwen/Qwen3-4B-Instruct-2507                      │   7.5 GB │   ready    ││ Qwen/Qwen3-8B                                    │  15.3 GB │   ready    ││ Qwen/Qwen3-Embedding-0.6B                        │   1.1 GB │   ready    ││ Qwen/Qwen3-Embedding-8B                          │  14.1 GB │   ready    ││ Qwen/Qwen3.5-0.8B                                │   1.6 GB │   ready    ││ Qwen/Qwen3.5-27B                                 │  51.8 GB │   ready    ││ Qwen/Qwen3.5-2B                                  │   4.3 GB │   ready    ││ Qwen/Qwen3.5-9B                                  │  18.0 GB │   ready    ││ Qwen/Qwen3.6-27B                                 │  51.8 GB │   ready    ││ Systran/faster-whisper-base                      │   0.1 GB │   ready    ││ antofuller/CROMA                                 │   0.7 GB │   ready    ││ cis-lmu/glotlid                                  │   1.6 GB │   ready    ││ google/gemma-2-2b-it                             │   4.9 GB │   ready    ││ google/gemma-3-12b-it                            │   0.0 GB │ incomplete ││ google/gemma-3-4b-it                             │   8.0 GB │   ready    ││ ibm-esa-geospatial/TerraMind-1.0-Tokenizer-S1GRD │   1.1 GB │   ready    ││ ibm-esa-geospatial/TerraMind-1.0-Tokenizer-S2L2A │   1.1 GB │   ready    ││ ibm-esa-geospatial/TerraMind-1.0-base            │   1.4 GB │   ready    ││ ibm-esa-geospatial/TerraMind-1.0-large           │   3.5 GB │   ready    ││ meta-llama/Llama-3.1-8B-Instruct                 │  15.0 GB │   ready    ││ meta-llama/Llama-3.2-1B-Instruct                 │   2.3 GB │   ready    ││ meta-llama/Llama-3.2-3B-Instruct                 │  12.0 GB │   ready    ││ meta-llama/Llama-3.3-70B-Instruct                │   0.0 GB │ incomplete ││ microsoft/Phi-3.5-mini-instruct                  │   7.1 GB │   ready    ││ microsoft/Phi-4-mini-instruct                    │   7.2 GB │   ready    ││ microsoft/Phi-4-mini-reasoning                   │   7.2 GB │   ready    ││ microsoft/Phi-4-reasoning                        │  27.3 GB │   ready    ││ microsoft/Phi-4-reasoning-plus                   │  27.3 GB │   ready    ││ microsoft/phi-4                                  │  27.3 GB │   ready    ││ mistralai/Mistral-Small-24B-Instruct-2501        │  87.8 GB │   ready    ││ openai/gpt-oss-120b                              │ 182.3 GB │   ready    ││ openai/gpt-oss-20b                               │  38.5 GB │   ready    ││ sentence-transformers/all-MiniLM-L6-v2           │   0.1 GB │   ready    ││ zss01/PixelCraft-3B                              │   7.6 GB │   ready    │└──────────────────────────────────────────────────┴──────────┴────────────┘

Local engines

Weights in the process, including on a Mac

A provider adapter calls someone else’s server. A local engine loads the weights where your code is running — no key, no network, and the model id is the repository id. 4 engines cover a GPU box, a throughput server, a laptop with no GPU and Apple Silicon.

engine="transformers"

The default. Runs the weights in-process on a local GPU, or on the CPU.

engine="vllm"

Higher throughput on the same weights, for a machine serving many requests.

engine="gguf"

Quantised GGUF weights through llama.cpp — a laptop, or a box with no GPU.

engine="mlx"

Apple Silicon, through MLX. A vision-language variant runs multimodal models.

python

from effgen.models import load_model

model = load_model("Qwen/Qwen2.5-0.5B-Instruct", engine="transformers")
print(model.generate("Name one primary colour.", max_tokens=16).text.strip())

what that printed

One primary color is blue.

Run for this page on a machine with the weights already in the local cache. The same call with engine="vllm" serves the same weights at higher throughput, engine="gguf" takes a quantised file through llama.cpp, and engine="mlx" runs on Apple Silicon.

effgen models browse --include-local adds what is already downloaded in the local Hugging Face cache to the table, so the zero-key option is visible beside the hosted ones.

The catalog and its prices

A price effGen does not have is never printed as $0

A published rate is shown as it is. A genuine free tier reads free. A model billed by something other than tokens reads metered. Anything else — including a catalog entry that carries an explicit zero with no free tier behind it — reads unpriced, because a fabricated $0 is the one answer that makes a spend total wrong without looking wrong.

effgen models browse

$ effgen models browse -t --max-price-in 0.10 --sort price-in --limit 10 Model Catalog                                      Models across providers — showing 10 of 55                                      ┏━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━┓┃ Provider ┃ Model ID                              ┃ Context ┃ Max Out ┃  $/1M in ┃ $/1M out ┃ Tools ┃ Vision ┃ Free ┃┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━┩│ hf       │ CohereLabs/c4ai-command-a-03-2025     │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │        │      ││ hf       │ CohereLabs/c4ai-command-r-08-2024     │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │        │      ││ hf       │ CohereLabs/c4ai-command-r7b-12-2024   │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │        │      ││ hf       │ CohereLabs/c4ai-command-r7b-arabic-02 │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │        │      ││          │ -2025                                 │         │         │          │          │       │        │      ││ hf       │ CohereLabs/command-a-reasoning-08-202 │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │        │      ││          │ 5                                     │         │         │          │          │       │        │      ││ hf       │ CohereLabs/command-a-translate-08-202 │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │        │      ││          │ 5                                     │         │         │          │          │       │        │      ││ together │ deepcogito/cogito-v1-preview-llama-70 │ 131,072 │   4,096 │ unpriced │ unpriced │   ✓   │        │      ││          │ B                                     │         │         │          │          │       │        │      ││ together │ deepcogito/cogito-v1-preview-llama-70 │ 131,072 │   4,096 │ unpriced │ unpriced │   ✓   │        │      ││          │ B-Turbo                               │         │         │          │          │       │        │      ││ together │ deepcogito/cogito-v1-preview-llama-8B │ 131,072 │   4,096 │ unpriced │ unpriced │   ✓   │        │      ││ hf       │ google/gemma-3-27b-it                 │   8,192 │   8,192 │ unpriced │ unpriced │   ✓   │   ✓    │      │└──────────┴───────────────────────────────────────┴─────────┴─────────┴──────────┴──────────┴───────┴────────┴──────┘ More: --offset 10 for the next page. Pricing source: catalog snapshot. Update: effgen models refresh  ·  Detail: effgen models info <id>

A price ceiling drops every model whose rate the catalog does not carry at all. What survives here are entries carrying an explicit zero — and the table refuses to call those $0.

effgen models browse

$ effgen models browse --free -t --limit 10 Model Catalog                                Models across providers — showing 5 of 5                                ┏━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━┓┃ Provider ┃ Model ID               ┃   Context ┃ Max Out ┃ $/1M in ┃ $/1M out ┃ Tools ┃ Vision ┃ Free ┃┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━┩│ cerebras │ gpt-oss-120b           │    65,536 │  32,768 │    free │     free │   ✓   │        │  ✓   ││ gemini   │ gemini-2.5-flash       │ 1,000,000 │  16,384 │    $0.3 │     $2.5 │   ✓   │   ✓    │  ✓   ││ gemini   │ gemini-2.5-flash-lite  │ 1,000,000 │  16,384 │    $0.1 │     $0.4 │   ✓   │   ✓    │  ✓   ││ gemini   │ gemini-3-flash-preview │ 1,000,000 │  32,768 │    $0.5 │       $3 │   ✓   │   ✓    │  ✓   ││ gemini   │ gemini-3.1-flash-lite  │ 1,000,000 │  32,768 │   $0.25 │     $1.5 │   ✓   │   ✓    │  ✓   │└──────────┴────────────────────────┴───────────┴─────────┴─────────┴──────────┴───────┴────────┴──────┘ Pricing source: catalog snapshot. Update: effgen models refresh  ·  Detail: effgen models info <id>

A genuine free tier is a different thing and is labelled as one. Filters compose: a model has to satisfy every one supplied.

Sort, filter, page

effgen models browse

$ effgen models browse --vision --min-context 200000 --sort context --desc --limit 10 Model Catalog                                      Models across providers — showing 10 of 58                                      ┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━┓┃ Provider  ┃ Model ID                           ┃   Context ┃ Max Out ┃  $/1M in ┃ $/1M out ┃ Tools ┃ Vision ┃ Free ┃┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━┩│ gemini    │ gemini-3.1-pro-preview             │ 2,000,000 │  65,536 │       $2 │      $12 │   ✓   │   ✓    │      ││ gemini    │ gemini-2.5-pro                     │ 2,000,000 │  32,768 │    $1.25 │      $10 │   ✓   │   ✓    │      ││ hf        │ meta-llama/Llama-Guard-4-12B       │ 1,048,576 │  16,384 │     $0.2 │     $0.2 │       │   ✓    │      ││ hf        │ meta-llama/Llama-4-Maverick-17B-12 │ 1,048,576 │  16,384 │    $0.27 │    $0.85 │       │   ✓    │      ││           │ 8E-Instruct-FP8                    │           │         │          │          │       │        │      ││ fireworks │ accounts/fireworks/models/kimi-k3  │ 1,048,576 │  16,384 │ unpriced │ unpriced │   ✓   │   ✓    │      ││ fireworks │ accounts/fireworks/models/inkling  │ 1,048,576 │  16,384 │ unpriced │ unpriced │   ✓   │   ✓    │      ││ openai    │ gpt-5.4-pro                        │ 1,047,576 │  32,768 │      $30 │     $180 │   ✓   │   ✓    │      ││ openai    │ gpt-5.4-nano                       │ 1,047,576 │  32,768 │     $0.2 │    $1.25 │   ✓   │   ✓    │      ││ openai    │ gpt-5.4-mini                       │ 1,047,576 │  32,768 │    $0.75 │     $4.5 │   ✓   │   ✓    │      ││ openai    │ gpt-5.4                            │ 1,047,576 │  32,768 │     $2.5 │      $15 │   ✓   │   ✓    │      │└───────────┴────────────────────────────────────┴───────────┴─────────┴──────────┴──────────┴───────┴────────┴──────┘ More: --offset 10 for the next page. Pricing source: catalog snapshot. Update: effgen models refresh  ·  Detail: effgen models info <id>

A missing number sorts last on an ascending sort, so an unknown price never masquerades as the cheapest.

effgen models browse

$ effgen models browse --provider cerebras Model Catalog                          Models across providers — showing 2 of 2                          ┏━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━┓┃ Provider ┃ Model ID     ┃ Context ┃ Max Out ┃ $/1M in ┃ $/1M out ┃ Tools ┃ Vision ┃ Free ┃┡━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━┩│ cerebras │ gpt-oss-120b │  65,536 │  32,768 │    free │     free │   ✓   │        │  ✓   ││ cerebras │ zai-glm-4.7  │  65,536 │  40,960 │    free │     free │       │        │  ✓   │└──────────┴──────────────┴─────────┴─────────┴─────────┴──────────┴───────┴────────┴──────┘ Pricing source: catalog snapshot. Update: effgen models refresh  ·  Detail: effgen models info <id>

effgen models browse

$ effgen models browse --search qwen --limit 10 Model Catalog                                      Models across providers — showing 10 of 99                                      ┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━┳━━━━━━┓┃ Provider  ┃ Model ID                             ┃ Context ┃ Max Out ┃  $/1M in ┃ $/1M out ┃ Tools ┃ Vision ┃ Free ┃┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━╇━━━━━━┩│ fireworks │ accounts/fireworks/models/qwen3-embe │  40,960 │       — │     $0.1 │       $0 │       │        │      ││           │ dding-8b                             │         │         │          │          │       │        │      ││ fireworks │ accounts/fireworks/models/qwen3-rera │  40,960 │       — │     $0.1 │       $0 │       │        │      ││           │ nker-8b                              │         │         │          │          │       │        │      ││ fireworks │ accounts/fireworks/models/qwen3p7-pl │       — │  16,384 │ unpriced │ unpriced │   ✓   │   ✓    │      ││           │ us                                   │         │         │          │          │       │        │      ││ groq      │ qwen/qwen3.6-27b                     │ 131,072 │  16,384 │     $0.6 │       $3 │   ✓   │   ✓    │      ││ hf        │ aisingapore/Qwen-SEA-LION-v4-32B-IT  │   8,192 │   8,192 │    $0.25 │     $0.5 │   ✓   │        │      ││ hf        │ deepseek-ai/DeepSeek-R1-Distill-Qwen │ 131,072 │  16,384 │     $0.1 │     $0.1 │       │        │      ││           │ -1.5B                                │         │         │          │          │       │        │      ││ hf        │ deepseek-ai/DeepSeek-R1-Distill-Qwen │ 131,072 │  16,384 │     $0.2 │     $0.2 │       │        │      ││           │ -14B                                 │         │         │          │          │       │        │      ││ hf        │ deepseek-ai/DeepSeek-R1-Distill-Qwen │ 131,072 │  16,384 │     $0.3 │     $0.3 │       │        │      ││           │ -32B                                 │         │         │          │          │       │        │      ││ hf        │ deepseek-ai/DeepSeek-R1-Distill-Qwen │ 131,072 │  16,384 │    $0.15 │    $0.15 │       │        │      ││           │ -7B                                  │         │         │          │          │       │        │      ││ hf        │ Qwen/Qwen2.5-72B-Instruct            │  32,000 │  16,384 │    $0.38 │     $0.4 │   ✓   │        │      │└───────────┴──────────────────────────────────────┴─────────┴─────────┴──────────┴──────────┴───────┴────────┴──────┘ More: --offset 10 for the next page. Pricing source: catalog snapshot. Update: effgen models refresh  ·  Detail: effgen models info <id>

Every filter `browse` takes

Flag	Type	Description
`--search TEXT`	—	Case-insensitive substring match on model id, family, or provider
`--provider PROVIDER`	—	Limit to one provider
`--free`	—	Only free-tier models
`-t, --tools`	—	Only tool-calling models
`--vision`	—	Only vision-capable models
`--audio`	—	Only audio-capable models
`--min-context N`	—	Only models with a context window of at least N tokens
`--max-price-in USD`	—	Only models whose input price ($/1M) is at most USD
`--max-price-out USD`	—	Only models whose output price ($/1M) is at most USD
`--sort {provider,id,context,max-out,price-in,price-out}`	—	Sort order (default: provider then id)
`--desc`	—	Sort in descending order
`--limit N`	—	Show at most N rows
`--offset N`	—	Skip the first N rows (paging)
`--include-local`	—	Also list models downloaded in the local HuggingFace cache
`--json`	—	Output as JSON

effgen models browse --help · effGen 1.0.0

One model, in detail

effgen models info

$ effgen models info openai:gpt-5-mini Model: openai:gpt-5-mini┌───────────────────────────┬─────────────────┐│ Provider                  │ openai          ││ Display name              │ gpt-5-mini      ││ Family                    │ chat            ││ Context window            │ 1,047,576       ││ Max output                │ 32,768          ││ Price ($/1M in / out)     │ $0.25/$2        ││ Tool calling              │ yes             ││ Coding                    │ suitable        ││ Vision                    │ yes             ││ Audio                     │ no              ││ Free tier                 │ no              ││ Rate limits (rpm/tpm/rpd) │ — / — / —       ││ Deprecated                │ no              ││ Price source              │ bundled-catalog ││ Verified on               │ 2026-06-17      ││ Auth ready                │ yes             │└───────────────────────────┴─────────────────┘ Use: effgen run --provider openai -m gpt-5-mini "..."

The same catalog in the browser

The Model catalog panel, headed: 417 models across 9 providers, pricing from catalog snapshot verified 2026-06-08, run effgen models refresh to update. Under it a search box, a provider picker, a sort picker, tools, vision, audio and free checkboxes and a minimum-context field, then a table of provider, model id, context, max output, input and output price per million tokens, and tool and vision ticks. The pager reads 1 to 25 of 417. — The dashboard’s catalog panel reads the same source, with the same pricing labels and the same filters.`GET /dashboard/catalog.json`

The web surfaces

Routing

Name more than one model, and a router is built across them

AgentConfig(models=[...]) loads every model named and routes between them. For a policy rather than a list, ModelRouter selects on capability, cost or measured latency, and records why every candidate it rejected was rejected.

python

from effgen import Agent, AgentConfig

with Agent(AgentConfig(
    model="gemini:gemini-3.1-flash-lite",
    models=["openai:gpt-5-nano"],
)) as agent:
    r = agent.run("Reply with the single word ok.")
    print(r.output)
    print(r.metadata["total_tokens"], "tokens ·  $%.6f" % r.metadata["cost_usd"])

what that printed

ok
20 tokens ·  $0.000006

A model in the list that cannot be loaded is reported and skipped rather than taking the agent down with it, so a list can name a model that is not configured on every machine.

Three policies

FirstAvailablePolicy takes the first provider with a configured key that supports the capabilities asked for. CostBasedPolicy takes the cheapest that fits the budget, ranking a free tier ahead of a paid one and breaking remaining ties deterministically. LatencyBasedPolicy takes the fastest that meets a latency budget, from measured p50s rather than from seeds once any real measurement exists.

Every rejection is on the record

A RouterDecision carries the pair it chose, the policy that chose it, that policy’s score, and eliminated — one entry per rejected candidate with the reason: no key and the variable it would come from, a capability the model does not have, a model that needs a dedicated endpoint, or simply not the cheapest.

When nothing fits the budget, NoCandidateWithinBudgetError is raised carrying the cheapest option that exists, so the caller can say what the budget would have to be.

docs/models/router.md

When it does not work

A backend that never answered is not a result

A task that ran and failed is something you can inspect. A connection that was refused, a host that does not resolve and a route that does not exist are not, and returning one quietly is how a whole batch completes against nothing and looks healthy in the summary. So they raise BackendUnreachableError — whatever raise_on_error says.

python

from effgen import Agent, AgentConfig
from effgen.models.errors import BackendUnreachableError

agent = Agent(AgentConfig(
    model="Qwen/Qwen2.5-7B-Instruct",
    base_url="http://127.0.0.1:9/v1",
    raise_on_error=False,
))
try:
    agent.run("What is 6 times 7?")
except BackendUnreachableError as e:
    print(type(e).__name__)
    print(str(e).split(". ")[-1])
finally:
    agent.close()

what that printed

BackendUnreachableError
Nothing answered at that endpoint — check the server is running and the base_url, host and port are right.

A model id that is not there names the ones that are

bash

curl -s http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"cerebras:llama3.1-8b","messages":[{"role":"user","content":"hello"}]}'

the 404 body, verbatim

{"error":{"message":"cerebras error (model='llama3.1-8b'): Unknown Cerebras model 'llama3.1-8b'. Did you mean: zai-glm-4.7, gpt-oss-120b? Available cerebras models: gpt-oss-120b, zai-glm-4.7. Model id not found — run `effgen models list` to see ids, `effgen models refresh` to update the catalog, and verify the id/provider prefix.","type":"invalid_request_error","param":null,"code":"not_found"}}

And no endpoint at all names the variable

python

from effgen.models import load_model

try:
    load_model("my-model", provider="openai_compatible")
except ValueError as e:
    print(e)

what that printed

An OpenAI-compatible endpoint needs a base_url. Pass base_url='http://host:port/v1', or set EFFGEN_BASE_URL (or OPENAI_BASE_URL) in the environment. To call OpenAI itself, use provider='openai' instead.

The reference for models

The registry and how a model id is resolved, one page per provider with its rate limits and its quirks, the tool-call dialects, and the router with its policies and its capability matrix.

docs/models/openai-compatible.md — pointing effGen at your own server

docs/models/registry.md

Any model, anywhere you already run it

One argument, and effGen drives the server you already have

Why serve the model separately

Ask the endpoint what it serves

Where the endpoint comes from

What effGen refuses to assume about your server

The context window is a guess, and it says so

Cost: nothing rather than $0

Ids, sampling and reasoning

10 adapters, 9 with a catalog

OpenAI

Anthropic

Gemini

Groq

Cerebras

Together

Fireworks

Hugging Face

Replicate

Any OpenAI-protocol server

Weights in the process, including on a Mac

A price effGen does not have is never printed as $0

Sort, filter, page

Every filter browse takes

One model, in detail

The same catalog in the browser

Name more than one model, and a router is built across them

Three policies

Every rejection is on the record

A backend that never answered is not a result

A model id that is not there names the ones that are

And no endpoint at all names the variable

The reference for models

Every filter `browse` takes