LangChain tools for ZeroGPU.
ZeroGPU is a compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge network for the high-volume tasks you run constantly — classification, extraction, moderation, routing, summarization — at ~10x lower latency and 50%+ lower cost than frontier-model workflows. Auto-scaling, with zero GPU infrastructure to manage. Plug in and you're live.
This package exposes those models as first-class LangChain
BaseTool subclasses, so
any LangChain agent — including create_agent and LangGraph graphs — can offload
these repeatable NLP tasks (classification, summarization, moderation, routing
decisions, signal / entity / JSON extraction, PII redaction, and chat) to
ZeroGPU instead of
spending frontier-model tokens — plus larger open-weight models for the calls
that do need reasoning headroom.
All calls go through the official zerogpu-api
Python SDK.
pip install langchain-zerogpuEvery request needs a ZeroGPU API key (starts with zgpu-api-). A
project id is optional — set one to scope requests to a specific project.
Provide them via environment variables:
export ZEROGPU_API_KEY="zgpu-api-..."
export ZEROGPU_PROJECT_ID="your-project-id" # optional…or pass them directly to any tool or the toolkit:
from langchain_zerogpu import ZeroGPUSummarizeTool
tool = ZeroGPUSummarizeTool(api_key="zgpu-api-...", project_id="your-project-id")The API key is stored as a pydantic.SecretStr and is never logged.
| Tool class | ZeroGPU model | Purpose |
|---|---|---|
ZeroGPUChatTool |
LFM2.5-1.2B-Instruct |
Short single-turn chat reply |
ZeroGPUChatThinkingTool |
LFM2.5-1.2B-Thinking |
Chat with a visible reasoning trace |
ZeroGPUReasonTool |
gpt-oss-120b |
Heavier reasoning, 131K context |
ZeroGPUReasonMultilingualTool |
qwen3-30b-a3b-fp8 |
Reasoning across 100+ languages |
ZeroGPUReasonQwenTool |
qwen3.6-35b-a3b |
Fast 35B MoE reasoning, 16K context |
ZeroGPUReasonGLMTool |
glm-5.3-flash |
Coding and agentic work, 1M context |
ZeroGPUReasonDeepseekTool |
deepseek-v4.1-flash |
DeepSeek V4.1 Flash reasoning, 1M context |
ZeroGPUReasonGPTLunaTool |
gpt-5.6-luna |
Cost-optimized GPT-5.6, 272K context |
ZeroGPUReasonGPT6LunaTool |
gpt-6-luna |
Most efficient GPT-6, 272K context |
ZeroGPUReasonGPTMiniTool |
gpt-4.1-mini |
GPT-4.1 mini, 1M context |
ZeroGPUReasonGPTNanoTool |
gpt-5.4-nano |
Cost-efficient GPT-5.4, 400K context |
ZeroGPUModerateTool |
llama-guard-4-12b |
Safe / unsafe moderation verdict, 160K context |
ZeroGPUSummarizeTool |
llama-3.1-8b-instruct-fast |
Condense a passage |
ZeroGPUSummarize3BTool |
llama-3.2-3b-instruct-fast |
Condense a passage with a cheaper 3B model |
ZeroGPUClassifyIABTool |
zlm-v1-iab-classify-edge |
IAB taxonomy classification |
ZeroGPUClassifyIABEnrichedTool |
zlm-v2-iab-classify-edge-enriched |
IAB + topics / keywords / intent |
ZeroGPUClassifyDomainTool |
zlm-v1-iab-domain-classifier |
IAB classification from a domain name |
ZeroGPUExtractSignalsTool |
zlm-v1-signal-extract |
Topics / keywords / intent signals |
ZeroGPUDecideToolsTool |
zlm-v1-tool-decisions |
Pick a tool and generate its arguments |
ZeroGPUClassifyZeroShotTool |
deberta-v3-small |
Zero-shot vs. custom labels |
ZeroGPUClassifyStructuredTool |
gliner2.5-base-v1 |
Multi-axis schema classification |
ZeroGPUDecideTool |
gliner2.5-decide-v1 |
Zero-shot decisions and routing |
ZeroGPUExtractEntitiesTool |
gliner2.5-base-v1 |
Custom-label NER |
ZeroGPUExtractPIITool |
gliner2.5-multi-v1 |
Extract PII entities (JSON) |
ZeroGPURedactPIITool |
gliner2.5-multi-v1 |
Mask PII inline with [LABEL] |
ZeroGPUExtractJSONTool |
gliner2.5-base-v1 |
Schema-driven JSON extraction |
from langchain_zerogpu import ZeroGPUClassifyZeroShotTool
tool = ZeroGPUClassifyZeroShotTool() # reads creds from the environment
print(tool.invoke({
"text": "The new GPU smashes every benchmark we threw at it.",
"labels": ["tech", "politics", "sports"],
}))Tools work asynchronously too:
result = await tool.ainvoke({"text": "...", "labels": ["a", "b"]})Use the toolkit to get all 26 tools — wired to a single shared client — and bind them to an agent:
from langchain.agents import create_agent
from langchain_zerogpu import ZeroGPUToolkit
toolkit = ZeroGPUToolkit() # reads ZEROGPU_API_KEY / ZEROGPU_PROJECT_ID
tools = toolkit.get_tools()
agent = create_agent("anthropic:claude-sonnet-4-6", tools=tools)
agent.invoke({
"messages": [
{"role": "user", "content": "Redact the PII in: 'Call Jane at 555-0100.'"}
]
})Or bind a single tool to a chat model directly:
from langchain.chat_models import init_chat_model
from langchain_zerogpu import ZeroGPUExtractPIITool
llm = init_chat_model("anthropic:claude-sonnet-4-6")
llm_with_tools = llm.bind_tools([ZeroGPUExtractPIITool()])Failures surface as clear, typed exceptions instead of raw stack traces:
ZeroGPUAuthError— missing / malformed credentials,401, or403.ZeroGPUError— rate limits (429), server errors (5xx), and network failures.
make install # uv sync --all-groups
make lint # ruff check + format --check
make mypy # mypy (disallow_untyped_defs)
make test # unit tests, sockets disabled
make integration_test # integration tests (needs real ZeroGPU creds)
make bump # bump the version; merging to main releases itReleases are automatic — see RELEASING.md.
MIT — see LICENSE.