Skip to content

About

LangChain tools for ZeroGPU's nano language models — classify, summarize, extract PII, and more.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

ZeroGPU

langchain-zerogpu

LangChain tools for ZeroGPU.

ZeroGPU is a compute-efficient inference provider for apps and agents. We run purpose-built small and nano language models across an edge network for the high-volume tasks you run constantly — classification, extraction, moderation, routing, summarization — at ~10x lower latency and 50%+ lower cost than frontier-model workflows. Auto-scaling, with zero GPU infrastructure to manage. Plug in and you're live.

This package exposes those models as first-class LangChain BaseTool subclasses, so any LangChain agent — including create_agent and LangGraph graphs — can offload these repeatable NLP tasks (classification, summarization, moderation, routing decisions, signal / entity / JSON extraction, PII redaction, and chat) to ZeroGPU instead of spending frontier-model tokens — plus larger open-weight models for the calls that do need reasoning headroom.

All calls go through the official zerogpu-api Python SDK.

Install

pip install langchain-zerogpu

Authenticate

Every request needs a ZeroGPU API key (starts with zgpu-api-). A project id is optional — set one to scope requests to a specific project. Provide them via environment variables:

export ZEROGPU_API_KEY="zgpu-api-..."
export ZEROGPU_PROJECT_ID="your-project-id"   # optional

…or pass them directly to any tool or the toolkit:

from langchain_zerogpu import ZeroGPUSummarizeTool

tool = ZeroGPUSummarizeTool(api_key="zgpu-api-...", project_id="your-project-id")

The API key is stored as a pydantic.SecretStr and is never logged.

The tools

Tool class ZeroGPU model Purpose
ZeroGPUChatTool LFM2.5-1.2B-Instruct Short single-turn chat reply
ZeroGPUChatThinkingTool LFM2.5-1.2B-Thinking Chat with a visible reasoning trace
ZeroGPUReasonTool gpt-oss-120b Heavier reasoning, 131K context
ZeroGPUReasonMultilingualTool qwen3-30b-a3b-fp8 Reasoning across 100+ languages
ZeroGPUReasonQwenTool qwen3.6-35b-a3b Fast 35B MoE reasoning, 16K context
ZeroGPUReasonGLMTool glm-5.3-flash Coding and agentic work, 1M context
ZeroGPUReasonDeepseekTool deepseek-v4.1-flash DeepSeek V4.1 Flash reasoning, 1M context
ZeroGPUReasonGPTLunaTool gpt-5.6-luna Cost-optimized GPT-5.6, 272K context
ZeroGPUReasonGPT6LunaTool gpt-6-luna Most efficient GPT-6, 272K context
ZeroGPUReasonGPTMiniTool gpt-4.1-mini GPT-4.1 mini, 1M context
ZeroGPUReasonGPTNanoTool gpt-5.4-nano Cost-efficient GPT-5.4, 400K context
ZeroGPUModerateTool llama-guard-4-12b Safe / unsafe moderation verdict, 160K context
ZeroGPUSummarizeTool llama-3.1-8b-instruct-fast Condense a passage
ZeroGPUSummarize3BTool llama-3.2-3b-instruct-fast Condense a passage with a cheaper 3B model
ZeroGPUClassifyIABTool zlm-v1-iab-classify-edge IAB taxonomy classification
ZeroGPUClassifyIABEnrichedTool zlm-v2-iab-classify-edge-enriched IAB + topics / keywords / intent
ZeroGPUClassifyDomainTool zlm-v1-iab-domain-classifier IAB classification from a domain name
ZeroGPUExtractSignalsTool zlm-v1-signal-extract Topics / keywords / intent signals
ZeroGPUDecideToolsTool zlm-v1-tool-decisions Pick a tool and generate its arguments
ZeroGPUClassifyZeroShotTool deberta-v3-small Zero-shot vs. custom labels
ZeroGPUClassifyStructuredTool gliner2.5-base-v1 Multi-axis schema classification
ZeroGPUDecideTool gliner2.5-decide-v1 Zero-shot decisions and routing
ZeroGPUExtractEntitiesTool gliner2.5-base-v1 Custom-label NER
ZeroGPUExtractPIITool gliner2.5-multi-v1 Extract PII entities (JSON)
ZeroGPURedactPIITool gliner2.5-multi-v1 Mask PII inline with [LABEL]
ZeroGPUExtractJSONTool gliner2.5-base-v1 Schema-driven JSON extraction

Quick start

from langchain_zerogpu import ZeroGPUClassifyZeroShotTool

tool = ZeroGPUClassifyZeroShotTool()  # reads creds from the environment

print(tool.invoke({
    "text": "The new GPU smashes every benchmark we threw at it.",
    "labels": ["tech", "politics", "sports"],
}))

Tools work asynchronously too:

result = await tool.ainvoke({"text": "...", "labels": ["a", "b"]})

Bind the tools to an agent

Use the toolkit to get all 26 tools — wired to a single shared client — and bind them to an agent:

from langchain.agents import create_agent
from langchain_zerogpu import ZeroGPUToolkit

toolkit = ZeroGPUToolkit()  # reads ZEROGPU_API_KEY / ZEROGPU_PROJECT_ID
tools = toolkit.get_tools()

agent = create_agent("anthropic:claude-sonnet-4-6", tools=tools)

agent.invoke({
    "messages": [
        {"role": "user", "content": "Redact the PII in: 'Call Jane at 555-0100.'"}
    ]
})

Or bind a single tool to a chat model directly:

from langchain.chat_models import init_chat_model
from langchain_zerogpu import ZeroGPUExtractPIITool

llm = init_chat_model("anthropic:claude-sonnet-4-6")
llm_with_tools = llm.bind_tools([ZeroGPUExtractPIITool()])

Errors

Failures surface as clear, typed exceptions instead of raw stack traces:

  • ZeroGPUAuthError — missing / malformed credentials, 401, or 403.
  • ZeroGPUError — rate limits (429), server errors (5xx), and network failures.

Development

make install            # uv sync --all-groups
make lint               # ruff check + format --check
make mypy               # mypy (disallow_untyped_defs)
make test               # unit tests, sockets disabled
make integration_test   # integration tests (needs real ZeroGPU creds)
make bump               # bump the version; merging to main releases it

Releases are automatic — see RELEASING.md.

License

MIT — see LICENSE.

About

LangChain tools for ZeroGPU's nano language models — classify, summarize, extract PII, and more.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages