This project has been created as part of the 42 curriculum by mapadron.
- About the Project
- Features
- Installation & Usage
- How It Works
- Design Decisions
- Performance Analysis
- Challenges Faced
- Resources
- Example usage
Call Me Maybe is a technical project focused on the intersection of Natural Language Processing and structured data extraction. Using a small-scale Large Language Model (LLM) with 0.6B parameters, the project demonstrates how to implement constrained decoding to convert unstructured user prompts into valid, type-safe function calls. The primary goal is to achieve high reliability and strict schema compliance, ensuring that even a model with limited parameters can serve as a dependable interface for programmatic API calls.
-
Grammar-Constrained Decoding: Implements a token-level logit processor that enforces a context-free grammar, ensuring the model's output is mathematically incapable of violating the predefined JSON schema.
-
Trie-Based Prefix Filtering:
- Deterministic Selection: Uses a prefix tree (Trie) to mask the model's vocabulary, forcing 100% accuracy in function name identification.
- Vocabulary Pruning: Dynamically reduces the search space at each generation step, preventing the model from wandering into "hallucinated" function names.
-
Strict Type-Safe Extraction:
- Numeric Enforcement: A specialized masking layer that limits the model to numeric tokens (digits, decimals) when a
numbertype is expected. - String Priming: Automatically injects structural delimiters (like quotes) to steer the model into "content mode," preventing the generation of meta-text or type descriptions.
- Numeric Enforcement: A specialized masking layer that limits the model to numeric tokens (digits, decimals) when a
-
Robust Resource Management:
- Fail-Fast Validation: Integrates Pydantic for real-time schema validation, ensuring data integrity before any disk write operations.
- Graceful Exception Handling: A terminal-safe architecture that catches
OSErrorandJSONDecodeError, providing specific feedback viastderrrather than generic crashes.
-
Deterministic Inference Engine: A discrete-time generation loop using Argmax Selection to ensure consistent, reproducible results across multiple runs of the same prompt.
-
Production-Grade Infrastructure: Fully compliant with
flake8andmypy --strictstandards. The codebase follows strict Object-Oriented Programming (OOP) principles and PEP 257 documentation standards.
The project relies on uv for environment management. You can set up the environment using the provided Makefile:
# Install dependencies and sync the local llm_sdk
make install
Alternatively, you can run the sync command directly:
uv sync
The program is designed to be executed as a Python module. It accepts three optional flags to specify the input prompts, the output destination, and the function definitions.
# Standard execution format with optional flags
uv run python -m src [--functions_definition <functions_definition_file>] [--input <input_file>] [--output <output_file>]
Example:
uv run python -m src --functions_definition data/function_definitions.json --input data/input.json --output data/output/results.json
Using the Makefile: A shortcut is provided to run the project with the default evaluation paths:
# Runs with default paths for input, output, and definitions
make run
To ensure the code meets the 42 curriculum's strict quality standards, use the following commands:
# Run mandatory linting (flake8 and mypy --strict)
make lint
make lint-strict
# Remove __pycache__ and build artifacts
make clean
The core of this project is a Constrained Decoding Engine. Small models (0.6B) are prone to "syntax drift"—losing track of JSON braces or hallucinating function names. To prevent this, I implemented a two-stage Logit Masking system that intervenes before the model even selects its next token.
Instead of allowing the model to freely generate text, the engine uses a Prefix-Matching Trie. At each generation step:
- The engine identifies all valid tokens that could continue a function name from the schema.
- All other tokens in the vocabulary are "masked" (set to
$-\infty$ ). - This effectively forces the model to traverse a path that leads exclusively to a valid function name.
Once a function is selected, the engine enforces strict data typing for each parameter:
- Numeric Fields: The engine masks all non-numeric tokens, allowing only
0-9,., and JSON separators. This guarantees a valid float/integer string. - String Fields: The engine "primes" the model by forcing a leading quote (
") and implements a stop-on-delimiter logic. The generation breaks immediately upon the first unescaped closing quote, preventing the model from leaking metadata into the value.
- Pydantic for Schema Integrity: Used to define internal data structures, providing a "fail-fast" mechanism. If the extracted data violates the schema, Pydantic catches it during instantiation, preventing corrupt JSON writes.
- O(1) Vocabulary Lookup: I reconstructed the SDK’s JSON vocabulary into a flattened
vocab_list. This allows for near-instant validation of token strings during the logit masking loop. - Argmax Selection: I chose
np.argmaxover stochastic sampling. While less "creative," it ensures determinism—critical for a system meant to function as a reliable API bridge.
- Accuracy: Achieved 100% success in function identification via the Trie filter and high reliability in parameter extraction for complex types (e.g., Regex).
- Latency: By pruning the search space through masking, the model avoids generating redundant or irrelevant tokens, keeping the inference time optimal for a CPU-bound 0.6B model.
- Resilience: The integration of
OSErrorhandling and Pydantic validation ensures the program reports errors tostderrand exits gracefully rather than crashing.
- The Regex Infinite Loop: Complex patterns often caused the model to repeat characters. I mitigated this by implementing a
max_tokenssafety cap and strict delimiter-break logic. - Mypy strictness with Local SDK: Integrating a local package (
llm_sdk) while maintaining--strictcompliance required utilizingtype: ignoreand careful import management to satisfy the type checker. - Semantic "Type Traps": The model occasionally generated the word
"string"instead of an actual string value. Refining the prompt context and removing redundant type-labels forced the model to focus on value extraction.
- Qwen3 Model — the LLM model used in this project
- Documentation: Python
jsonlibrary,Pydanticv2 docs, andnumpylogit manipulation. - Videos:
- Taking Control of LLM Outputs: An Introductory Journey into Logits – A deep dive into the theoretical and practical implementation of logit manipulation.
- AI Usage: AI was used for:
- Generating PEP 257 compliant docstrings and asisting with writing the README.
- Providing useful resources to deeper understand the underlaying theory of the project.
- Giving useful information and clearing doubts on theorecical concepts regarding LLMs.
To run a minimal test of the project, ensure you have the following three files created in your directory.
data/example_definitions.json (Function Schema)
[
{
"name": "fn_add_numbers",
"description": "Add two numbers together and return their sum.",
"parameters": {
"a": {
"type": "number"
},
"b": {
"type": "number"
}
},
"returns": {
"type": "number"
}
}
]
data/example_input.json (User Prompts)
[
{ "prompt": "What is the sum of 10 and 21?" }
]
Execute the module using the named flags. The program will process the prompt using the 0.6B model and extract the structured arguments.
uv run python3 -m src \
--input data/example_input.json \
--functions_definition data/example_definitions.json \
--output data/output/example_results.json
The resulting data/output/example_results.json will contain the mapped function call:
[
{
"prompt": "What is the sum of 10 and 21?",
"name": "fn_add_numbers",
"parameters": {
"a": 10.0,
"b": 21.0
}
}
]