Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

14 Commits

Folders and files

Repository files navigation

Call Me Maybe

C 42 Project Score Badge

This project has been created as part of the 42 curriculum by mapadron.

📑 Table of Contents

📖 About the Project

Call Me Maybe is a technical project focused on the intersection of Natural Language Processing and structured data extraction. Using a small-scale Large Language Model (LLM) with 0.6B parameters, the project demonstrates how to implement constrained decoding to convert unstructured user prompts into valid, type-safe function calls. The primary goal is to achieve high reliability and strict schema compliance, ensuring that even a model with limited parameters can serve as a dependable interface for programmatic API calls.

✨ Features

  • Grammar-Constrained Decoding: Implements a token-level logit processor that enforces a context-free grammar, ensuring the model's output is mathematically incapable of violating the predefined JSON schema.

  • Trie-Based Prefix Filtering:

    • Deterministic Selection: Uses a prefix tree (Trie) to mask the model's vocabulary, forcing 100% accuracy in function name identification.
    • Vocabulary Pruning: Dynamically reduces the search space at each generation step, preventing the model from wandering into "hallucinated" function names.
  • Strict Type-Safe Extraction:

    • Numeric Enforcement: A specialized masking layer that limits the model to numeric tokens (digits, decimals) when a number type is expected.
    • String Priming: Automatically injects structural delimiters (like quotes) to steer the model into "content mode," preventing the generation of meta-text or type descriptions.
  • Robust Resource Management:

    • Fail-Fast Validation: Integrates Pydantic for real-time schema validation, ensuring data integrity before any disk write operations.
    • Graceful Exception Handling: A terminal-safe architecture that catches OSError and JSONDecodeError, providing specific feedback via stderr rather than generic crashes.
  • Deterministic Inference Engine: A discrete-time generation loop using Argmax Selection to ensure consistent, reproducible results across multiple runs of the same prompt.

  • Production-Grade Infrastructure: Fully compliant with flake8 and mypy --strict standards. The codebase follows strict Object-Oriented Programming (OOP) principles and PEP 257 documentation standards.

🛠️ Installation & Usage

Installation

The project relies on uv for environment management. You can set up the environment using the provided Makefile:

# Install dependencies and sync the local llm_sdk
make install

Alternatively, you can run the sync command directly:

uv sync

Execution

The program is designed to be executed as a Python module. It accepts three optional flags to specify the input prompts, the output destination, and the function definitions.

# Standard execution format with optional flags
uv run python -m src [--functions_definition <functions_definition_file>] [--input <input_file>] [--output <output_file>]

Example:

uv run python -m src --functions_definition data/function_definitions.json --input data/input.json --output data/output/results.json

Using the Makefile: A shortcut is provided to run the project with the default evaluation paths:

# Runs with default paths for input, output, and definitions
make run

Maintenance and Linting

To ensure the code meets the 42 curriculum's strict quality standards, use the following commands:

# Run mandatory linting (flake8 and mypy --strict)
make lint
make lint-strict

# Remove __pycache__ and build artifacts
make clean

🧠 How It Works

The core of this project is a Constrained Decoding Engine. Small models (0.6B) are prone to "syntax drift"—losing track of JSON braces or hallucinating function names. To prevent this, I implemented a two-stage Logit Masking system that intervenes before the model even selects its next token.

1. Function Selection (Trie Filtering)

Instead of allowing the model to freely generate text, the engine uses a Prefix-Matching Trie. At each generation step:

  • The engine identifies all valid tokens that could continue a function name from the schema.
  • All other tokens in the vocabulary are "masked" (set to $-\infty$).
  • This effectively forces the model to traverse a path that leads exclusively to a valid function name.

2. Parameter Extraction (Grammar Enforcement)

Once a function is selected, the engine enforces strict data typing for each parameter:

  • Numeric Fields: The engine masks all non-numeric tokens, allowing only 0-9, ., and JSON separators. This guarantees a valid float/integer string.
  • String Fields: The engine "primes" the model by forcing a leading quote (") and implements a stop-on-delimiter logic. The generation breaks immediately upon the first unescaped closing quote, preventing the model from leaking metadata into the value.

🛠 Design Decisions

  • Pydantic for Schema Integrity: Used to define internal data structures, providing a "fail-fast" mechanism. If the extracted data violates the schema, Pydantic catches it during instantiation, preventing corrupt JSON writes.
  • O(1) Vocabulary Lookup: I reconstructed the SDK’s JSON vocabulary into a flattened vocab_list. This allows for near-instant validation of token strings during the logit masking loop.
  • Argmax Selection: I chose np.argmax over stochastic sampling. While less "creative," it ensures determinism—critical for a system meant to function as a reliable API bridge.

📈 Performance Analysis

  • Accuracy: Achieved 100% success in function identification via the Trie filter and high reliability in parameter extraction for complex types (e.g., Regex).
  • Latency: By pruning the search space through masking, the model avoids generating redundant or irrelevant tokens, keeping the inference time optimal for a CPU-bound 0.6B model.
  • Resilience: The integration of OSError handling and Pydantic validation ensures the program reports errors to stderr and exits gracefully rather than crashing.

🚧 Challenges Faced

  • The Regex Infinite Loop: Complex patterns often caused the model to repeat characters. I mitigated this by implementing a max_tokens safety cap and strict delimiter-break logic.
  • Mypy strictness with Local SDK: Integrating a local package (llm_sdk) while maintaining --strict compliance required utilizing type: ignore and careful import management to satisfy the type checker.
  • Semantic "Type Traps": The model occasionally generated the word "string" instead of an actual string value. Refining the prompt context and removing redundant type-labels forced the model to focus on value extraction.

📚 Resources

  • Qwen3 Model — the LLM model used in this project
  • Documentation: Python json library, Pydantic v2 docs, and numpy logit manipulation.
  • Videos:
  • AI Usage: AI was used for:
    • Generating PEP 257 compliant docstrings and asisting with writing the README.
    • Providing useful resources to deeper understand the underlaying theory of the project.
    • Giving useful information and clearing doubts on theorecical concepts regarding LLMs.

🚀 Example Usage

To run a minimal test of the project, ensure you have the following three files created in your directory.

1. Create Example Files

data/example_definitions.json (Function Schema)

[
  {
    "name": "fn_add_numbers",
    "description": "Add two numbers together and return their sum.",
    "parameters": {
      "a": {
        "type": "number"
      },
      "b": {
        "type": "number"
      }
    },
    "returns": {
      "type": "number"
    }
  }
]

data/example_input.json (User Prompts)

[
  { "prompt": "What is the sum of 10 and 21?" }
]

2. Run the Extraction

Execute the module using the named flags. The program will process the prompt using the 0.6B model and extract the structured arguments.

uv run python3 -m src \
  --input data/example_input.json \
  --functions_definition data/example_definitions.json \
  --output data/output/example_results.json

3. Verify the Result

The resulting data/output/example_results.json will contain the mapped function call:

[
  {
    "prompt": "What is the sum of 10 and 21?",
    "name": "fn_add_numbers",
    "parameters": {
      "a": 10.0,
      "b": 21.0
    }
  }
]

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages