Split inference includeLogits bool into orthogonal tokens + logits axes

[stikves]

Summary

A code-health cleanup of the inference layer. It replaces the InferenceOptions.includeLogits: Bool flag with two orthogonal request axes, so all four engines — sequential, pipelined, static/ANE, and VLM — describe a forward step the same way:

• tokens: TokenRequest (.none / .sample) — should the engine sample and return a token?
• logits: LogitsRequest (.none / .lastPosition / .allPositions) — which positions' logits should it surface?

A few presets cover the everyday combinations:

Preset	tokens	logits	Use
.prefill	.none	.none	warm the KV cache
.extend (alias .default)	.sample	.none	decode
.eval	.none	.allPositions	perplexity / evaluation
.guided	.none	.lastPosition	constrained decoding
.sampleWithLogits	.sample	.lastPosition	decode and keep the last-row logits

Motivation

The single boolean had to answer two unrelated questions — whether to sample a token, and which logits (if any) to return — and it couldn't answer either well:

• The pipelined engine samples on the GPU and never brings logits back to the CPU, so it can only return a token. A logits flag has no way to express that.
• Perplexity and evaluation need logits at every position, while constrained decoding needs only the last row. A boolean can't tell those apart.
• Logit inspection and scoring want a token and its logits together.

Splitting the request into an action axis and an output axis lets a caller ask for exactly what it needs, and lets an engine turn down a request it can't serve up front rather than fail partway through a forward pass.

Compatibility

The default, (tokens: .sample, logits: .none), is equivalent to the old includeLogits: false, so decode is unchanged. Every call site now uses a preset, and engines derive returnsLogits = (logits != .none).

It does remove the public includeLogits property, which is a breaking change; the replacements are the two enums, the tokens and logits properties, and the preset factories.

Verification

This is a behavior-preserving refactor, and the output is byte-identical to the base commit:

• 39 unit tests pass, including coverage of every preset and the default.
• A golden byte-identical greedy-decode check passes on a 135M fp16 model and a 0.8B hybrid model (both the sequential and pipelined engines), and on a static/accelerator model.
• The pipelined engine's rejection of logit-returning requests is unchanged, checked against the base binary.

Follow-ups

Intentionally left for separate changes:

• Narrow the CPU logits readback for .lastPosition and .none — a decode-throughput win the sequential and VLM paths don't take yet.
• Skip sampling entirely when tokens is .none on the decode path.
• Support .allPositions on the static and pipelined engines.
• The engine-consolidation and warmup work that builds on this contract.

[carinapeng]

grepped for .tokens, .logits, .allPositions and .lastPosition in swift/ and internal/. Outside InferenceEngine.swift, the only readers are the tests. Every engine (sequential, VLM, static-shape, pipelined) reads returnsLogits, which is just logits != .none.

So tokens: .none and .allPositions change nothing at runtime, so these are declared but no engine reads them 🤔

[stikves]

Yes, this will be handled in a follow up.

(The engines do extra work today)

[carinapeng]

Public API breaking when we remove this

[stikves]

let's mark this deprecated

[carinapeng]

.prefill is documented as "no token or logits generation" but an engine still samples even though .guided and .eval set tokens: .none

[stikves]

Updating documentation