Skip to content

icp: reject missing AES parameters before allocating a key schedule - #19408

Open
Zaczero wants to merge 1 commit into
openzfs:masterfrom
Zaczero:fix/aes-null-parameters
Open

Zaczero wants to merge 1 commit into
openzfs:masterfrom
Zaczero:fix/aes-null-parameters

Conversation

@Zaczero

@Zaczero Zaczero commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Motivation and Context

AES mechanism validation accepts a null parameter pointer. Without a context template, context initialization then allocates a key schedule before rejecting the missing CCM or GCM parameters. That early return bypasses cleanup, leaking the schedule on both encrypt and decrypt calls.

Description

Reject missing parameters at the existing validation boundary, before allocating the key schedule. The crypto test driver checks rejection and preservation of output bytes, length and offset with both zero and expected parameter lengths.

How Has This Been Tested?

A Linux x86-64 native allocation observer measured a 512-byte leak per rejected call before the change and no allocation afterward, across all eight combinations of CCM/GCM, encrypt/decrypt and zero/expected parameter lengths.

These valid-key calls retain CRYPTO_MECHANISM_PARAM_INVALID and leave output unchanged. Debug and release builds passed the null-parameter cases and existing GCM/CCM known-answer suites. Both corresponding functional ZTS scripts passed.

Types of Changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Quality assurance (non-breaking change which makes the code more robust against bugs)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist

aes_check_mech_param() accepts a null parameter pointer.  Without a
context template, aes_common_init_ctx() then allocates a key schedule
before rejecting the missing CCM or GCM parameters.  That early return
bypasses key-schedule cleanup, leaking the allocation on both encrypt
and decrypt calls.

Reject a missing parameter at the existing mechanism validation
boundary, before context initialization allocates the key schedule.
Exercise null parameters with zero and expected parameter lengths in
the crypto test driver.

Signed-off-by: Kamil Monicz <kamil@monicz.dev>
@Zaczero
Zaczero force-pushed the fix/aes-null-parameters branch from 53c3314 to 5f974a2 Compare October 15, 2026 12:22
@Zaczero
Zaczero marked this pull request as ready for review October 15, 2026 12:51
@github-actions github-actions Bot added the Status: Code Review Needed Ready for review and testing label Oct 15, 2026
@AttilaFueloep

Copy link
Copy Markdown
Contributor

I'll review in couple of days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Status: Code Review Needed Ready for review and testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants