Skip to content

Update mlir-aie to v1.4.0 and migrate the operator library to its APIs - #145

Open
atassis wants to merge 1 commit into
amd:develfrom
atassis:migrate/mlir-aie-v1.4.0
Open

Update mlir-aie to v1.4.0 and migrate the operator library to its APIs#145
atassis wants to merge 1 commit into
amd:develfrom
atassis:migrate/mlir-aie-v1.4.0

Conversation

@atassis

@atassis atassis commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

This updates mlir-aie from v1.3.5.dev20+g167f34d to v1.4.0 and migrates the
operator library to the APIs that release ships, so the bump builds.

Supersedes #144, which changes requirements.txt alone. I hit this running the
operator library against a v1.4.0-era mlir-aie on Strix, so the changes here are
what it took to get back to green rather than a reading of the changelog.

Why the bump alone does not build

Two breaking changes landed in mlir-aie after the pinned v1.3.5.dev20 (which is
mlir-aie 167f34d, 8 July) and before v1.4.0.

#3387 reworked the IRON Runtime into a callback body.

before after
Runtime() + with rt.sequence(*tys) as (...) Runtime(seq_fn, fn_args)
rt.fill(h, src, ...) / rt.drain(h, dst, ...) h.fill(src, ...) / h.drain(dst, ...)
rt.start(*workers) Program(dev, rt, workers=workers)
rt.task_group() / rt.finish_task_group(tg) TaskGroup() / tg.finish()
rt.set_barrier(b, 1) b.set(1)
rt.inline_ops(fn, args) call fn directly
rt.enable_trace(...) Program.enable_trace(...)
rt.sync_parameters() iron.sync_parameters()
task_group= group=

The per-transfer tile= argument also goes away: placement is now a property of
the handle, so it is bound once on .prod(tile=...) / .cons(tile=...) instead
of being repeated on every fill/drain.

#3364 unified aiecc's output selection under --get-<name>, removing
--aie-generate-xclbin, --aie-generate-npu-insts, --generate-full-elf and
--no-compile-host. iron/common/compilation/base.py still emitted all of them,
so nothing compiled at all. The xclbin path and the full-ELF path both died on
the first argument aiecc did not recognise.

before after
--aie-generate-xclbin --get-xclbin
--aie-generate-npu-insts --get-npu-insts
--generate-full-elf --get-full-elf
--no-compile-host dropped
--no-compile dropped

The last two are deletions rather than renames. Requesting only --get-npu-insts
and not --get-xclbin is precisely what --no-compile used to mean, so output
selection by omission replaces it.

Two places that needed more than a rename

gemm records tensor access patterns from inside the sequence body. The body
now runs at resolve_program() rather than at Runtime() construction, so with a
literal translation generate_taps=True would have returned three empty
sequences, quietly and with no error. The program is now resolved before the taps
are read.

mem_copy guarded rt.start on not bypass. Since workers moved to the
Program constructor, that becomes workers=None if bypass else my_workers;
translating the guard literally would have started workers in the bypass path.

Testing

pytest iron/operators on Strix (npu2), full suite including the extensive
set, against a local build of mlir-aie at the v1.4.0 API.

2810 passed, 5 skipped in 1021.64s

All 21 operator suites green. That includes the configurations that exercise the
parts of this change that needed judgement rather than a rename: mem_copy with
bypass=True (the conditional-workers case), mha at both 4 and 8 pipelines
(the 4-pipeline arm is the one where the second Q/O handle pair is absent) and
its GQA case, and swiglu_decode/swiglu_prefill, which are the only operators
taking the full-ELF path.

Three branches no test reaches, checked by generating their MLIR directly:

  • gemm with generate_taps=True returns 8/8/4 non-empty access patterns. Under
    a literal translation this returns three empty sequences with no error, which
    is what motivated the reorder above.
  • gemm with separate_c_tiles=True, used by iron/applications/llama_3.2_1b
    but by no test, still builds.
  • strided_copy has no test.py at all. Both paths build, and the
    offset-parameter path still emits aiex.sync_scratchpad_parameters_from_host,
    the offset_parameter BD attributes and the aiex.scratchpad_parameter
    declaration.

Still unverified: softmax with vector_size_parameter set, and the
llama_3.2_1b end-to-end chain. Both are reachable only from that application,
whose tests skip without IRON_EXAMPLE_WEIGHTS_DIR, so I could not run them.

black and reuse lint clean.

Not included

The nightly update-mlir-aie.yml run that opened #144 reported failure even
though the PR was created: after pushing the branch it calls workflow_dispatch
on a workflow that has no such trigger, gets HTTP 422, and exits 1. That is why
#144 carries no checks. It is unrelated to this migration, and would fail on any
bump, so it is worth a separate fix.

andrej
andrej approved these changes Jul 31, 2026
View reviewed changes

@andrej andrej left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much appreciated -- thanks for picking up the numerous upstream API changes!

Side note, I have activated the merge queue so I will be able to enqueue multiple PRs at once and have them rebase and test automatically, so hopefully I can merge a swath of your PRs today. Thanks for your patience so far.

@andrej andrej mentioned this pull request Jul 31, 2026
Closed
3 tasks
@atassis @andrej
Update mlir-aie to v1.4.0 and migrate the operator library to its APIs
0bdebc0 
v1.4.0 carries two breaking changes that the pinned v1.3.5.dev20 predates, so
the version bump alone does not build.

mlir-aie #3387 reworked the IRON Runtime into a callback body. Runtime() plus
'with rt.sequence(...)' is gone; the constructor now takes (seq_fn, fn_args),
fill/drain moved onto the ObjectFifo handle, workers moved to Program(workers=),
task_group()/finish_task_group() became TaskGroup()/tg.finish(), set_barrier()
became barrier.set(), inline_ops() became a direct call, enable_trace() moved to
Program and sync_parameters() to module scope. The per-transfer tile= argument is
now bound on .prod()/.cons(), since placement is a property of the handle.

mlir-aie #3364 unified aiecc's output selection under --get-<name>, removing
--aie-generate-xclbin, --aie-generate-npu-insts and --no-compile-host. Asking
only for the insts is what --no-compile used to mean, so that flag goes away
rather than being renamed.

Two spots needed more than a rename. gemm returns tensor access patterns that are
recorded while the sequence body runs, and the body now runs at resolve_program()
rather than at construction, so the program is resolved before the taps are read.
mem_copy guarded rt.start on 'not bypass', which becomes a conditional workers=
argument.

Verified on Strix (npu2): the axpy suite passes 20/20 on device.
@andrej
andrej force-pushed the migrate/mlir-aie-v1.4.0 branch from 4017de7 to 0bdebc0 Compare August 3, 2026 17:30
@andrej

andrej commented Aug 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Rebased. The tracing changes conflicted but it was an easy fix.

As anticipated, @asyms the Stream codegen will need some changes to work with MLIR-AIE 1.4.0. If they're not too involved, I think it would be easiest if you can push those changes to this branch as well (or open a PR with this branch as the base, since you likely won't have push rights to this branch).

There are also a couple tests that fail for me locally:

iron/tests/infrastructure/sequence.py::test_auto_dispatch_selects_platform_default[size_4096] FAILED                                                                                                        [ 76%]
iron/operators/swiglu_decode/test.py::test_swiglu_decode[embedding_dim_2048-hidden_dim_2048] FAILED                                                                                                         [ 68%]
iron/operators/swiglu_decode/test.py::test_swiglu_decode[embedding_dim_1024-hidden_dim_3584] FAILED                                                                                                         [ 68%]
iron/operators/swiglu_prefill/test.py::test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False] FAILED
iron/operators/rms_norm/test.py::test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False] FAILED                                                                      [ 50%]
iron/operators/gemm/test.py::test_gemm[M_2048-K_2048-N_2048-num_aie_columns_8-b_col_maj_True-c_col_maj_True-m_64-k_64-n_64-trace_size_0-partition_N_1] FAILED                                               [ 17%]
iron/operators/gemm/test.py::test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] FAILED   
iron/operators/elementwise_add/test.py::test_elementwise_add[input_length_2048-num_aie_columns_8-tile_size_256] FAILED                                                                                      [ 10%]

Edit: Turns out some of these failures are just because I ran this on a machine with the wrong XRT/driver version.

@github-actions

github-actions Bot commented Aug 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

CI Test Results

b154dba (2026_08_03_17_37_33)

IRON - CI Summary

Examples

iron/applications/llama_3.2_1b
Test Krackan Status Krackan Phoenix Status Phoenix
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40] ❌ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1] ✅ - - -
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40] ❌ - - -

Small

iron/operators/axpy
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0] - - ✅ 323.14
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0] - - ✅ 419.62
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0] - - ✅ 430.86
iron/operators/dequant
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32] - - ✅ 317.24
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32] - - ✅ 319.24
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32] - - ✅ 367.84
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32] - - ✅ 375.92
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32] - - ✅ 477.48
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32] - - ✅ 489.18
iron/operators/elementwise_add
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048] - - ✅ 486.66
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024] - - ✅ 825.30
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512] - - ✅ 739.84
iron/operators/elementwise_mul
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048] - - ✅ 344.38
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024] - - ✅ 434.20
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512] - - ✅ 317.02
iron/operators/gelu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 302.54
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 363.42
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 456.02
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 455.04
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 356.84
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 449.32
iron/operators/gemm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1] - - ✅ 585.60
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1] - - ✅ 630.04
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] - - ✅ 82297.20
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1] - - ✅ 24831.70
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1] - - ✅ 3322.88
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4] - - ✅ 6393.00
iron/operators/gemv
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] - - ✅ 0.07
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] - - ✅ 3.59
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024] - - ✅ 6.12
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512] - - ✅ 9.12
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] - - ✅ 3.61
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024] - - ✅ 6.13
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024] - - ✅ 9.45
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2] - - ✅ 1.88
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2] - - ✅ 0.31
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4] - - ✅ 0.53
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8] - - ✅ 1.47
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128] - - ❌ -
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048] - - ❌ -
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024] - - ❌ -
iron/operators/layer_norm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 417.90
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 454.86
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 416.06
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 454.72
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 380.96
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 523.64
iron/operators/leaky_relu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01] - - ✅ 410.10
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1] - - ✅ 393.66
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25] - - ✅ 436.48
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01] - - ✅ 403.36
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01] - - ✅ 504.30
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01] - - ✅ 501.98
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01] - - ✅ 497.36
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01] - - ✅ 852.68
iron/operators/mem_copy
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048] - - ✅ 410.36
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024] - - ✅ 407.40
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024] - - ✅ 414.64
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512] - - ✅ 931.80
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512] - - ✅ 454.90
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256] - - ✅ 462.24
iron/operators/relu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 346.20
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 337.96
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 345.06
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 394.54
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 337.40
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 519.74
iron/operators/rms_norm
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True] - - ❌ -
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False] - - ❌ -
iron/operators/rope
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0] - - ✅ 447.50
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0] - - ✅ 374.86
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0] - - ✅ 469.18
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0] - - ✅ 647.26
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0] - - ✅ 394.22
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0] - - ✅ 425.64
iron/operators/sigmoid
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 384.02
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 427.26
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 461.26
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 516.06
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 449.62
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 568.56
iron/operators/silu
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 384.00
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 437.16
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 413.68
iron/operators/softmax
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024] - - ✅ 594.64
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048] - - ✅ 677.26
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 516.72
iron/operators/swiglu_decode
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584] - - ✅ 15252.51
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048] - - ✅ 14852.79
iron/operators/swiglu_prefill
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False] - - ✅ 25082.37
iron/operators/tanh
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048] - - ✅ 367.10
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024] - - ✅ 380.02
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024] - - ✅ 318.20
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512] - - ✅ 415.96
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512] - - ✅ 495.18
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256] - - ✅ 430.34
iron/operators/transpose
Test Krackan Status Krackan Phoenix Status Phoenix Latency (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1] - - ✅ 696.48
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2] - - ✅ 2217.38
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1] - - ✅ 362.96
Krackan - Examples

IRON

Tested on 2026_08_03_17_37_33 at commit b154dba.

iron/applications/llama_3.2_1b
TestChecksTTFT (mean)TPS (mean)
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]✅ 5/52.06n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]❌ 0/5n/an/a
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]✅ 5/52.03n/a
test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]❌ 0/5n/an/a

Trends:

IRON Trends

iron/applications/llama_3.2_1b

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
b154dba — 2026-08-03 17:32:582.06 (-0.58%)2.06 (+0.11%)2.06 (+0.19%)2.04 (+0.64%)0.01 (-44.57%)
cdc48e9 — 2026-08-03 16:32:132.08 (n/a)2.05 (n/a)2.06 (n/a)2.03 (n/a)0.02 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_1024_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
b154dba — 2026-08-03 17:32:58n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:137.79 (n/a)7.74 (n/a)7.72 (n/a)7.70 (n/a)0.03 (n/a)2.19 (n/a)2.08 (n/a)2.06 (n/a)2.04 (n/a)0.06 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_1]

Commit/Date TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
b154dba — 2026-08-03 17:32:582.04 (-0.58%)2.03 (+0.25%)2.03 (+0.49%)2.00 (+0.25%)0.02 (-19.02%)
cdc48e9 — 2026-08-03 16:32:132.06 (n/a)2.02 (n/a)2.02 (n/a)2.00 (n/a)0.02 (n/a)

test_llama_3_2_1b[llama_3.2_1b_prompt_13_tokens_40]

Commit/Date TPS (max)TPS (mean)TPS (median)TPS (min)TPS (stddev)TTFT (max)TTFT (mean)TTFT (median)TTFT (min)TTFT (stddev)
b154dba — 2026-08-03 17:32:58n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:137.60 (n/a)7.54 (n/a)7.53 (n/a)7.51 (n/a)0.04 (n/a)2.04 (n/a)2.02 (n/a)2.02 (n/a)2.01 (n/a)0.01 (n/a)
Phoenix - Small

IRON

Tested on 2026_08_03_17_39_17 at commit b154dba.

iron/operators/axpy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]✅ 5/5323.140.04n/a
test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]✅ 5/5419.620.03n/a
test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]✅ 5/5430.860.03n/a
iron/operators/dequant
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]✅ 5/5317.240.02n/a
test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]✅ 5/5319.240.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]✅ 5/5367.840.02n/a
test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]✅ 5/5375.920.02n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]✅ 5/5477.480.01n/a
test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]✅ 5/5489.180.01n/a
iron/operators/elementwise_add
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5486.660.03n/a
test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5825.300.02n/a
test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5739.840.03n/a
iron/operators/elementwise_mul
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]✅ 5/5344.380.04n/a
test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]✅ 5/5434.200.03n/a
test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]✅ 5/5317.020.04n/a
iron/operators/gelu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5302.540.03n/a
test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5363.420.03n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5456.020.02n/a
test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5455.040.02n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5356.840.03n/a
test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5449.320.02n/a
iron/operators/gemm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5585.600.3916.62
test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]✅ 5/5630.040.3816.29
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/582297.200.31208.76
test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]✅ 5/524831.701.01692.07
test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]✅ 5/53322.882.79732.91
test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]✅ 5/56393.000.2010.67
iron/operators/gemv
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]✅ 5/5n/a0.070.07
test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]✅ 5/5n/a3.593.59
test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]✅ 5/5n/a6.126.11
test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]✅ 5/5n/a9.129.11
test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a3.613.60
test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a6.136.12
test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]✅ 5/5n/a9.459.45
test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]✅ 5/5n/a1.881.88
test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]✅ 5/5n/a0.310.31
test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]✅ 5/5n/a0.530.52
test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]✅ 5/5n/a1.471.44
test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]❌ 0/5n/an/an/a
test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]❌ 0/5n/an/an/a
test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]❌ 0/5n/an/an/a
iron/operators/layer_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5417.900.02n/a
test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5454.860.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5416.060.02n/a
test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5454.720.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5380.960.02n/a
test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5523.640.02n/a
iron/operators/leaky_relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]✅ 5/5410.100.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]✅ 5/5393.660.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]✅ 5/5436.480.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]✅ 5/5403.360.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]✅ 5/5504.300.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]✅ 5/5501.980.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]✅ 5/5497.360.02n/a
test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]✅ 5/5852.680.02n/a
iron/operators/mem_copy
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]✅ 5/5410.360.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]✅ 5/5407.400.02n/a
test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]✅ 5/5414.640.02n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]✅ 5/5931.800.01n/a
test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]✅ 5/5454.900.02n/a
test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]✅ 5/5462.240.02n/a
iron/operators/relu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5346.200.03n/a
test_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5337.960.03n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5345.060.03n/a
test_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5394.540.02n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5337.400.03n/a
test_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5519.740.02n/a
iron/operators/rms_norm
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]❌ 0/5n/an/an/a
test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]❌ 0/5n/an/an/a
iron/operators/rope
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]✅ 5/5447.500.25n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]✅ 5/5374.860.30n/a
test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]✅ 5/5469.180.22n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]✅ 5/5647.260.19n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]✅ 5/5394.220.21n/a
test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]✅ 5/5425.640.19n/a
iron/operators/sigmoid
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5384.020.02n/a
test_sigmoid[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5427.260.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5461.260.02n/a
test_sigmoid[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5516.060.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5449.620.02n/a
test_sigmoid[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5568.560.02n/a
iron/operators/silu
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_silu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5384.000.02n/a
test_silu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5437.160.02n/a
test_silu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5413.680.02n/a
iron/operators/softmax
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]✅ 5/5594.640.26n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]✅ 5/5677.260.21n/a
test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5516.720.26n/a
iron/operators/swiglu_decode
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]✅ 5/515252.510.00n/a
test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]✅ 5/514852.790.00n/a
iron/operators/swiglu_prefill
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]✅ 5/525082.370.09n/a
iron/operators/tanh
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_tanh[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]✅ 5/5367.100.02n/a
test_tanh[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]✅ 5/5380.020.02n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]✅ 5/5318.200.03n/a
test_tanh[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]✅ 5/5415.960.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]✅ 5/5495.180.02n/a
test_tanh[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]✅ 5/5430.340.02n/a
iron/operators/transpose
TestChecksLatency (mean)Bandwidth (mean)Throughput (mean)
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]✅ 5/5696.480.83n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]✅ 5/52217.380.98n/a
test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]✅ 5/5362.961.47n/a

Trends:

IRON Trends

iron/operators/axpy

test_axpy[input_length_2048-num_aie_columns_1-tile_size_2048-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.05 (+17.69%)0.04 (+29.54%)0.05 (+19.66%)0.02 (-5.09%)0.01 (+15.91%)639.90 (+5.37%)323.14 (-20.66%)247.10 (-16.41%)230.90 (-15.02%)177.24 (+6.93%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.03 (n/a)0.04 (n/a)0.02 (n/a)0.01 (n/a)607.30 (n/a)407.28 (n/a)295.60 (n/a)271.70 (n/a)165.76 (n/a)

test_axpy[input_length_2048-num_aie_columns_2-tile_size_1024-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.06 (+5.49%)0.03 (-16.51%)0.03 (-44.31%)0.02 (-3.10%)0.02 (+32.71%)599.70 (+3.20%)419.62 (+27.58%)473.30 (+79.55%)223.40 (-5.22%)175.96 (+22.01%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.04 (n/a)0.05 (n/a)0.02 (n/a)0.01 (n/a)581.10 (n/a)328.90 (n/a)263.60 (n/a)235.70 (n/a)144.22 (n/a)

test_axpy[input_length_2048-num_aie_columns_4-tile_size_512-scalar_factor_3.0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.04 (-20.50%)0.03 (-2.66%)0.03 (+2.46%)0.02 (+7.71%)0.01 (-32.89%)600.30 (-7.16%)430.86 (-2.67%)455.60 (-2.40%)301.60 (+25.77%)121.88 (-22.76%)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)646.60 (n/a)442.68 (n/a)466.80 (n/a)239.80 (n/a)157.79 (n/a)
iron/operators/dequant

test_dequant[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-11.66%)0.02 (+14.41%)0.02 (-4.74%)0.01 (+67.50%)0.00 (-39.63%)483.70 (-40.29%)317.24 (-26.82%)310.30 (+4.97%)230.00 (+13.19%)100.25 (-60.04%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)810.10 (n/a)433.48 (n/a)295.60 (n/a)203.20 (n/a)250.85 (n/a)

test_dequant[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (+1.19%)0.02 (+7.94%)0.02 (+21.05%)0.01 (-0.64%)0.01 (+20.57%)449.90 (+0.65%)319.24 (-4.60%)249.20 (-17.37%)223.70 (-1.15%)117.17 (+17.34%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)447.00 (n/a)334.62 (n/a)301.60 (n/a)226.30 (n/a)99.85 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-15.46%)0.02 (-10.72%)0.02 (-16.80%)0.01 (-7.05%)0.00 (-22.03%)533.70 (+7.60%)367.84 (+9.90%)349.40 (+20.19%)265.00 (+18.25%)107.40 (-2.80%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)496.00 (n/a)334.70 (n/a)290.70 (n/a)224.10 (n/a)110.49 (n/a)

test_dequant[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (+27.85%)0.02 (+5.03%)0.02 (+0.97%)0.01 (+2.32%)0.01 (+47.44%)574.60 (-2.28%)375.92 (-0.05%)300.10 (-0.96%)223.80 (-21.78%)148.91 (+17.23%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)588.00 (n/a)376.12 (n/a)303.00 (n/a)286.10 (n/a)127.02 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-2.52%)0.01 (-21.64%)0.01 (-36.50%)0.01 (-47.46%)0.01 (+55.59%)888.60 (+90.32%)477.48 (+52.62%)484.40 (+57.48%)227.50 (+2.62%)265.59 (+180.87%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)466.90 (n/a)312.86 (n/a)307.60 (n/a)221.70 (n/a)94.56 (n/a)

test_dequant[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-group_size_32]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-12.41%)0.01 (-10.79%)0.01 (-12.56%)0.01 (-21.42%)0.01 (+0.83%)750.50 (+27.27%)489.18 (+16.68%)517.80 (+14.36%)279.20 (+14.19%)198.30 (+40.83%)
cdc48e9 — 2026-08-03 16:32:080.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)589.70 (n/a)419.24 (n/a)452.80 (n/a)244.50 (n/a)140.81 (n/a)
iron/operators/elementwise_add

test_elementwise_add[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.05 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)661.30 (n/a)486.66 (n/a)614.40 (n/a)264.70 (n/a)200.16 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)2048.10 (n/a)825.30 (n/a)520.50 (n/a)441.80 (n/a)686.14 (n/a)

test_elementwise_add[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.05 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)0.02 (n/a)1848.40 (n/a)739.84 (n/a)598.80 (n/a)249.20 (n/a)639.65 (n/a)
iron/operators/elementwise_mul

test_elementwise_mul[input_length_2048-num_aie_columns_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.05 (n/a)0.04 (n/a)0.04 (n/a)0.03 (n/a)0.01 (n/a)457.30 (n/a)344.38 (n/a)328.60 (n/a)243.60 (n/a)79.96 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.05 (n/a)0.03 (n/a)0.04 (n/a)0.01 (n/a)0.01 (n/a)920.00 (n/a)434.20 (n/a)346.50 (n/a)270.50 (n/a)273.93 (n/a)

test_elementwise_mul[input_length_2048-num_aie_columns_4-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.05 (n/a)0.04 (n/a)0.05 (n/a)0.03 (n/a)0.01 (n/a)433.10 (n/a)317.02 (n/a)258.50 (n/a)248.90 (n/a)87.01 (n/a)
iron/operators/gelu

test_gelu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)432.30 (n/a)302.54 (n/a)296.80 (n/a)190.40 (n/a)92.57 (n/a)

test_gelu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)615.40 (n/a)363.42 (n/a)243.80 (n/a)228.90 (n/a)177.35 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)649.70 (n/a)456.02 (n/a)504.50 (n/a)194.40 (n/a)177.22 (n/a)

test_gelu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)544.90 (n/a)455.04 (n/a)528.60 (n/a)306.30 (n/a)110.83 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)615.30 (n/a)356.84 (n/a)277.80 (n/a)226.40 (n/a)160.03 (n/a)

test_gelu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.00 (n/a)582.50 (n/a)449.32 (n/a)432.90 (n/a)304.60 (n/a)108.20 (n/a)
iron/operators/gemm

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_False-c_col_maj_False-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.49 (-8.16%)0.39 (-5.37%)0.36 (-12.00%)0.31 (+10.36%)0.08 (-18.29%)714.10 (-9.39%)585.60 (+3.97%)615.90 (+13.63%)454.80 (+8.88%)112.59 (-21.53%)20.75 (-8.16%)16.62 (-5.37%)15.32 (-12.00%)13.22 (+10.36%)3.33 (-18.29%)
cdc48e9 — 2026-08-03 16:32:080.53 (n/a)0.41 (n/a)0.41 (n/a)0.28 (n/a)0.10 (n/a)788.10 (n/a)563.26 (n/a)542.00 (n/a)417.70 (n/a)143.47 (n/a)22.60 (n/a)17.57 (n/a)17.41 (n/a)11.97 (n/a)4.07 (n/a)

test_gemm[M_192-K_384-N_64-num_aie_columns_4-b_col_maj_True-c_col_maj_True-m_48-k_96-n_16-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.63 (+32.12%)0.38 (-0.88%)0.32 (-8.04%)0.27 (-3.56%)0.14 (+68.44%)813.00 (+3.69%)630.04 (+5.41%)680.80 (+8.74%)349.80 (-24.30%)172.31 (+28.57%)26.98 (+32.12%)16.29 (-0.88%)13.86 (-8.04%)11.61 (-3.56%)6.13 (+68.44%)
cdc48e9 — 2026-08-03 16:32:080.48 (n/a)0.39 (n/a)0.35 (n/a)0.28 (n/a)0.09 (n/a)784.10 (n/a)597.72 (n/a)626.10 (n/a)462.10 (n/a)134.02 (n/a)20.42 (n/a)16.43 (n/a)15.07 (n/a)12.04 (n/a)3.64 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_1-b_col_maj_False-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.31 (+0.45%)0.31 (+0.21%)0.31 (-0.19%)0.30 (+1.01%)0.00 (-35.49%)82515.90 (-1.00%)82297.20 (-0.21%)82440.80 (+0.19%)81745.00 (-0.45%)318.29 (-36.53%)210.16 (+0.45%)208.76 (+0.21%)208.39 (-0.19%)208.20 (+1.01%)0.81 (-35.49%)
cdc48e9 — 2026-08-03 16:32:080.31 (n/a)0.31 (n/a)0.31 (n/a)0.30 (n/a)0.00 (n/a)83350.90 (n/a)82467.46 (n/a)82282.50 (n/a)82115.60 (n/a)501.45 (n/a)209.22 (n/a)208.33 (n/a)208.79 (n/a)206.12 (n/a)1.26 (n/a)

test_gemm[M_2048-K_2048-N_2048-num_aie_columns_2-b_col_maj_True-c_col_maj_False-m_64-k_64-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:591.03 (+0.95%)1.01 (+1.34%)1.02 (+1.40%)0.98 (+2.48%)0.02 (-21.27%)25682.90 (-2.42%)24831.70 (-1.34%)24619.70 (-1.38%)24465.50 (-0.94%)501.30 (-24.07%)702.21 (+0.95%)692.07 (+1.34%)697.81 (+1.40%)668.92 (+2.48%)13.69 (-21.27%)
cdc48e9 — 2026-08-03 16:32:081.02 (n/a)1.00 (n/a)1.01 (n/a)0.96 (n/a)0.03 (n/a)26319.80 (n/a)25168.86 (n/a)24964.50 (n/a)24697.80 (n/a)660.22 (n/a)695.60 (n/a)682.95 (n/a)688.17 (n/a)652.74 (n/a)17.38 (n/a)

test_gemm[M_384-K_1536-N_1792-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_32-k_48-n_64-trace_size_0-partition_N_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:593.77 (-1.55%)2.79 (+17.72%)3.32 (+62.78%)1.44 (-4.99%)1.04 (+14.10%)5578.50 (+5.25%)3322.88 (-11.55%)2430.90 (-38.57%)2139.40 (+1.58%)1504.87 (+22.78%)988.08 (-1.55%)732.91 (+17.72%)869.61 (+62.78%)378.94 (-4.99%)271.83 (+14.10%)
cdc48e9 — 2026-08-03 16:32:083.83 (n/a)2.37 (n/a)2.04 (n/a)1.52 (n/a)0.91 (n/a)5300.10 (n/a)3756.70 (n/a)3956.90 (n/a)2106.20 (n/a)1225.67 (n/a)1003.68 (n/a)622.60 (n/a)534.24 (n/a)398.85 (n/a)238.25 (n/a)

test_gemm[M_64-K_512-N_256-num_aie_columns_4-b_col_maj_True-c_col_maj_False-m_16-k_64-n_64-trace_size_0-partition_N_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.24 (-18.84%)0.20 (-6.85%)0.20 (+0.57%)0.16 (-8.92%)0.03 (-41.42%)7646.20 (+9.79%)6393.00 (+5.50%)6307.10 (-0.56%)5151.40 (+23.21%)892.23 (-18.57%)13.03 (-18.84%)10.67 (-6.85%)10.64 (+0.57%)8.78 (-8.92%)1.53 (-41.42%)
cdc48e9 — 2026-08-03 16:32:080.30 (n/a)0.21 (n/a)0.20 (n/a)0.18 (n/a)0.05 (n/a)6964.20 (n/a)6059.96 (n/a)6342.80 (n/a)4180.90 (n/a)1095.64 (n/a)16.05 (n/a)11.45 (n/a)10.58 (n/a)9.64 (n/a)2.62 (n/a)
iron/operators/gemv

test_gemv[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.11 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.02 (n/a)0.11 (n/a)0.07 (n/a)0.07 (n/a)0.05 (n/a)0.02 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:593.80 (n/a)3.59 (n/a)3.62 (n/a)3.40 (n/a)0.15 (n/a)3.80 (n/a)3.59 (n/a)3.61 (n/a)3.40 (n/a)0.15 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_2-tile_size_input_1-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:597.44 (+31.32%)6.12 (+17.24%)6.11 (+15.77%)4.37 (-8.11%)1.15 (+159.98%)7.44 (+31.32%)6.11 (+17.24%)6.11 (+15.77%)4.36 (-8.11%)1.15 (+159.98%)
cdc48e9 — 2026-08-03 16:32:085.67 (n/a)5.22 (n/a)5.28 (n/a)4.75 (n/a)0.44 (n/a)5.66 (n/a)5.21 (n/a)5.28 (n/a)4.75 (n/a)0.44 (n/a)

test_gemv[M_2048-K_8192-num_aie_columns_4-tile_size_input_1-tile_size_output_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:5913.50 (+4.52%)9.12 (-7.95%)8.23 (-17.33%)7.52 (+17.28%)2.49 (-0.27%)13.50 (+4.52%)9.11 (-7.95%)8.22 (-17.33%)7.52 (+17.27%)2.49 (-0.27%)
cdc48e9 — 2026-08-03 16:32:0812.92 (n/a)9.90 (n/a)9.95 (n/a)6.41 (n/a)2.49 (n/a)12.91 (n/a)9.90 (n/a)9.95 (n/a)6.41 (n/a)2.49 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:593.71 (n/a)3.61 (n/a)3.61 (n/a)3.51 (n/a)0.08 (n/a)3.70 (n/a)3.60 (n/a)3.61 (n/a)3.51 (n/a)0.08 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_2-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:597.42 (-1.80%)6.13 (-11.32%)5.73 (-16.58%)5.59 (-7.38%)0.77 (+18.78%)7.42 (-1.80%)6.12 (-11.32%)5.73 (-16.58%)5.59 (-7.38%)0.77 (+18.78%)
cdc48e9 — 2026-08-03 16:32:087.56 (n/a)6.91 (n/a)6.87 (n/a)6.04 (n/a)0.65 (n/a)7.55 (n/a)6.90 (n/a)6.87 (n/a)6.03 (n/a)0.65 (n/a)

test_gemv[M_8192-K_2048-num_aie_columns_4-tile_size_input_4-tile_size_output_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:5913.22 (-4.88%)9.45 (-22.99%)8.67 (-30.78%)7.40 (-22.17%)2.28 (+25.41%)13.21 (-4.88%)9.45 (-22.99%)8.66 (-30.78%)7.40 (-22.17%)2.28 (+25.41%)
cdc48e9 — 2026-08-03 16:32:0813.89 (n/a)12.27 (n/a)12.52 (n/a)9.51 (n/a)1.82 (n/a)13.89 (n/a)12.27 (n/a)12.52 (n/a)9.50 (n/a)1.82 (n/a)

test_gemv_batched[M_1024-K_1024-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:592.92 (+5.77%)1.88 (+27.64%)1.67 (+33.43%)0.94 (-3.10%)0.95 (+30.25%)2.92 (+5.77%)1.88 (+27.64%)1.67 (+33.43%)0.94 (-3.10%)0.95 (+30.25%)
cdc48e9 — 2026-08-03 16:32:082.76 (n/a)1.47 (n/a)1.25 (n/a)0.97 (n/a)0.73 (n/a)2.76 (n/a)1.47 (n/a)1.25 (n/a)0.97 (n/a)0.73 (n/a)

test_gemv_batched[M_1026-K_64-num_aie_columns_1-tile_size_input_1-tile_size_output_2-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.55 (+20.99%)0.31 (+36.37%)0.34 (+84.21%)0.07 (-0.47%)0.21 (+23.30%)0.54 (+20.99%)0.31 (+36.37%)0.33 (+84.21%)0.07 (-0.47%)0.21 (+23.30%)
cdc48e9 — 2026-08-03 16:32:080.45 (n/a)0.23 (n/a)0.18 (n/a)0.07 (n/a)0.17 (n/a)0.45 (n/a)0.22 (n/a)0.18 (n/a)0.07 (n/a)0.17 (n/a)

test_gemv_batched[M_256-K_128-num_aie_columns_1-tile_size_input_1-tile_size_output_256-num_batches_4]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:590.76 (+73.71%)0.53 (+73.81%)0.62 (+117.97%)0.28 (+251.21%)0.21 (+41.42%)0.75 (+73.71%)0.52 (+73.81%)0.62 (+117.97%)0.28 (+251.21%)0.20 (+41.42%)
cdc48e9 — 2026-08-03 16:32:080.44 (n/a)0.30 (n/a)0.29 (n/a)0.08 (n/a)0.15 (n/a)0.43 (n/a)0.30 (n/a)0.28 (n/a)0.08 (n/a)0.14 (n/a)

test_gemv_batched[M_64-K_1536-num_aie_columns_1-tile_size_input_1-tile_size_output_64-num_batches_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Throughput (max)Throughput (mean)Throughput (median)Throughput (min)Throughput (stddev)
b154dba — 2026-08-03 17:34:592.44 (-19.40%)1.47 (-23.32%)1.53 (-16.86%)0.45 (-56.30%)0.71 (-1.15%)2.41 (-19.40%)1.44 (-23.32%)1.50 (-16.86%)0.44 (-56.30%)0.70 (-1.15%)
cdc48e9 — 2026-08-03 16:32:083.03 (n/a)1.91 (n/a)1.84 (n/a)1.03 (n/a)0.72 (n/a)2.98 (n/a)1.88 (n/a)1.81 (n/a)1.01 (n/a)0.71 (n/a)

test_gemv_gelu[M_128-K_128-num_aie_columns_1-tile_size_input_32-tile_size_output_128]

No metrics available.

test_gemv_gelu[M_2048-K_8192-num_aie_columns_1-tile_size_input_1-tile_size_output_2048]

No metrics available.

test_gemv_gelu[M_8192-K_2048-num_aie_columns_1-tile_size_input_4-tile_size_output_1024]

No metrics available.

iron/operators/layer_norm

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)527.80 (n/a)417.90 (n/a)478.80 (n/a)284.40 (n/a)117.01 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)638.50 (n/a)454.86 (n/a)472.90 (n/a)276.30 (n/a)136.14 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)573.70 (n/a)416.06 (n/a)438.20 (n/a)289.90 (n/a)124.42 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)653.70 (n/a)454.72 (n/a)530.50 (n/a)234.70 (n/a)188.84 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (n/a)0.02 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)482.10 (n/a)380.96 (n/a)410.10 (n/a)266.30 (n/a)102.65 (n/a)

test_layer_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.00 (n/a)685.90 (n/a)523.64 (n/a)565.00 (n/a)338.80 (n/a)132.48 (n/a)
iron/operators/leaky_relu

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (+8.43%)0.02 (+17.11%)0.02 (+39.73%)0.01 (+6.30%)0.01 (-1.34%)595.50 (-5.92%)410.10 (-16.12%)384.30 (-28.44%)276.30 (-7.78%)131.38 (-16.92%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)633.00 (n/a)488.92 (n/a)537.00 (n/a)299.60 (n/a)158.15 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-19.21%)0.02 (-22.66%)0.02 (-28.87%)0.01 (-12.44%)0.01 (-19.92%)624.60 (+14.19%)393.66 (+27.23%)358.60 (+40.57%)266.20 (+23.76%)145.04 (+6.58%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)547.00 (n/a)309.42 (n/a)255.10 (n/a)215.10 (n/a)136.08 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-alpha_0.25]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-21.80%)0.02 (-31.14%)0.02 (-42.08%)0.02 (-19.50%)0.00 (-36.61%)497.40 (+24.23%)436.48 (+42.38%)452.80 (+72.69%)304.30 (+27.91%)78.42 (-2.67%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)400.40 (n/a)306.56 (n/a)262.20 (n/a)237.90 (n/a)80.57 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.04 (-2.25%)0.02 (-12.12%)0.02 (-30.28%)0.01 (-16.03%)0.01 (+4.84%)571.40 (+19.09%)403.36 (+16.02%)410.40 (+43.45%)231.20 (+2.30%)138.13 (+19.54%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)479.80 (n/a)347.66 (n/a)286.10 (n/a)226.00 (n/a)115.56 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-35.96%)0.02 (-36.67%)0.02 (-40.35%)0.01 (-34.24%)0.00 (-51.40%)698.90 (+52.07%)504.30 (+51.35%)504.10 (+67.64%)348.70 (+56.16%)129.22 (+12.95%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)459.60 (n/a)333.20 (n/a)300.70 (n/a)223.30 (n/a)114.40 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (+2.97%)0.02 (-8.85%)0.02 (+3.92%)0.01 (+66.64%)0.01 (-22.91%)600.90 (-39.99%)501.98 (-5.40%)541.80 (-3.78%)249.60 (-2.88%)143.37 (-52.72%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)1001.30 (n/a)530.62 (n/a)563.10 (n/a)257.00 (n/a)303.27 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (+17.85%)0.02 (-11.27%)0.02 (-24.88%)0.01 (-19.57%)0.01 (+31.01%)755.70 (+24.33%)497.36 (+18.98%)511.00 (+33.11%)243.00 (-15.15%)187.80 (+34.56%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)607.80 (n/a)418.02 (n/a)383.90 (n/a)286.40 (n/a)139.56 (n/a)

test_leaky_relu[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-alpha_0.01]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-6.21%)0.02 (-16.46%)0.01 (-12.46%)0.00 (-72.08%)0.01 (+30.87%)2419.10 (+258.17%)852.68 (+84.44%)552.90 (+14.24%)277.40 (+6.61%)886.20 (+473.64%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)675.40 (n/a)462.30 (n/a)484.00 (n/a)260.20 (n/a)154.49 (n/a)
iron/operators/mem_copy

test_mem_copy[input_length_2048-num_cores_1-num_channels_1-bypass_False-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-32.31%)0.02 (-14.31%)0.02 (+7.83%)0.02 (+276.42%)0.01 (-64.76%)525.20 (-73.43%)410.36 (-36.95%)392.70 (-7.25%)274.90 (+47.72%)99.05 (-86.80%)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.02 (n/a)0.02 (n/a)0.00 (n/a)0.02 (n/a)1977.00 (n/a)650.86 (n/a)423.40 (n/a)186.10 (n/a)750.31 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_1-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-9.79%)0.02 (-5.30%)0.02 (+6.61%)0.01 (+4.13%)0.01 (-37.62%)578.20 (-3.97%)407.40 (-3.40%)390.10 (-6.20%)267.40 (+10.86%)116.88 (-33.73%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)602.10 (n/a)421.72 (n/a)415.90 (n/a)241.20 (n/a)176.36 (n/a)

test_mem_copy[input_length_2048-num_cores_2-num_channels_2-bypass_False-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-13.87%)0.02 (+5.76%)0.02 (+18.72%)0.01 (-16.07%)0.01 (-10.86%)654.60 (+19.15%)414.64 (-4.39%)366.60 (-15.76%)292.30 (+16.08%)147.09 (+30.78%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)549.40 (n/a)433.68 (n/a)435.20 (n/a)251.80 (n/a)112.47 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_1-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-46.55%)0.01 (-44.01%)0.02 (-13.47%)0.00 (-77.18%)0.01 (-37.78%)2428.50 (+338.20%)931.80 (+130.04%)538.60 (+15.58%)441.90 (+87.09%)847.05 (+451.93%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)554.20 (n/a)405.06 (n/a)466.00 (n/a)236.20 (n/a)153.47 (n/a)

test_mem_copy[input_length_2048-num_cores_4-num_channels_2-bypass_False-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.03 (-2.96%)0.02 (-16.74%)0.02 (-27.00%)0.01 (+1.25%)0.01 (-19.06%)586.90 (-1.23%)454.90 (+17.03%)444.70 (+37.00%)297.40 (+3.05%)107.52 (-17.98%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)594.20 (n/a)388.70 (n/a)324.60 (n/a)288.60 (n/a)131.10 (n/a)

test_mem_copy[input_length_2048-num_cores_8-num_channels_2-bypass_False-tile_size_256]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.02 (-28.95%)0.02 (-6.28%)0.02 (+26.36%)0.02 (+38.91%)0.00 (-76.40%)518.90 (-28.01%)462.24 (-6.98%)459.50 (-20.86%)400.80 (+40.73%)50.63 (-74.14%)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)0.01 (n/a)720.80 (n/a)496.92 (n/a)580.60 (n/a)284.80 (n/a)195.77 (n/a)
iron/operators/rms_norm

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)583.40 (n/a)404.30 (n/a)392.90 (n/a)261.60 (n/a)144.43 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_1-tile_size_2048-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.05 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)693.60 (n/a)501.36 (n/a)499.80 (n/a)269.50 (n/a)152.43 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)550.30 (n/a)422.88 (n/a)449.30 (n/a)272.10 (n/a)129.54 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_1-num_channels_2-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)572.50 (n/a)415.56 (n/a)434.90 (n/a)289.30 (n/a)108.25 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)628.90 (n/a)377.14 (n/a)290.60 (n/a)219.10 (n/a)167.00 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_1-tile_size_1024-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)2018.00 (n/a)810.12 (n/a)606.90 (n/a)261.90 (n/a)690.59 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.02 (n/a)0.01 (n/a)536.20 (n/a)323.00 (n/a)274.20 (n/a)215.30 (n/a)128.08 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_2-num_channels_2-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.03 (n/a)0.01 (n/a)0.01 (n/a)626.10 (n/a)395.70 (n/a)317.00 (n/a)205.10 (n/a)181.28 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)614.70 (n/a)463.90 (n/a)524.80 (n/a)226.50 (n/a)158.65 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_1-tile_size_512-weighted_True]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.04 (n/a)0.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)563.10 (n/a)390.46 (n/a)405.80 (n/a)246.10 (n/a)133.62 (n/a)

test_rms_norm[input_length_2048-num_aie_columns_4-num_channels_2-tile_size_256-weighted_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:59n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)n/a (n/a)
cdc48e9 — 2026-08-03 16:32:080.03 (n/a)0.02 (n/a)0.02 (n/a)0.01 (n/a)0.01 (n/a)886.20 (n/a)540.54 (n/a)517.80 (n/a)235.40 (n/a)236.94 (n/a)
iron/operators/rope

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.39 (+7.71%)0.25 (-7.17%)0.22 (-34.60%)0.13 (-15.52%)0.10 (-3.19%)745.20 (+18.36%)447.50 (+6.94%)441.80 (+52.92%)254.70 (-7.18%)192.65 (+3.17%)
cdc48e9 — 2026-08-03 16:32:080.36 (n/a)0.27 (n/a)0.34 (n/a)0.16 (n/a)0.10 (n/a)629.60 (n/a)418.44 (n/a)288.90 (n/a)274.40 (n/a)186.73 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.49 (+3.43%)0.30 (-8.46%)0.23 (-34.50%)0.19 (-3.35%)0.13 (+5.80%)522.10 (+3.47%)374.86 (+9.85%)419.80 (+52.65%)200.10 (-3.33%)135.52 (-0.31%)
cdc48e9 — 2026-08-03 16:32:080.47 (n/a)0.33 (n/a)0.36 (n/a)0.19 (n/a)0.12 (n/a)504.60 (n/a)341.26 (n/a)275.00 (n/a)207.00 (n/a)135.94 (n/a)

test_rope[rows_32-cols_512-angle_rows_32-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.34 (-4.87%)0.22 (-7.80%)0.21 (+9.49%)0.16 (-5.22%)0.07 (-24.05%)633.30 (+5.51%)469.18 (+4.20%)467.90 (-8.67%)290.20 (+5.14%)124.26 (-17.74%)
cdc48e9 — 2026-08-03 16:32:080.36 (n/a)0.24 (n/a)0.19 (n/a)0.16 (n/a)0.09 (n/a)600.20 (n/a)450.26 (n/a)512.30 (n/a)276.00 (n/a)151.05 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_1-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.27 (-7.13%)0.19 (+7.21%)0.22 (+48.18%)0.04 (-64.02%)0.09 (+8.42%)1952.30 (+177.95%)647.26 (+34.03%)333.10 (-32.52%)275.60 (+7.66%)730.23 (+257.32%)
cdc48e9 — 2026-08-03 16:32:080.29 (n/a)0.18 (n/a)0.15 (n/a)0.10 (n/a)0.08 (n/a)702.40 (n/a)482.92 (n/a)493.60 (n/a)256.00 (n/a)204.36 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_2-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.30 (+4.30%)0.21 (+8.08%)0.21 (+42.69%)0.13 (+1.43%)0.07 (-3.16%)562.70 (-1.40%)394.22 (-8.39%)357.70 (-29.92%)246.20 (-4.13%)136.68 (-5.29%)
cdc48e9 — 2026-08-03 16:32:080.29 (n/a)0.19 (n/a)0.14 (n/a)0.13 (n/a)0.07 (n/a)570.70 (n/a)430.32 (n/a)510.40 (n/a)256.80 (n/a)144.31 (n/a)

test_rope[rows_32-cols_512-angle_rows_8-aie_columns_4-method_type_0]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.30 (+0.14%)0.19 (-3.57%)0.15 (-14.75%)0.13 (-15.12%)0.08 (+30.39%)558.00 (+17.80%)425.64 (+9.90%)498.20 (+17.31%)243.10 (-0.16%)144.18 (+63.04%)
cdc48e9 — 2026-08-03 16:32:080.30 (n/a)0.20 (n/a)0.17 (n/a)0.16 (n/a)0.06 (n/a)473.70 (n/a)387.28 (n/a)424.70 (n/a)243.50 (n/a)88.43 (n/a)
iron/operators/softmax

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_1024]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.50 (-3.88%)0.26 (-29.67%)0.23 (-38.66%)0.13 (-45.15%)0.14 (+10.59%)990.30 (+82.31%)594.64 (+54.36%)562.50 (+63.04%)264.60 (+4.05%)258.65 (+94.91%)
cdc48e9 — 2026-08-03 16:32:080.52 (n/a)0.37 (n/a)0.38 (n/a)0.24 (n/a)0.12 (n/a)543.20 (n/a)385.22 (n/a)345.00 (n/a)254.30 (n/a)132.70 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.26 (+1.79%)0.21 (-11.42%)0.23 (-4.22%)0.12 (-42.05%)0.06 (+193.46%)1080.70 (+72.58%)677.26 (+21.16%)558.90 (+4.41%)506.30 (-1.75%)236.89 (+407.82%)
cdc48e9 — 2026-08-03 16:32:080.25 (n/a)0.24 (n/a)0.24 (n/a)0.21 (n/a)0.02 (n/a)626.20 (n/a)559.00 (n/a)535.30 (n/a)515.30 (n/a)46.65 (n/a)

test_softmax[input_length_32768-num_aie_columns_2-num_channels_2-tile_size_512]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.33 (-35.67%)0.26 (+28.53%)0.23 (+29.13%)0.22 (+232.98%)0.05 (-71.46%)603.60 (-69.97%)516.72 (-53.96%)575.50 (-22.55%)394.20 (+55.44%)96.16 (-87.85%)
cdc48e9 — 2026-08-03 16:32:080.52 (n/a)0.20 (n/a)0.18 (n/a)0.07 (n/a)0.18 (n/a)2009.90 (n/a)1122.34 (n/a)743.10 (n/a)253.60 (n/a)791.49 (n/a)
iron/operators/swiglu_decode

test_swiglu_decode[embedding_dim_1024-hidden_dim_3584]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.00 (+100.00%)0.00 (+36.36%)0.00 (+0.00%)0.00 (+0.00%)0.00 (+287.30%)19852.86 (+1.82%)15252.51 (-15.74%)17168.46 (-5.77%)7096.71 (-56.67%)5045.95 (+313.52%)
cdc48e9 — 2026-08-03 16:32:080.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)19497.20 (n/a)18101.11 (n/a)18219.31 (n/a)16380.05 (n/a)1220.24 (n/a)

test_swiglu_decode[embedding_dim_2048-hidden_dim_2048]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.00 (+8.33%)0.00 (+24.14%)0.00 (+25.00%)0.00 (+0.00%)0.00 (+17.00%)22602.61 (-1.64%)14852.79 (-13.23%)15816.70 (-16.44%)6509.12 (-4.23%)7144.14 (+12.77%)
cdc48e9 — 2026-08-03 16:32:080.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)0.00 (n/a)22980.49 (n/a)17117.16 (n/a)18929.10 (n/a)6796.32 (n/a)6335.01 (n/a)
iron/operators/swiglu_prefill

test_swiglu_prefill[seq_len_256-embedding_dim_2048-hidden_dim_2048-prio_accuracy_False]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:590.13 (-0.76%)0.09 (-10.93%)0.08 (-16.15%)0.07 (-5.65%)0.03 (-2.80%)29862.69 (+5.94%)25082.37 (+12.24%)27667.15 (+19.20%)16017.55 (+0.73%)5728.45 (+2.95%)
cdc48e9 — 2026-08-03 16:32:080.13 (n/a)0.10 (n/a)0.09 (n/a)0.07 (n/a)0.03 (n/a)28189.10 (n/a)22347.59 (n/a)23211.24 (n/a)15901.35 (n/a)5564.14 (n/a)
iron/operators/transpose

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:591.11 (-36.51%)0.83 (-26.06%)0.78 (-16.98%)0.50 (-30.88%)0.27 (-39.74%)1053.50 (+44.69%)696.48 (+31.71%)671.90 (+20.46%)470.90 (+57.49%)243.87 (+28.48%)
cdc48e9 — 2026-08-03 16:32:081.75 (n/a)1.12 (n/a)0.94 (n/a)0.72 (n/a)0.45 (n/a)728.10 (n/a)528.78 (n/a)557.80 (n/a)299.00 (n/a)189.82 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8-num_batches_2]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:592.06 (-25.23%)0.98 (-48.80%)0.33 (-78.57%)0.30 (-75.95%)0.91 (+37.03%)3447.90 (+315.81%)2217.38 (+267.97%)3171.70 (+366.63%)510.10 (+33.74%)1541.16 (+699.37%)
cdc48e9 — 2026-08-03 16:32:082.75 (n/a)1.91 (n/a)1.54 (n/a)1.26 (n/a)0.66 (n/a)829.20 (n/a)602.60 (n/a)679.70 (n/a)381.40 (n/a)192.80 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_1-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.58 (-2.67%)1.21 (+5.65%)1.29 (+41.89%)0.66 (-20.05%)0.38 (-2.53%)792.90 (+25.08%)479.12 (-3.80%)406.80 (-29.52%)331.10 (+2.76%)189.18 (+26.62%)
4bb8427 — 2026-06-23 22:46:491.63 (n/a)1.14 (n/a)0.91 (n/a)0.83 (n/a)0.38 (n/a)633.90 (n/a)498.02 (n/a)577.20 (n/a)322.20 (n/a)149.40 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8-num_batches_1]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
b154dba — 2026-08-03 17:34:591.78 (+0.95%)1.47 (+37.14%)1.39 (+48.58%)1.25 (+57.27%)0.22 (-44.05%)420.60 (-36.42%)362.96 (-31.54%)378.00 (-32.70%)294.10 (-0.94%)52.06 (-63.02%)
cdc48e9 — 2026-08-03 16:32:081.77 (n/a)1.07 (n/a)0.93 (n/a)0.79 (n/a)0.40 (n/a)661.50 (n/a)530.14 (n/a)561.70 (n/a)296.90 (n/a)140.75 (n/a)

test_transpose[M_2048-N_64-aie_columns_1-channels_2-m_64-n_64-s_8]

Commit/Date Bandwidth (max)Bandwidth (mean)Bandwidth (median)Bandwidth (min)Bandwidth (stddev)Latency (max)Latency (mean)Latency (median)Latency (min)Latency (stddev)
4bb8427 — 2026-06-25 20:01:371.75 (-15.44%)1.32 (-0.50%)1.36 (+22.47%)0.77 (-19.20%)0.36 (-21.75%)678.10 (+23.76%)430.02 (-0.09%)385.10 (-18.34%)300.10 (+18.29%)146.96 (+19.46%)
4bb8427 — 2026-06-23 22:46:492.07 (n/a)1.32 (n/a)1.11 (n/a)0.96 (n/a)0.46 (n/a)547.90 (n/a)430.40 (n/a)471.60 (n/a)253.70 (n/a)123.02 (n/a)
Phoenix - Examples

IRON

Tested on 2026_08_03_17_33_01 at commit b154dba.

Trends:

IRON Trends

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrej andrej andrej approved these changes
@jgmelber jgmelber Awaiting requested review from jgmelber jgmelber is a code owner
@hunhoffe hunhoffe Awaiting requested review from hunhoffe hunhoffe is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@atassis @andrej