wqueue: Support custom user work queues. by 13022591351 · Pull Request #19982 · apache/nuttx

13022591351 · 2026-08-27T08:47:32Z

Summary

Add dynamically created user-mode work queues for Protected and Kernel
builds through the existing handle-based work queue API.
Implement configurable pthread worker pools in libs/libc/wqueue while
retaining the predefined USRWORK queue.
Harden scheduler custom queue lifecycle handling so both backends reject
submissions during teardown, release pending work ownership, wait for all
workers, clean up partial creation, and synchronously cancel every callback
using the same work_s.
Align delay validation, pending replacement, periodic requeue, priority,
and idempotent cancellation behavior between scheduler and libc backends.
Document the custom queue APIs and explicitly restrict libc user work queue
calls to task context; they must not be called from an ISR.
Related test PR: testing/ostest: Exercise custom work queues. nuttx-apps#3759.
Related STM32H7 hardware-enablement PR: stm32h7: Fix Protected user SRAM placement and attributes. #19983.

Commit organization

b817fb59b5 sched/wqueue: Harden custom queue lifecycle.
7d31e87df1 libc/wqueue: Support custom user work queues.
5be853a5eb libc/wqueue: Use the uninterruptible wait helper.
d3b65e96ce Documentation/wqueue: Document custom user queues.

The libc functional commit deliberately contains an explicit
nxsem_wait() retry on -EINTR. The next master-only cleanup commit replaces
that loop with nxsem_wait_uninterruptible() and produces the same behavior.
An older release branch where that helper is not available to Protected user
space can therefore cherry-pick the scheduler and libc functional commits,
omit only the helper cleanup, and take the documentation commit without a
downstream-only compatibility patch.

Please keep the functional and helper-cleanup commits separate for that
reason. If a squashed history is preferred, that older PX4 release branch
will not be supported by this series rather than carrying a special NuttX
patch downstream.

Impact

New feature: YES. Protected/Kernel user processes can create independent
work queues with configurable priority, stack size, and worker count.
User adaptation: NO. Existing USRWORK, HPWORK, and LPWORK users
remain supported. New user custom-queue APIs are task-context only.
Build process: NO. No new configuration symbol or build step is added.
Hardware: NO. The work queue implementation is architecture-independent.
Documentation: YES. Documentation/reference/os/wqueue.rst and public
header comments describe custom queues, teardown, errors, and execution
context.
Security: NO known impact.
Compatibility: No source or ABI break. Idle libc cancellation is now a
successful no-op, matching the scheduler backend's existing behavior.

Testing

Build host:

Linux 6.8.0-60-generic, x86_64
GNU Arm Embedded Toolchain 10.3.1 20210824

Target:

ARM Cortex-M7, STM32H7, PX4 FMUv6C
Apache NuttX master c6b349b0234466a54a624ae35ba77adb73a9ea0e
Flat and Protected builds; the Protected hardware test also contains the
patch-equivalent STM32H7 Protected-memory series from stm32h7: Fix Protected user SRAM placement and attributes. #19983.
Both configurations were rebuilt after make distclean, uploaded through
the PX4 bootloader, and tested through USB NSH with minicom.

Before change:

The libc user backend only provides the predefined USRWORK queue. A
Protected user application using work_queue_create(), work_queue_free(), or
the handle-based queue/cancel APIs cannot link a custom user queue backend.

Build output after change:

master_flat:
  FLASH     1470396 B / 1920 KB (74.79%)
  AXI_SRAM    63380 B / 512 KB  (12.09%)

master_protected kernel:
  kflash      877528 B / 896 KB (95.64%)
  ksram        54320 B / 128 KB (41.44%)

master_protected user:
  uflash      751000 B / 1 MB   (71.62%)
  usram         8192 B / 384 KB (2.08%)

Runtime commands and results, three runs per backend:

nsh> time "ostest wqueue"
Flat:             30.1490 / 30.1490 / 30.1480 s
Protected user:  15.6540 / 15.6540 / 15.6550 s

nsh> time "wqueue_test_kernel"
Protected kernel: 30.1480 / 30.1490 / 30.1490 s

Steady-state memory after the first run remained unchanged through all later
runs:

Flat:       Umem used 14600 B, 58 used / 5 free nodes
Protected:  Kmem used 10104 B, 50 used / 4 free nodes
            Umem used  9232 B, 14 used / 4 free nodes

Every run covered one- and two-worker custom queues, explicit caller
priorities, invalid arguments, periodic requeue, pending replacement,
synchronous cancellation, two concurrent callbacks using one work_s, four
simultaneous queues with 32 work items, self-destruction rejection, and
pending/running teardown. All assertions passed, heap usage did not grow, and
all custom worker pools completed teardown.

The final PR WQ source tree matches the runtime-tested source tree.

Validation:

tools/checkpatch.sh -m -g upstream/master..feature/custom-user-wqueue: pass
tools/checkpatch.sh -g upstream/master..feature/custom-user-wqueue: pass
git diff --check upstream/master..feature/custom-user-wqueue: pass
pre/post split path-limited git diff --exit-code: identical

The documentation HTML build was not run locally because sphinx-build is
not installed on the build host.

PR verification Self-Check

This PR introduces only one functional change.
I have updated all required description fields above.
My PR adheres to the contributing guidelines and coding standard.
My PR is still work in progress.
My PR is ready for review and can be safely merged.

github-actions · 2026-08-27T09:01:13Z

MemBrowse Memory Report

qemu-armv8a

Code: .rodata +48 B, .text.work_qcancel +48 B, .text.work_queue_create +56 B, .text.work_queue_free +300 B, .text.work_queue_wq +16 B, .text.work_thread +36 B, .text.work_thread_create +360 B (+0.3%, 319,956 B)

stm32-nucleo-f103rb

flash: .data +8 B, .text +176 B (+0.5%, 34,284 B / 131,072 B, total: 26% used)
sram: .data +8 B (+0.3%, 2,448 B / 20,480 B, total: 12% used)
No memory changes detected for:
arduino-mega2560
esp32-devkitc
hifive1-revb
mirtoo
qemu-intel64
rx65n-rsk2mb
s698pm-dkit

Prevent work_queue_free() from destroying predefined queues or freeing a custom queue from one of its own callbacks. Mark teardown under the queue lock, reject new submissions, return pending work to its owner, and wait for every worker before releasing queue resources. Clean up partially created worker pools, reject invalid delays, safely replace pending periodic work, and make synchronous cancellation wait for every concurrent callback using the same work structure. Tested on an STM32H7 PX4 FMUv6C with the matching ostest suite in Flat and Protected kernel builds. Assisted-by: Codex:GPT-5 Signed-off-by: DuoYuWang <thirteenking.wang@gmail.com>

Implement the handle-based create, queue, priority, cancellation, and teardown APIs for CONFIG_LIBC_USRWORK. Custom queues use configurable pthread worker pools while the predefined USRWORK queue remains available. Match scheduler-backend delay, replacement, cancellation, and lifecycle semantics. Restrict the libc backend to task context because it uses blocking synchronization. Tested on an STM32H7 PX4 FMUv6C with ostest wqueue in Protected user space. Assisted-by: Codex:GPT-5 Signed-off-by: DuoYuWang <thirteenking.wang@gmail.com>

Replace the local EINTR retry loop with nxsem_wait_uninterruptible(). This keeps the master implementation aligned with the libc semaphore API without changing cancellation behavior. Keep the cleanup separate so release branches where the helper is not available to Protected user space can use the functional commit without a downstream compatibility patch. Assisted-by: Codex:GPT-5 Signed-off-by: DuoYuWang <thirteenking.wang@gmail.com>

Describe the handle-based custom queue APIs, worker-pool creation and teardown, periodic requeue, cancellation semantics, and return values. Clarify that libc user work queue APIs use blocking synchronization and must only be called from task context, while kernel and Flat queue and asynchronous cancellation operations remain ISR-safe. Assisted-by: Codex:GPT-5 Signed-off-by: DuoYuWang <thirteenking.wang@gmail.com>

13022591351 · 2026-08-27T12:55:50Z

Updated the series organization and repeated the complete master hardware
validation on PX4 FMUv6C.

The series is now split into scheduler functionality, libc functionality with
the explicit nxsem_wait() EINTR retry, a separate master cleanup using
nxsem_wait_uninterruptible(), and RST documentation. Please keep the libc
functional and helper-cleanup commits separate; the rationale and selective
release-branch path are documented in the PR description.

Fresh make distclean build results:

master_flat:
  FLASH     1470396 B / 1920 KB (74.79%)
  AXI_SRAM    63380 B / 512 KB  (12.09%)

master_protected kernel:
  kflash      877528 B / 896 KB (95.64%)
  ksram        54320 B / 128 KB (41.44%)

master_protected user:
  uflash      751000 B / 1 MB   (71.62%)
  usram         8192 B / 384 KB (2.08%)

Three USB NSH runs per backend:

Flat kernel:      30.1490 / 30.1490 / 30.1480 s
Protected user:  15.6540 / 15.6540 / 15.6550 s
Protected kernel: 30.1480 / 30.1490 / 30.1490 s

All assertions passed. Steady-state heap usage remained unchanged after the
first run: Flat Umem 14,600 B; Protected Kmem 10,104 B and Umem 9,232 B. All
custom worker pools completed teardown.

The final PR WQ source tree matches the runtime-tested source tree. Full-range
commit-message checkpatch, code checkpatch, and git diff --check all pass.

xiaoxiang781216 · 2026-08-27T10:20:11Z

-       * the wqueue timer.
+      FAR sem_t *sync_wait = NULL;
+      int wndx;
+      FAR struct kworker_s *worker = wq_get_worker(wqueue);


move before line 59

xiaoxiang781216 · 2026-08-28T12:20:27Z

-
  pthread_attr_init(&attr);
  pthread_attr_setstacksize(&attr, CONFIG_LIBC_USRWORKSTACKSIZE);
-


add pthread_attr_setdetachstate and remove line 617

xiaoxiang781216 · 2026-08-28T12:21:43Z

-
-  return (pid_t)usrwork;
+  pthread_detach(g_usrworker.tid);
+  return (pid_t)g_usrworker.tid;


move after line 619 and remove line 601

xiaoxiang781216 · 2026-08-28T12:22:03Z

  list_initialize(&g_usrwork.q);
+  g_usrwork.exit = false;
+  g_usrworker.work = NULL;
+  g_usrworker.wait_count = 0;


why need zero again

xiaoxiang781216 · 2026-08-28T12:25:40Z


-  /* Initialize the work queue */
-
  list_initialize(&g_usrwork.q);


move the init to the variable definition

xiaoxiang781216 · 2026-08-28T12:38:58Z

-  int semcount;
+  int ret;
+
+  if (wqueue == NULL || work == NULL || worker == NULL || delay < 0 ||


move delay < 0 to next line

xiaoxiang781216 · 2026-08-28T12:40:10Z

+
+  if (work->worker != NULL)
+    {
+      list_delete(&work->node);


it's wrong to modify the list without lock, let's call cancel directly

xiaoxiang781216 · 2026-08-28T12:42:03Z

   */

-  if (curr == head)
+  if (delay == 0 || curr == head)


why need check delay

xiaoxiang781216 · 2026-08-28T12:49:05Z

  bool retimer;

-  if (wqueue == NULL || work == NULL || worker == NULL ||
+  if (wqueue == NULL || work == NULL || worker == NULL || delay < 0 ||


move delay < 0 || to next line

xiaoxiang781216 · 2026-08-28T12:49:23Z

+  int ret = OK;

-  if (wqueue == NULL || work == NULL || worker == NULL ||
+  if (wqueue == NULL || work == NULL || worker == NULL || delay < 0 ||


move to delay < 0 || next line

13022591351 requested review from Donny9, GUIDINGLI, hartmannathan, jerpelea, pkarashchenko and xiaoxiang781216 as code owners August 27, 2026 08:47

This was referenced Aug 27, 2026

testing/ostest: Exercise custom work queues. apache/nuttx-apps#3759

Open

stm32h7: Fix Protected user SRAM placement and attributes. #19983

Merged

github-actions Bot added Area: Documentation Improvements or additions to documentation Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. labels Aug 27, 2026

13022591351 mentioned this pull request Aug 27, 2026

feat(platforms): upgrade NuttX to 12.12.0 PX4/PX4-Autopilot#26215

Open

13022591351 added 4 commits August 27, 2026 20:35

13022591351 force-pushed the feature/custom-user-wqueue branch from 9dd9a74 to d3b65e9 Compare August 27, 2026 12:54

xiaoxiang781216 reviewed Aug 28, 2026

View reviewed changes

Navigation Menu

Uh oh!

wqueue: Support custom user work queues. - #19982

wqueue: Support custom user work queues.#19982
13022591351 wants to merge 4 commits into
apache:masterfrom
13022591351:feature/custom-user-wqueue

13022591351 commented Aug 27, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 27, 2026 •

edited

Loading

Uh oh!

13022591351 commented Aug 27, 2026

Uh oh!

xiaoxiang781216 Aug 27, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

xiaoxiang781216 Aug 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		pthread_attr_init(&attr);
		pthread_attr_setstacksize(&attr, CONFIG_LIBC_USRWORKSTACKSIZE);


		/* Initialize the work queue */

		list_initialize(&g_usrwork.q);

Uh oh!

Conversation

13022591351 commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Commit organization

Impact

Testing

PR verification Self-Check

Uh oh!

github-actions Bot commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MemBrowse Memory Report

Uh oh!

13022591351 commented Aug 27, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

13022591351 commented Aug 27, 2026 •

edited

Loading

github-actions Bot commented Aug 27, 2026 •

edited

Loading