properly close HDF5 filles and percussions for opening. by sergey-yaroslavtsev · Pull Request #1237 · silx-kit/pymca

sergey-yaroslavtsev · 2026-07-24T15:23:06Z

If user try to open hdf5 fille during fast scan it crashes.

crash should not appear
if any crash happens close file should close it completely (so reopen helps)

I could not test it (no simulation of fast scan exist), so please check it carefully...

Error

CRITICAL: Cannot access HDF5 file path <77.1>
ERROR: source.refresh() failed: Can't synchronously read data (address of object past end of allocation)
CRITICAL: Cannot access HDF5 file path <77.1>
ERROR: <class 'OSError'> Can't synchronously read data (address of object past end of allocation)   File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 587, in rowCount
    return len(self.getProxyFromIndex(index))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 372, in __len__
    return len(self.children)
               ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 228, in children
    self._children = [H5NodeProxy(self.file, i[1], self)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 330, in __init__
    nodeattr = node.attrs[cname]
               ~~~~~~~~~~^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/usr/local/lib/python3.12/dist-packages/h5py/_hl/attrs.py", line 77, in __getitem__
    attr.read(arr, mtype=htype)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 364, in h5py.h5a.AttrID.read
  File "h5py/_proxy.pyx", line 65, in h5py._proxy.attr_rw

CRITICAL: Cannot access HDF5 file path <77.1>
ERROR: <class 'OSError'> Can't synchronously read data (address of object past end of allocation)   File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 587, in rowCount
    return len(self.getProxyFromIndex(index))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 372, in __len__
    return len(self.children)
               ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 228, in children
    self._children = [H5NodeProxy(self.file, i[1], self)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py", line 330, in __init__
    nodeattr = node.attrs[cname]
               ~~~~~~~~~~^^^^^^^
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/usr/local/lib/python3.12/dist-packages/h5py/_hl/attrs.py", line 77, in __getitem__
    attr.read(arr, mtype=htype)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5a.pyx", line 364, in h5py.h5a.AttrID.read
  File "h5py/_proxy.pyx", line 65, in h5py._proxy.attr_rw

woutdenolf · 2026-07-25T13:17:59Z

    """
    sigFileUpdated = qt.pyqtSignal(object)
    sigFileAppended = qt.pyqtSignal(object)
+    sigReadError = qt.pyqtSignal(object)


I learned from @t20100 the signal names are usually somethingHappened or AboutToHappen.

sigReadError -> sigReadFailed?

sergey-yaroslavtsev · 2026-07-27T15:24:24Z

For me it is not clear if this "closing mechanism" is enough, it defiantly will not harm but is it sufficient?

t20100 · 2026-07-27T15:48:45Z

For me it is not clear if this "closing mechanism" is enough, it defiantly will not harm but is it sufficient?

It should be.
With WA_DeleteOnClose the QMessageBox gets deleted when closed, but since it also has the QNexusWidget as its parent, it also gets deleted when the QNexusWidget is deleted.

sergey-yaroslavtsev · 2026-07-27T16:17:52Z

It should be.
With WA_DeleteOnClose the QMessageBox gets deleted when closed, but since it also has the QNexusWidget as its parent, it also gets deleted when the QNexusWidget is deleted.

Sorry i meant about closing instance for HDF5 source.

woutdenolf · 2026-07-28T17:53:54Z

rowCount fails (seems like a random function which I'm assuming fails first at the moment?) -> absorb error and pop-up message box the first time, nothing related to closing HDF5 here.
_sourceSelectorSlot gets {"event":"SourceClosed"} -> close all HDF5 files.

Those two are not related as far as I can tell.

As for the non-blocking message box: this is the first time we introduce this pattern. Probably good to make a helper?

def non_blocking_user_warning(...)
    msg = qt.QMessageBox(self)
    msg.setIcon(qt.QMessageBox.Warning)
    msg.setWindowTitle("...")
    msg.setText("...")
    msg.setInformativeText("...")
    msg.setAttribute(qt.Qt.WA_DeleteOnClose)
    msg.open()

sergey-yaroslavtsev · 2026-08-04T19:28:50Z

rebased

add short comments about msg.open() and msg.setAttribute(qt.Qt.WA_DeleteOnClose)

Fail message appears on expand and collapse the index which was tried to be read + reset fail flag -> try exapnd again it will fail again. It works also for the file - on default PyMca expand the file to see all groups so if read is failed it will appear collapsed. I think it is quite elegant solution.
In the worst case refresh works as before (for some unseen issues).

I think fail should live in rowCount since it is the one who trigger children . We are not "afraid" of file/source/group/... except the one who could not be read and even want to keep the non readable source but collapsed. Thus, make it for example on init makes no sense.

let me know if i missed smthg.

sergey-yaroslavtsev · 2026-08-04T21:14:29Z

UPDATED
two potential issues:

if file is already open and HDF5 tree is expanded + user start a new scan which writes to this file (and block reading). The question is if already expanded groups/entries/... will collapse or not.

This one I tried to mimic (corruption of a single dataset) and it seems that there is no problem.

if the file could not be open at all - before even read the children (via rowCount). I am not sure it is possible. But i am also not sure at which stage the LOCK will block the opening.

woutdenolf · 2026-08-06T12:01:26Z

+        if index is not None and index.isValid():
+            self.hdf5Widget.collapse(index)
+        # A later failed expansion should warn again
+        if hasattr(model, "_readErrorReported"):


Why wouldn't it have model._readErrorReported. Instead of setting a private variable, perhaps better add a public method model.allowErrorDialog().

Public method added

woutdenolf · 2026-08-06T12:07:22Z

two potential issues

I don't understand the descriptions. Code LGTM.

sergey-yaroslavtsev · 2026-08-06T14:12:08Z

Two issues I refer seems to behave OK. Anyway now the refresh mechanism should help if smthg it broken during reading at different place (before rowCount).

sergey-yaroslavtsev requested a review from woutdenolf July 24, 2026 15:23

sergey-yaroslavtsev changed the title ~~close instance on close file; catch error and notify the user~~ properly close HDF5 filles and percussions for opening. Jul 24, 2026

woutdenolf reviewed Jul 25, 2026

View reviewed changes

sergey-yaroslavtsev requested a review from woutdenolf July 27, 2026 15:22

woutdenolf reviewed Jul 29, 2026

View reviewed changes

Comment thread src/PyMca5/PyMcaGui/pymca/QDispatcher.py

woutdenolf reviewed Jul 29, 2026

View reviewed changes

Comment thread src/PyMca5/PyMcaGui/io/hdf5/HDF5Widget.py

sergey-yaroslavtsev added 4 commits August 4, 2026 20:13

close instance on close file; catch error and notify the user

1b0b0b9

error to failed

f1b3868

open+flag instead of QTimer

0d3a3c0

collapse on fail so reexpand will fail again

4422c37

sergey-yaroslavtsev force-pushed the h5_crush branch from e81dd76 to 4422c37 Compare August 4, 2026 19:20

sergey-yaroslavtsev requested a review from woutdenolf August 4, 2026 19:29

clean the message

9763587

woutdenolf approved these changes Aug 6, 2026

View reviewed changes

add public method to reset error psoibility to raise

424255d

Navigation Menu

Uh oh!

properly close HDF5 filles and percussions for opening. - #1237

properly close HDF5 filles and percussions for opening.#1237
sergey-yaroslavtsev wants to merge 6 commits into
masterfrom
h5_crush

sergey-yaroslavtsev commented Jul 24, 2026 •

edited by woutdenolf

Loading

Uh oh!

woutdenolf Jul 25, 2026

Uh oh!

Uh oh!

sergey-yaroslavtsev commented Jul 27, 2026 •

edited

Loading

Uh oh!

t20100 commented Jul 27, 2026

Uh oh!

sergey-yaroslavtsev commented Jul 27, 2026

Uh oh!

woutdenolf commented Jul 28, 2026

Uh oh!

Uh oh!

Uh oh!

sergey-yaroslavtsev commented Aug 4, 2026

Uh oh!

sergey-yaroslavtsev commented Aug 4, 2026 •

edited

Loading

Uh oh!

woutdenolf Aug 6, 2026

Uh oh!

sergey-yaroslavtsev Aug 6, 2026

Uh oh!

woutdenolf commented Aug 6, 2026

Uh oh!

sergey-yaroslavtsev commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

sergey-yaroslavtsev commented Jul 24, 2026 • edited by woutdenolf Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

woutdenolf Jul 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sergey-yaroslavtsev commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

t20100 commented Jul 27, 2026

Uh oh!

sergey-yaroslavtsev commented Jul 27, 2026

Uh oh!

woutdenolf commented Jul 28, 2026

Uh oh!

Uh oh!

Uh oh!

sergey-yaroslavtsev commented Aug 4, 2026

Uh oh!

sergey-yaroslavtsev commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

woutdenolf Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

sergey-yaroslavtsev Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

woutdenolf commented Aug 6, 2026

Uh oh!

sergey-yaroslavtsev commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sergey-yaroslavtsev commented Jul 24, 2026 •

edited by woutdenolf

Loading

sergey-yaroslavtsev commented Jul 27, 2026 •

edited

Loading

sergey-yaroslavtsev commented Aug 4, 2026 •

edited

Loading