JFileChooser causes Could not initialize COM: HRESULT=0x80010106 (Thread?)

[cocoon]

Using current latest NUGET 8.14.0 in NET 8 console app.

I try to run a jar that uses JFileChooser to let one open files.

This causes the following error:

Exception in thread "Swing-Shell" java.lang.InternalError: Could not initialize COM: HRESULT=0x80010106
        at java.lang.reflect.Constructor.newInstance(Constructor.java:413)
        at IKVM.Runtime.JNI.JNIEnv.ThrowNew(Unknown Source)
        at sun.awt.shell.Win32ShellFolderManager2.initializeCom(Native Method)
        at sun.awt.shell.Win32ShellFolderManager2$ComInvoker$3.run(Win32ShellFolderManager2.java:585)
        at java.lang.Thread.run(Thread.java:978)

I could simply replicate it if I just try to create it in the csharp code like this:

using javax.swing;
JFileChooser fileChooser = new JFileChooser();

Any idea how to get it running?

I also found this bug report here, but could not see if that is in any way related.

https://bugs.openjdk.org/browse/JDK-8189938

[wasabii]

So I'm pretty sure this is us allowing this code to run:

https://github.com/ikvmnet/jdk8u/blob/3f2b6ffee4b2a077f5fc8e00d58bce906a80cca0/jdk/src/windows/native/sun/windows/ShellFolder2.cpp#L243C3-L265C1

Trying to double initialize COM on a .NET thread which already has it initialized.

[wasabii]

So this turns out to be pretty hard to solve.

.NET will initialize ALL managed threads to MTA by default. There is no way to switch on the fly. There is no way to uninitialize on the fly and let Java reinitialize.

It looks like the only solution I can come up with will just be to override and rewrite the Java AND JNI code for this one, to directly request a thread already on STA.

[wasabii]

Looks like another option is available: not creating Java threads using System.Threading.Thread. Doing that will initialize CoInit them.

However it looks like .NET does not CoInit them if they are created outside .NET, and merely enter into managed code. So, calling CreateThread explicitly seems to not CoInit them. They end up MTA/IMPLICIT_MTA. And in that state they can be CoInitialized to STA explicitely. A simple test case I wrote demonstrates this.

What this would mean is we'd rewrite all of java.lang.Thread to call CreateThread specifically on Windows, instead of creating it through .NET. We can then grab the managed Thread object for it just fine. But that would create them in IMPLICIT_MTA, allowing the existing Java code to function without alteration.

I'm going to go down this path.

[cocoon]

Wow thank you very much for the analysis!

What I ended up currently is to manually modify IL code and replace JFileChooser with FileOpen dialog + Dispatcher in the parts where I needed it most, but did not replace all occurrences.

Would be nice to have it working out of the box of course!

[wasabii]

JDK8482-b08 has added a test that covers this I believe:

jdk\test\javax\swing\JFileChooser\FileSystemView\Win32FolderSort.java

This is a lot of work to fix, so I'm not going to be be doing that today, just disabling the test to get JDK482 through. Note made here so I can reenable the test when I fix this issue.