Skip to content

[BUG]Failed to run execute(wait=False) in another thread #3229

Description

@VessaliusOz

Describe the bug
execute(wait=False) failed when running in a thread pool

import os
import time
import mars.tensor as mt
import mars.dataframe as md
import concurrent.futures


def func():
    df = md.DataFrame(
            mt.random.rand(4, 4, chunk_size=2),
            columns=list("abcd"),
        )
    df.execute()
    pid = os.getpid()
    info = df.apply(
        lambda s: s if os.getpid() == pid else (time.sleep(5) or s)
    ).execute(wait=False)
    result = info.result()
    return result


with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
    future = executor.submit(func)
    print(future.result())

To Reproduce
To help us reproducing this bug, please provide information below:

  1. Your Python version 3.9.13
  2. The version of Mars you use 0.9.0
  3. Versions of crucial packages, such as numpy, scipy and pandas
    i. numpy: 1.23.0
    ii. ray: 1.13.0
    iii. scipy: 1.8.1
    iv. pandas: 1.4.3
  4. Full stack of the error.
Traceback (most recent call last):
  File "/Users/vessalius/.local/share/virtualenvs/outer_mars-bUMhzz2w/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 101, in _ensure_future
    self._future_local.future
AttributeError: '_thread._local' object has no attribute 'future'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/vessalius/Desktop/ray_serving_test/problem.py", line 25, in <module>
    print(future.result())
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 446, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/vessalius/Desktop/ray_serving_test/problem.py", line 19, in func
    result = info.result()
  File "/Users/vessalius/.local/share/virtualenvs/outer_mars-bUMhzz2w/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 131, in result
    self._ensure_future()
  File "/Users/vessalius/.local/share/virtualenvs/outer_mars-bUMhzz2w/lib/python3.9/site-packages/mars/deploy/oscar/session.py", line 110, in _ensure_future
    self._future_local.aio_future = asyncio.wrap_future(fut)
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/futures.py", line 411, in wrap_future
    loop = events.get_event_loop()
  File "/usr/local/Cellar/python@3.9/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/events.py", line 642, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-1_0'.
  1. Minimized code to reproduce the error.

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

type: bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions