Why not just use multiprocessing? As far as I can tell from your description, threading won’t help you much, anyway… Matplotlib already threads so that you can display and interact with multiple figures at once. If you want to speed up batch processing on a multicore machine, you’re going to need multiprocessing regardless. As a … Read more
I think you may need to make your add_task method aware of whether or not its being called from a thread other than the event loop’s. That way, if it’s being called from the same thread, you can just call asyncio.async directly, otherwise, it can do some extra work to pass the task from the … Read more
threading.Thread(target=some_callable_function).start() or if you wish to pass arguments, threading.Thread(target=some_callable_function, args=(tuple, of, args), kwargs={‘dict’: ‘of’, ‘keyword’: ‘args’}, ).start()
Python’s ThreadPoolExecutor doesn’t have the feature you’re looking for, but the provided class can be easily sub-classed as follows to provide it: from concurrent import futures import queue class ThreadPoolExecutorWithQueueSizeLimit(futures.ThreadPoolExecutor): def __init__(self, maxsize=50, *args, **kwargs): super(ThreadPoolExecutorWithQueueSizeLimit, self).__init__(*args, **kwargs) self._work_queue = queue.Queue(maxsize=maxsize)
Quite some numpy routines release GIL, so they can be efficiently parallel in threads (info). Maybe you don’t need to do anything special! You can use this question to find whether the routines you need are among the ones that release GIL. In short, search for ALLOW_THREADS or nogil in the source. (Also note that … Read more
The solution is to write to the file in one thread only. import Queue # or queue in Python 3 import threading class PrintThread(threading.Thread): def __init__(self, queue): threading.Thread.__init__(self) self.queue = queue def printfiles(self, p): for path, dirs, files in os.walk(p): for f in files: print(f, file=output) def run(self): while True: result = self.queue.get() self.printfiles(result) self.queue.task_done() … Read more
The args kwarg of threading.Thread expects an iterable, and each element in that iterable is being passed to the target function. Since you are providing a string for args: t = threading.Thread(target=startSuggestworker, args=(start_keyword)) each character is being passed as a separate argument to startSuggestworker. Instead, you should provide args a tuple: t = threading.Thread(target=startSuggestworker, args=(start_keyword,)) … Read more
Use the kwargs parameter: threading.Thread(target=test, args=(arg1,), kwargs={‘arg2’:arg2}).start()
If it is acceptable to block the main thread when user haven’t provided an answer: from threading import Timer timeout = 10 t = Timer(timeout, print, [‘Sorry, times up’]) t.start() prompt = “You have %d seconds to choose the correct answer…\n” % timeout answer = input(prompt) t.cancel() Otherwise, you could use @Alex Martelli’s answer (modified … Read more
A somewhat clumsy ascii-art to demonstrate the mechanism: The join() is presumably called by the main-thread. It could also be called by another thread, but would needlessly complicate the diagram. join-calling should be placed in the track of the main-thread, but to express thread-relation and keep it as simple as possible, I choose to place … Read more