Python

Python Multiprocessing Module and Closures
Python closures python
Published: 2013-01-16
Python Multiprocessing Module and Closures

At work, I wrote a Python script which uses the multiprocessing module to process many servers in parallel. The code looks something like:

1
2
3
4
5
6
7
def processServer(server):
    # Do work...

numParallelTasks = ...
servers = [...]
pool = multiprocessing.Pool(processes=numParallelTasks)
results = pool.map(processServer, servers)

I wanted to pass some extra state to processServer without using a global variable. My first attempt was to use a closure, so I wrote the following:

Read more...