API-Reference

burla.remote_parallel_map

Run any python function on many remote computers at the same time. See our overview for a more user friendly description of how to use remote_parallel_map.

remote_parallel_map(
  function_,
  inputs,
  func_cpu=1,
  func_ram=4,
  background=False,
  spinner=True,
  generator=False,
  max_parallelism=None,
)

Run provided function_ on each item in inputs at the same time, each on a separate CPU. If more inputs are provided than there are available CPU's, they are queued and processed sequentially on each worker. remote_parallel_map can reliably queue millions of inputs.

While running:

• If the provided function_ raises an exception, the exception, including stack trace, is re-raised on the client machine in a way that looks like it was running locally.

• Your print statements (anything written to stdout/stderr) are streamed back to your local machine, appearing like they would have if running the same code locally.

When finished remote_parallel_map returns a list of values returned by each function_ call.

Parameters
Name	Description
function_	Callable Python function. Must have single input argument, eg: function_(inputs[0]) does not raise an exception.
inputs	List[Any] Iterable of elements passable to function_.
func_cpu	int (Optional) Number of CPU's made available to each running instance of function_. Max possible value is determined by your cluster machine type.
func_ram	int (Optional) Amount of RAM (GB) made available to each running instance of function_. Max possible value is determined by your cluster machine type.
background	bool (Optional) remote_parallel_map returns as soon as your inputs and function have been uploaded. The job then continues to run independently in the background until done.
spinner	bool (Optional) Set to False to hide the status indicator/spinner.
generator	bool (Optional) Set to True to return a Generator instead of a List. The generator will yield outputs as they are produced, instead of all at once.
max_parallelism	int (Optional) Maximum number of function_ instances allowed to be running at the same time.

Returns
Type	Description
List or Generator	List of objects returned by function_ in no particular order. If Generator=True, returns generator yielding objects returned by function_ in the order they are produced.

---

Questions? Schedule a call with us, or email [email protected]. We're always happy to talk.