API-Reference

burla.remote_parallel_map

Run an arbitrary python function on many remote computers at the same time.

remote_parallel_map(
  function_,
  inputs,
  func_cpu=1,
  func_ram=4,
  spinner=True,
  max_parallelism=None,
  api_key=None,
)

Run provided function_ on each item in inputs at the same time, each on a separate CPU, up to 256 CPUs (as of 10/11). If more than 256 inputs are provided, inputs are queued and processed sequentially on each worker.

Parameters

Name

Description

function_

Callable

Python function. Must have single input argument, eg: function_(inputs[0]) does not raise an exception.

inputs

Iterable[Any]

Iterable of elements passable to function_.

func_cpu

int

(Optional) Number of CPU's made available to every instance of function_. The maximum possible value is 32.

func_ram

int

(Optional) Amount of RAM (GB) made available to every instance of function_. The maximum possible value is 128.

spinner

bool

(Optional) Set to False to prevent status indicator/spinner from being displayed.

max_parallelism

int

(Optional) Maximum number of function_ instances allowed to be running at the same time. Defaults to available-cpus / func_cpu.

api_key

str

(Optional) API key, for use in deployment environments where burla login cannot be run.

Returns

Type

Description

Generator

Generator yielding objects returned by function_ in no particular order.


Questions? Schedule a call with us, or email jake@burla.dev. We're always happy to talk.

Last updated