future.p2p: A Personal P2P Cluster

If you have access to different machines, locally or remotely, you can set up a “personal” P2P cluster to distribute you R tasks to those machines using the future.p2p package.

This document gives instructions how to set up a personal P2P cluster. Setting up a “personal” cluster is very easy. You can either set up the plan() first or launch your P2P workers first - the order does not matter. I find in convenient to launch the P2P workers first. To launch workers, log into your different machines and run:¹ ²

$ Rscript -e future.p2p::worker

Launch as many workers as you needed. You can add more later, if needed.

With your P2P workers set up, you can start using your personal cluster, by setting the future plan. For example,

library(future)
plan(future.p2p::cluster)

f <- future(Sys.getpid())
v <- value(f)
print(v)

will give you the process ID of the P2P worker that took on this future.

Next, try the Mandelbrot demo of the future package;

library(future)
plan(future.p2p::cluster)
demo("mandelbrot", ask = TRUE)

Each tile will be processed by a separate P2P worker.

PS. Note how we did not have to call host_cluster(), which is only needed for shared P2P cluster.

Henrik Bengtsson