getting-started-parfor.RmdThis function is adapted from the excellent
foreach::foreach package. The difference is:
misc::parfor calls a function in a loop. Two of the
advantages of misc::parfor over
foreach::foreach are:
you don’t have to register a parallel backend
before using it. Just specify cl to use parallel
processing.
you can directly monitor the progress of the parallel computation with a progress bar.
library(misc)## Loading required package: foreach## Loading required package: doSNOW## Loading required package: iterators## Loading required package: snow## Loading required package: parallel##
## Attaching package: 'parallel'## The following objects are masked from 'package:snow':
##
## clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
## clusterExport, clusterMap, clusterSplit, makeCluster, parApply,
## parCapply, parLapply, parRapply, parSapply, splitIndices,
## stopCluster## Loading required package: compiler
misc::parfor(function(x) x^2, 1:10)## | | | 0% | |======= | 10% | |============== | 20% | |===================== | 30% | |============================ | 40% | |=================================== | 50% | |========================================== | 60% | |================================================= | 70% | |======================================================== | 80% | |=============================================================== | 90% | |======================================================================| 100%## [1] 1 4 9 16 25 36 49 64 81 100
misc::parfor(function(x) x^2, 1:10, cl = 2)## | | | 0% | |======= | 10% | |============== | 20% | |===================== | 30% | |============================ | 40% | |=================================== | 50% | |========================================== | 60% | |================================================= | 70% | |======================================================== | 80% | |=============================================================== | 90% | |======================================================================| 100%## [1] 1 4 9 16 25 36 49 64 81 100
misc::parfor(function(x) x^2, 1:10, verbose = TRUE)## | | | 0%evaluation # 1:
## $i
## [1] 1
##
## | |======= | 10%result of evaluating expression:
## [1] 1
## got results for task 1
## numValues: 1, numResults: 1, stopped: FALSE
## returning status FALSE
## evaluation # 2:
## $i
## [1] 2
##
## | |============== | 20%result of evaluating expression:
## [1] 4
## got results for task 2
## numValues: 2, numResults: 2, stopped: FALSE
## returning status FALSE
## evaluation # 3:
## $i
## [1] 3
##
## | |===================== | 30%result of evaluating expression:
## [1] 9
## got results for task 3
## numValues: 3, numResults: 3, stopped: FALSE
## returning status FALSE
## evaluation # 4:
## $i
## [1] 4
##
## | |============================ | 40%result of evaluating expression:
## [1] 16
## got results for task 4
## numValues: 4, numResults: 4, stopped: FALSE
## returning status FALSE
## evaluation # 5:
## $i
## [1] 5
##
## | |=================================== | 50%result of evaluating expression:
## [1] 25
## got results for task 5
## numValues: 5, numResults: 5, stopped: FALSE
## returning status FALSE
## evaluation # 6:
## $i
## [1] 6
##
## | |========================================== | 60%result of evaluating expression:
## [1] 36
## got results for task 6
## numValues: 6, numResults: 6, stopped: FALSE
## returning status FALSE
## evaluation # 7:
## $i
## [1] 7
##
## | |================================================= | 70%result of evaluating expression:
## [1] 49
## got results for task 7
## numValues: 7, numResults: 7, stopped: FALSE
## returning status FALSE
## evaluation # 8:
## $i
## [1] 8
##
## | |======================================================== | 80%result of evaluating expression:
## [1] 64
## got results for task 8
## numValues: 8, numResults: 8, stopped: FALSE
## returning status FALSE
## evaluation # 9:
## $i
## [1] 9
##
## | |=============================================================== | 90%result of evaluating expression:
## [1] 81
## got results for task 9
## numValues: 9, numResults: 9, stopped: FALSE
## returning status FALSE
## evaluation # 10:
## $i
## [1] 10
##
## | |======================================================================| 100%result of evaluating expression:
## [1] 100
## got results for task 10
## numValues: 10, numResults: 10, stopped: FALSE
## returning status FALSE
## numValues: 10, numResults: 10, stopped: TRUE
## first call to combine function
## evaluating call object to combine results:
## fun(result.1, result.2, result.3, result.4, result.5, result.6,
## result.7, result.8, result.9, result.10)## [1] 1 4 9 16 25 36 49 64 81 100
misc::parfor(function(x) x^3, 1:10, show_progress = FALSE)## [1] 1 8 27 64 125 216 343 512 729 1000
misc::parfor(function(x) x^3, 1:10, show_progress = FALSE)## [1] 1 8 27 64 125 216 343 512 729 1000
foo <- function(x)
{
print(x)
return(x*0.5)
}
misc::parfor(foo, 1:10, show_progress = FALSE,
verbose = TRUE, combine = rbind)## evaluation # 1:
## $i
## [1] 1
##
## [1] 1
## result of evaluating expression:
## [1] 0.5
## got results for task 1
## numValues: 1, numResults: 1, stopped: FALSE
## returning status FALSE
## evaluation # 2:
## $i
## [1] 2
##
## [1] 2
## result of evaluating expression:
## [1] 1
## got results for task 2
## numValues: 2, numResults: 2, stopped: FALSE
## returning status FALSE
## evaluation # 3:
## $i
## [1] 3
##
## [1] 3
## result of evaluating expression:
## [1] 1.5
## got results for task 3
## numValues: 3, numResults: 3, stopped: FALSE
## returning status FALSE
## evaluation # 4:
## $i
## [1] 4
##
## [1] 4
## result of evaluating expression:
## [1] 2
## got results for task 4
## numValues: 4, numResults: 4, stopped: FALSE
## returning status FALSE
## evaluation # 5:
## $i
## [1] 5
##
## [1] 5
## result of evaluating expression:
## [1] 2.5
## got results for task 5
## numValues: 5, numResults: 5, stopped: FALSE
## returning status FALSE
## evaluation # 6:
## $i
## [1] 6
##
## [1] 6
## result of evaluating expression:
## [1] 3
## got results for task 6
## numValues: 6, numResults: 6, stopped: FALSE
## returning status FALSE
## evaluation # 7:
## $i
## [1] 7
##
## [1] 7
## result of evaluating expression:
## [1] 3.5
## got results for task 7
## numValues: 7, numResults: 7, stopped: FALSE
## returning status FALSE
## evaluation # 8:
## $i
## [1] 8
##
## [1] 8
## result of evaluating expression:
## [1] 4
## got results for task 8
## numValues: 8, numResults: 8, stopped: FALSE
## returning status FALSE
## evaluation # 9:
## $i
## [1] 9
##
## [1] 9
## result of evaluating expression:
## [1] 4.5
## got results for task 9
## numValues: 9, numResults: 9, stopped: FALSE
## returning status FALSE
## evaluation # 10:
## $i
## [1] 10
##
## [1] 10
## result of evaluating expression:
## [1] 5
## got results for task 10
## numValues: 10, numResults: 10, stopped: FALSE
## returning status FALSE
## numValues: 10, numResults: 10, stopped: TRUE
## first call to combine function
## evaluating call object to combine results:
## fun(result.1, result.2, result.3, result.4, result.5, result.6,
## result.7, result.8, result.9, result.10)## [,1]
## result.1 0.5
## result.2 1.0
## result.3 1.5
## result.4 2.0
## result.5 2.5
## result.6 3.0
## result.7 3.5
## result.8 4.0
## result.9 4.5
## result.10 5.0
misc::parfor(foo, 1:10, show_progress = FALSE,
verbose = TRUE, combine = cbind)## evaluation # 1:
## $i
## [1] 1
##
## [1] 1
## result of evaluating expression:
## [1] 0.5
## got results for task 1
## numValues: 1, numResults: 1, stopped: FALSE
## returning status FALSE
## evaluation # 2:
## $i
## [1] 2
##
## [1] 2
## result of evaluating expression:
## [1] 1
## got results for task 2
## numValues: 2, numResults: 2, stopped: FALSE
## returning status FALSE
## evaluation # 3:
## $i
## [1] 3
##
## [1] 3
## result of evaluating expression:
## [1] 1.5
## got results for task 3
## numValues: 3, numResults: 3, stopped: FALSE
## returning status FALSE
## evaluation # 4:
## $i
## [1] 4
##
## [1] 4
## result of evaluating expression:
## [1] 2
## got results for task 4
## numValues: 4, numResults: 4, stopped: FALSE
## returning status FALSE
## evaluation # 5:
## $i
## [1] 5
##
## [1] 5
## result of evaluating expression:
## [1] 2.5
## got results for task 5
## numValues: 5, numResults: 5, stopped: FALSE
## returning status FALSE
## evaluation # 6:
## $i
## [1] 6
##
## [1] 6
## result of evaluating expression:
## [1] 3
## got results for task 6
## numValues: 6, numResults: 6, stopped: FALSE
## returning status FALSE
## evaluation # 7:
## $i
## [1] 7
##
## [1] 7
## result of evaluating expression:
## [1] 3.5
## got results for task 7
## numValues: 7, numResults: 7, stopped: FALSE
## returning status FALSE
## evaluation # 8:
## $i
## [1] 8
##
## [1] 8
## result of evaluating expression:
## [1] 4
## got results for task 8
## numValues: 8, numResults: 8, stopped: FALSE
## returning status FALSE
## evaluation # 9:
## $i
## [1] 9
##
## [1] 9
## result of evaluating expression:
## [1] 4.5
## got results for task 9
## numValues: 9, numResults: 9, stopped: FALSE
## returning status FALSE
## evaluation # 10:
## $i
## [1] 10
##
## [1] 10
## result of evaluating expression:
## [1] 5
## got results for task 10
## numValues: 10, numResults: 10, stopped: FALSE
## returning status FALSE
## numValues: 10, numResults: 10, stopped: TRUE
## first call to combine function
## evaluating call object to combine results:
## fun(result.1, result.2, result.3, result.4, result.5, result.6,
## result.7, result.8, result.9, result.10)## result.1 result.2 result.3 result.4 result.5 result.6 result.7 result.8
## [1,] 0.5 1 1.5 2 2.5 3 3.5 4
## result.9 result.10
## [1,] 4.5 5
foo2 <- function(x)
{
print(x)
return(x*0.5)
}
misc::parfor(foo2, 1:10, show_progress = FALSE,
verbose = TRUE, combine = '+')## evaluation # 1:
## $i
## [1] 1
##
## [1] 1
## result of evaluating expression:
## [1] 0.5
## got results for task 1
## numValues: 1, numResults: 1, stopped: FALSE
## returning status FALSE
## evaluation # 2:
## $i
## [1] 2
##
## [1] 2
## result of evaluating expression:
## [1] 1
## got results for task 2
## numValues: 2, numResults: 2, stopped: FALSE
## first call to combine function
## evaluating call object to combine results:
## fun(result.1, result.2)
## returning status FALSE
## evaluation # 3:
## $i
## [1] 3
##
## [1] 3
## result of evaluating expression:
## [1] 1.5
## got results for task 3
## numValues: 3, numResults: 3, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.3)
## returning status FALSE
## evaluation # 4:
## $i
## [1] 4
##
## [1] 4
## result of evaluating expression:
## [1] 2
## got results for task 4
## numValues: 4, numResults: 4, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.4)
## returning status FALSE
## evaluation # 5:
## $i
## [1] 5
##
## [1] 5
## result of evaluating expression:
## [1] 2.5
## got results for task 5
## numValues: 5, numResults: 5, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.5)
## returning status FALSE
## evaluation # 6:
## $i
## [1] 6
##
## [1] 6
## result of evaluating expression:
## [1] 3
## got results for task 6
## numValues: 6, numResults: 6, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.6)
## returning status FALSE
## evaluation # 7:
## $i
## [1] 7
##
## [1] 7
## result of evaluating expression:
## [1] 3.5
## got results for task 7
## numValues: 7, numResults: 7, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.7)
## returning status FALSE
## evaluation # 8:
## $i
## [1] 8
##
## [1] 8
## result of evaluating expression:
## [1] 4
## got results for task 8
## numValues: 8, numResults: 8, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.8)
## returning status FALSE
## evaluation # 9:
## $i
## [1] 9
##
## [1] 9
## result of evaluating expression:
## [1] 4.5
## got results for task 9
## numValues: 9, numResults: 9, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.9)
## returning status FALSE
## evaluation # 10:
## $i
## [1] 10
##
## [1] 10
## result of evaluating expression:
## [1] 5
## got results for task 10
## numValues: 10, numResults: 10, stopped: FALSE
## calling combine function
## evaluating call object to combine results:
## fun(accum, result.10)
## returning status FALSE
## numValues: 10, numResults: 10, stopped: TRUE## [1] 27.5