Skip to content

Commit

Permalink
update vignette
Browse files Browse the repository at this point in the history
  • Loading branch information
@shikokuchuo
shikokuchuo committed Jun 5, 2024
1 parent 6d904bf commit 408601b
Show file tree
Hide file tree
Showing 2 changed files with 65 additions and 65 deletions.
122 changes: 61 additions & 61 deletions vignettes/mirai.Rmd
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ To wait for and collect the evaluated result, use the mirai's `[]` method:

``` r
m[]
#> [1] 55.65924
#> [1] 56.47801
```
It is not necessary to wait, as the mirai resolves automatically whenever the async operation completes, the evaluated result then available at `$data`.

Expand All @@ -83,7 +83,7 @@ m

``` r
m$data
#> [1] 55.65924
#> [1] 56.47801
```
For easy programmatic use of `mirai()`, '.expr' accepts a pre-constructed language object, and also a list of named arguments passed via '.args'. So, the following would be equivalent to the above:

Expand All @@ -99,7 +99,7 @@ args <- list(mean = input$x, sd = input$y)
m <- mirai(.expr = expr, .args = args)

m[]
#> [1] 56.80728
#> [1] 57.96033
```

[&laquo; Back to ToC](#table-of-contents)
Expand Down Expand Up @@ -217,12 +217,12 @@ status()
#>
#> $daemons
#> i online instance assigned complete
#> abstract://1796721868fe833a4d79a9d2 1 1 1 0 0
#> abstract://409093e696175813a422e2d0 2 1 1 0 0
#> abstract://ed35107d353446c7272b788d 3 1 1 0 0
#> abstract://0dc671f0526afd9a6f9f23bd 4 1 1 0 0
#> abstract://436170218593260e359fbe8e 5 1 1 0 0
#> abstract://0cf06fe429656deb4e68fba3 6 1 1 0 0
#> abstract://209edb8a2214ffe9cce66fd3 1 1 1 0 0
#> abstract://2063e9c1b5bf794f55696286 2 1 1 0 0
#> abstract://ed68621ceb9e7d97453f2015 3 1 1 0 0
#> abstract://a7af94578063aeb81439c76b 4 1 1 0 0
#> abstract://ca5825275d36c73aec4e4e72 5 1 1 0 0
#> abstract://8ce12e8d36fbecf5ef47b430 6 1 1 0 0
```
The default `dispatcher = TRUE` creates a `dispatcher()` background process that connects to individual daemon processes on the local machine. This ensures that tasks are dispatched efficiently on a first-in first-out (FIFO) basis to daemons for processing. Tasks are queued at the dispatcher and sent to a daemon as soon as it can accept the task for immediate execution.

Expand Down Expand Up @@ -251,7 +251,7 @@ status()
#> [1] 6
#>
#> $daemons
#> [1] "abstract://5c6c6415209ac993f2ffc3bd"
#> [1] "abstract://30f0b5d783e02dc7e35b4956"
```
This implementation sends tasks immediately, and ensures that tasks are evenly-distributed amongst daemons. This means that optimal scheduling is not guaranteed as the duration of tasks cannot be known *a priori*. As an example, tasks could be queued at a daemon behind a long-running task, whilst other daemons are idle having already completed their tasks.

Expand All @@ -277,14 +277,14 @@ By super-assignment, the conenction 'con' will be available in the global enviro
``` r
m <- mirai(capture.output(str(con)))
m[]
#> [1] "Formal class 'SQLiteConnection' [package \"RSQLite\"] with 8 slots"
#> [2] " ..@ ptr :<externalptr> "
#> [3] " ..@ dbname : chr \"/tmp/RtmpW3mY6Z/file11a584826e313\""
#> [4] " ..@ loadable.extensions: logi TRUE"
#> [5] " ..@ flags : int 70"
#> [6] " ..@ vfs : chr \"\""
#> [7] " ..@ ref :<environment: 0x603dff46ed00> "
#> [8] " ..@ bigint : chr \"integer64\""
#> [1] "Formal class 'SQLiteConnection' [package \"RSQLite\"] with 8 slots"
#> [2] " ..@ ptr :<externalptr> "
#> [3] " ..@ dbname : chr \"/tmp/RtmpxYsMeM/file30184b384055\""
#> [4] " ..@ loadable.extensions: logi TRUE"
#> [5] " ..@ flags : int 70"
#> [6] " ..@ vfs : chr \"\""
#> [7] " ..@ ref :<environment: 0x655515b5e890> "
#> [8] " ..@ bigint : chr \"integer64\""
#> [9] " ..@ extended_types : logi FALSE"
```
Disconnect from the database everywhere, and set the number of daemons to zero to reset.
Expand Down Expand Up @@ -373,7 +373,7 @@ By specifying `dispatcher = FALSE`, remote daemons connect directly to the host

``` r
daemons(url = host_url(), dispatcher = FALSE)
#> [1] "tcp://hostname:32993"
#> [1] "tcp://hostname:46259"
```
Note that above, calling `host_url()` without a port value uses the default of '0'. This is a wildcard value that will automatically cause a free ephemeral port to be assigned. The actual assigned port is provided in the return value of the call, or it may be queried at any time via `status()`.

Expand All @@ -387,7 +387,7 @@ status()
#> [1] 0
#>
#> $daemons
#> [1] "tcp://hostname:32993"
#> [1] "tcp://hostname:46259"
```
To reset all connections and revert to default behaviour:

Expand Down Expand Up @@ -459,10 +459,10 @@ daemons(n = 2, url = host_url())
``` r
launch_remote(1:2)
#> [1]
#> Rscript -e "mirai::daemon('tcp://hostname:43647',rs=c(10407,484960323,-725392216,-1321459063,493984950,-173687873,-604042508))"
#> Rscript -e "mirai::daemon('tcp://hostname:33607',rs=c(10407,-902072707,1830763290,857515475,137330936,2022629017,-1437661818))"
#>
#> [2]
#> Rscript -e "mirai::daemon('tcp://hostname:44795',rs=c(10407,-701915261,1786274565,-1507697296,-1492186855,-1556181348,582193203))"
#> Rscript -e "mirai::daemon('tcp://hostname:41573',rs=c(10407,1887284906,-1386899533,-521032080,-520854166,1624118937,-1650200824))"
```

``` r
Expand Down Expand Up @@ -492,37 +492,37 @@ The generated self-signed certificate is available via `launch_remote()`. This f
``` r
launch_remote(1)
#> [1]
#> Rscript -e "mirai::daemon('wss://hostname:46177/1',tls=c('-----BEGIN CERTIFICATE-----
#> Rscript -e "mirai::daemon('wss://hostname:44651/1',tls=c('-----BEGIN CERTIFICATE-----
#> MIIFNzCCAx+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQDDAhrdW1h
#> bW90bzERMA8GA1UECgwITmFub25leHQxCzAJBgNVBAYTAkpQMB4XDTAxMDEwMTAw
#> MDAwMFoXDTMwMTIzMTIzNTk1OVowMzERMA8GA1UEAwwIa3VtYW1vdG8xETAPBgNV
#> BAoMCE5hbm9uZXh0MQswCQYDVQQGEwJKUDCCAiIwDQYJKoZIhvcNAQEBBQADggIP
#> ADCCAgoCggIBALBD7Iv58ciO67OpdheBhgOVCpRR6jz8nzVmSXgyaj35SQD942rH
#> QP/jTq95g4DwMxjH1E3GfJf7UIWMqKT6MVHkkkXGMIq8oQm826ytpYE5VHC6LOBc
#> 5Rg2dP9zp04h26aJMpidhNaMnEBML3HIPtWKEfXJgRThbnEzvUkRYtyCA9mUhEad
#> /IzVONEfU3GU78KaldoLJJpQRu9pftyLO+VtvFrbftMsdqB+aBdAh/dXB+48zKLb
#> rm98HVkN4LwtLzD9H0GILE/6AjS5ZBHfgsFymhv2BbDPtbSeh561p9VrUeP78jyp
#> kqnX1eAoa7U6bv0lsR9t0HjLkFxtH7TVarku170nQ2+0i6os4toK7EFw5iRT2g++
#> C33JqkiCEsOX1rz3DGeaHBcJ7XV7bbBFRA7DblasOdpbQMMOLo0+kHSNTI53mD1Y
#> gibdBX6xmBzEiNpvF713iQtaTgYbhA+tfxz3AD74jVWOb3jRNtAIzMoxq/mgYEHB
#> hlquieuvrCI21dffpbeI8JcnsTzEJvnhntYCCCgUy50EHnQ+iwKi0KUbyLnXUS1E
#> vdmR1frh8F/QZDNDRQ4oF7NhYf/F3uoZsW2wm01wRqR+86lUofuI491Ukbs3D2ZB
#> wuvOZRBTcNdur5/52rtHBkdGEOqNRbCNIVHwiuqkCbAjoscJ0SHH85xDAgMBAAGj
#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFEpbcC/HMAjutGRTLyM1
#> 8qV2/rplMB8GA1UdIwQYMBaAFEpbcC/HMAjutGRTLyM18qV2/rplMA0GCSqGSIb3
#> DQEBCwUAA4ICAQAnTc4jcs+mbXe+ahl8fUhZMl3VQ9FtwBVOGWgzwOHkZhIC0xp+
#> DImnzn909plHnjSZcDZfkLurMpIt+nqu7GnCQPKUMtFLWTAyO0hIjo4ZWONMWljx
#> a7FD1giqEKQfrQNGoNneF3NO31fJa49X8EE4WRZduSUHNMrL/8Rc5gBsTcGi0ym1
#> XIGajSol20lPB9DguGTNDxM2cjLBk+xXMIOVYxH8PseKwf09NTnVcSGr71O4Fyo6
#> IIKAxN0ssrh83Kf3+2l8x5T/hrXZk/+LUp1kfLWb+DHw/R2iNSQGUWdSTWpXx0bM
#> Odqa4VLiyJ9rQczvQ99QTw5aDpKEtr/oIJNr2Tg4renxWE+OoTcEjB0K82cxPfmO
#> PmM8oojWQ2sfaDuAErFhlIDoOm1aOmmNqOmcNzM4tRaDS/8QtvCjsM1ZKxIu6s2Q
#> ZcfeynCNV0/YL50auxMSFQm+NVGIliqoLFHm1ILOfOgR+9ioEO0D92OwG/rw89rP
#> 63mT5x1yV4FyPDHstHba2/gXSxYB4G9H8q99VulECLmNNKfwodh6yKM1FtPv2Hbc
#> /wvb3fRe/zZR7/mrho7UshBxjxpDRZdO0cqzgujZnXCISZM2vWjt4JyeMagORH9p
#> 1QRCkVaB9egFxX0G9oMbhXnBAu4Dx7nLcT8bnqUDe9zPJZuGMwt9CtjCsg==
#> ADCCAgoCggIBAMNqgH4u81CEsELBTYFXjmkn6uyG9RRe4F9hgmzvS8tMz6R4mhaj
#> UPOriFLKNe5i6H3FK/iMkT5n+OjdXhcK9tdDV5yJyWhBhLywfCWw8AjVFCUrAMSY
#> nsu/AxstTGt/cdYZRslsChl6pXVylDLmxvH7iGjFD8YHcDax5b91mcJTTZhfxX/f
#> NmihoIuJj5cz7+SdzNG3U+uhUvuwBZu3E41UlWppy/RE8CSJ0T13Fi8T8ugFF0Qt
#> hsVI36zNvpumiHpV82DpKxKKDKuh5BSjCYJvA44KsDQsFwYFtFDmCQ06EYgyTwlu
#> FV/bjkxaSTn9NxGqkkoIoZyRYGggMx6gJu0W96QAh5zh/oC/71E5tjQmOK2f/ITg
#> GsX1/e+GR3tOG4SYxLsufa8dmmZPrJpI1Dri0KhkgBRodaK6ikdwmMB4MMdQZCVs
#> FyKs0K5SMLbDyGvU8HW9ilGZolZ1DILYGGbq7C0hkxj0dSdJop+ZkzTNnoPgiRBv
#> xlwQ2oYjwmtDYQPeZi7BC9KAUHNtrRVS0MUtH0NVO38VOxWaOT1qk//oz2LIkVUj
#> B/1U6NcEQ/Ii3Gf+5GjkfZ1Vv8IsQYrGSVCjI7oy8t6J+IediMg0EhFNZ5luPqow
#> EwAl1vJNPg7I16fqciTJkV2RyG5YAti1QuvCCs7wkZiBGcnk9JpGmdPbAgMBAAGj
#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFIyCC//KcUKvxBzJEXzq
#> 53e9IOqMMB8GA1UdIwQYMBaAFIyCC//KcUKvxBzJEXzq53e9IOqMMA0GCSqGSIb3
#> DQEBCwUAA4ICAQADou5F1d1vg9+Rk5Ibl6wvzu+XNof/EKQEhyTuStnXKvi411/R
#> qYU+Y0dumNRdXkz43aRJqI1oF2jWc1PrkrihT9N7Oo35dq/hcM+q/W5tgDuZLsPM
#> /wFo472KsItVV+YugCtA/w87cMzyHAL/4w1rFTTdsQHAvLbz3RhAVgDl7VfcQQ8h
#> d+QvJfuogEHnZNg1QTwKBXUHixE1EI8FeaUecfziYEVfjrOQIp036kCmif7AHbZa
#> Cqom1XZkgtWL0yELK6zZlMvZFJp44KMgrnTBsM0cH5fTkMEBOirqkOpf7ctpr6Jx
#> qfuujYPEH1yK3faflPNDJ0Id5TLa1EuPkADntACip5UcOxBQSTtm5lC1bY9kIC/D
#> bcRJJzQuRpMhjMbkz36R5pHbP1gMyR5f0hqyz/wSGy5PprP/aH51zsJtQXEY4X86
#> BoabPzX1d/U55l+yxF5XbkNhzHVcu4faWYkZFhlR+1RlUqIwSzAwDZJt+071st1J
#> 5zIsET5hgsnguVJoHLpHZoTbRKxfYVz6Jz9Bjhr/ILhlOCx1F8XfS7UYTBw4l3/0
#> 2oK8vwkDsZPBxYGXgWpRj6yvQQs8uPx0DX5dxUMYXPkRtqwWZYutOzKTFPexNRh2
#> 5DMrwMg3ZfpDSuKj4JEXLTaOvEfllpfl/0AuuokV/N8H8YuhlidQw5ItLw==
#> -----END CERTIFICATE-----
#> ',''),rs=c(10407,2076945126,615221295,-1686802268,-462977195,-1340362734,506390635))"
#> ',''),rs=c(10407,1002557613,-2105941110,-1884080253,1675107048,139678665,969166070))"
```
The printed value may be deployed directly on a remote machine.

Expand Down Expand Up @@ -759,10 +759,10 @@ The results of a mirai_map `x` may be collected using `x[]`. This waits for all

Key advantages of `mirai_map()`:

- Returns immediately with all evaluations taking place asynchronously. Printing a 'mirai map' object shows the current completion progress.
- The '.promise' argument allows a promise to registered against each mirai, which can be used to perform side-effects.
- Returns evaluation errors as 'miraiError' or 'errorValue' as the case may be, rather than causing the entire operation to fail. This allows more efficient recovery from partial failure.
- Does not rely on a 'chunking' algorithm that attempts to split work into batches according to the number of available daemons, as implemented for example in the `parallel` package. Chunking cannot take into account varying or unpredictable compute times over the indices. It can be optimal to rely on `mirai` for scheduling instead. This is demonstrated in the example below.
1. Returns immediately with all evaluations taking place asynchronously. Printing a 'mirai map' object shows the current completion progress.
1. The '.promise' argument allows a promise to registered against each mirai, which can be used to perform side-effects.
1. Returns evaluation errors as 'miraiError' or 'errorValue' as the case may be, rather than causing the entire operation to fail. This allows more efficient recovery from partial failure.
1. Does not rely on a 'chunking' algorithm that attempts to split work into batches according to the number of available daemons, as implemented for example in the `parallel` package. Chunking cannot take into account varying or unpredictable compute times over the indices. It can be optimal to rely on `mirai` for scheduling instead. This is demonstrated in the example below.

``` r
library(mirai)
Expand All @@ -776,19 +776,19 @@ daemons(4, retry = FALSE)
vec <- c(1, 1, 4, 4, 1, 1, 1, 1)
system.time(mirai_map(vec, Sys.sleep)[])
#> user system elapsed
#> 0.006 0.000 4.040
#> 0.005 0.002 4.006
```

``` r
system.time(parLapplyLB(cl, vec, Sys.sleep))
#> user system elapsed
#> 0.006 0.009 5.010
#> 0.026 0.063 5.084
```

``` r
system.time(parLapply(cl, vec, Sys.sleep))
#> user system elapsed
#> 0.009 0.001 8.009
#> 0.008 0.073 8.075
```
`.args` is used to specify further constant arguments to `.f` - the 'mean' and 'sd' in the example below:

Expand All @@ -798,13 +798,13 @@ with(
mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[]
)
#> [[1]]
#> [1] 19.72028
#> [1] 21.47475
#>
#> [[2]]
#> [1] 20.56137 19.78063
#> [1] 19.96177 18.63839
#>
#> [[3]]
#> [1] 21.50056 24.13770 23.32697
#> [1] 18.61285 22.23874 15.69931
```
Use `...` to further specify objects referenced but not defined in `.f` - the 'do' in the anonymous function below:

Expand All @@ -826,13 +826,13 @@ ml
``` r
ml[]
#> $a
#> [1] "5b"
#> [1] "ea"
#>
#> $b
#> [1] 2d 7c
#> [1] b7 66
#>
#> $c
#> [1] "239565"
#> [1] "3c7368"
```
Use of `mirai_map()` assumes that `daemons()` have previously been set. If not then one (non-dispatcher) daemon is set to allow the function to proceed. This ensures safe behaviour, but is unlikely to be optimal, so please ensure daemons are set beforehand.

Expand Down
8 changes: 4 additions & 4 deletions vignettes/mirai.Rmd.orig
View file
Original file line number Diff line number Diff line change
Expand Up @@ -539,10 +539,10 @@ The results of a mirai_map `x` may be collected using `x[]`. This waits for all

Key advantages of `mirai_map()`:

- Returns immediately with all evaluations taking place asynchronously. Printing a 'mirai map' object shows the current completion progress.
- The '.promise' argument allows a promise to registered against each mirai, which can be used to perform side-effects.
- Returns evaluation errors as 'miraiError' or 'errorValue' as the case may be, rather than causing the entire operation to fail. This allows more efficient recovery from partial failure.
- Does not rely on a 'chunking' algorithm that attempts to split work into batches according to the number of available daemons, as implemented for example in the `parallel` package. Chunking cannot take into account varying or unpredictable compute times over the indices. It can be optimal to rely on `mirai` for scheduling instead. This is demonstrated in the example below.
1. Returns immediately with all evaluations taking place asynchronously. Printing a 'mirai map' object shows the current completion progress.
1. The '.promise' argument allows a promise to registered against each mirai, which can be used to perform side-effects.
1. Returns evaluation errors as 'miraiError' or 'errorValue' as the case may be, rather than causing the entire operation to fail. This allows more efficient recovery from partial failure.
1. Does not rely on a 'chunking' algorithm that attempts to split work into batches according to the number of available daemons, as implemented for example in the `parallel` package. Chunking cannot take into account varying or unpredictable compute times over the indices. It can be optimal to rely on `mirai` for scheduling instead. This is demonstrated in the example below.
```{r chunk}
library(mirai)
library(parallel)
Expand Down

0 comments on commit 408601b

Please sign in to comment.