From a2f0df33fb4d40f19e579d568329307a9b7a22cd Mon Sep 17 00:00:00 2001 From: shikokuchuo <53399081+shikokuchuo@users.noreply.github.com> Date: Wed, 30 Oct 2024 14:52:52 +0000 Subject: [PATCH] =?UTF-8?q?Deploying=20to=20gh-pages=20from=20@=20shikokuc?= =?UTF-8?q?huo/mirai@a555938332fc47ede3c946114733c3cb4b98cd46=20?= =?UTF-8?q?=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- articles/mirai.html | 124 ++++++++++++++++++----------------- pkgdown.yml | 2 +- reference/host_url.html | 10 +-- reference/remote_config.html | 2 +- reference/serial_config.html | 4 +- search.json | 2 +- 6 files changed, 73 insertions(+), 71 deletions(-) diff --git a/articles/mirai.html b/articles/mirai.html index 14b9a229d..3ca967a27 100644 --- a/articles/mirai.html +++ b/articles/mirai.html @@ -154,7 +154,7 @@

Example 1: Compute-intensive Ope [] method:

 m[]
-#> [1] 3.809949 3.009536 3.580907 4.174612 3.275564
+#> [1] 4.021118 4.931766 5.243967 3.375100 5.667178

As a mirai represents an async operation, it is never necessary to wait for it. Other code can continue to be run. Once it completes, the return value automatically becomes available at $data.

@@ -162,7 +162,7 @@

Example 1: Compute-intensive Ope m #> < mirai [$data] > m$data -#> [1] 3.809949 3.009536 3.580907 4.174612 3.275564 +#> [1] 4.021118 4.931766 5.243967 3.375100 5.667178

For easy programmatic use of mirai(), ‘.expr’ accepts a pre-constructed language object, and also a list of named arguments passed via ‘.args’. So, the following would be equivalent to the @@ -174,7 +174,7 @@

Example 1: Compute-intensive Ope m <- mirai(.expr = expr, .args = args) m[] -#> [1] 2.857742 4.677513 4.285234 3.773817 2.754848 +#> [1] 2.344214 5.151972 2.745938 5.707968 3.244619

« Back to ToC

@@ -250,6 +250,7 @@

Example 3: Resilient Pipelines} #> iteration 1 successful #> iteration 2 successful +#> Error: random error #> iteration 3 successful #> iteration 4 successful #> iteration 5 successful @@ -298,12 +299,12 @@

With Dispatcher (default)#> #> $daemons #> i online instance assigned complete -#> abstract://a35ccba82d7cec7e897e95a4 1 1 1 0 0 -#> abstract://e258258b1853578cddbecfc6 2 1 1 0 0 -#> abstract://854e181275167b1678b4a7fb 3 1 1 0 0 -#> abstract://40527295f9a12505b3f83b6b 4 1 1 0 0 -#> abstract://c67f46434dc534e30f48019e 5 1 1 0 0 -#> abstract://cf67ef9e1563adc905199a23 6 1 1 0 0

+#> abstract://f52b35b9a832a2152cb68451 1 1 1 0 0 +#> abstract://033141983db5a787d41c6b62 2 1 1 0 0 +#> abstract://f5b1e54c4524b0ad70f97cb1 3 1 1 0 0 +#> abstract://0c2dde598cb4db7faea10fa5 4 1 1 0 0 +#> abstract://4c233fb0f8673194b1f4e043 5 1 1 0 0 +#> abstract://32c59496251de9055106a772 6 1 1 0 0

The default dispatcher = "process" creates a dispatcher() background process that connects to individual daemon processes on the local machine. This ensures that tasks are @@ -341,7 +342,7 @@

Without Dispatcher#> [1] 6 #> #> $daemons -#> [1] "abstract://fe6011ba0b3913df8a01fb18" +#> [1] "abstract://d47e6c20df78276654d7893e"

This implementation sends tasks immediately, and ensures that tasks are evenly-distributed amongst daemons. This means that optimal scheduling is not guaranteed as the duration of tasks cannot be known @@ -374,14 +375,14 @@

Everywhere
 m <- mirai(capture.output(str(con)))
 m[]
-#> [1] "Formal class 'SQLiteConnection' [package \"RSQLite\"] with 8 slots"  
-#> [2] "  ..@ ptr                :<externalptr> "                            
-#> [3] "  ..@ dbname             : chr \"/tmp/RtmpzXpeSH/file34b5234a60eea\""
-#> [4] "  ..@ loadable.extensions: logi TRUE"                                
-#> [5] "  ..@ flags              : int 70"                                   
-#> [6] "  ..@ vfs                : chr \"\""                                 
-#> [7] "  ..@ ref                :<environment: 0x5df816664b50> "            
-#> [8] "  ..@ bigint             : chr \"integer64\""                        
+#> [1] "Formal class 'SQLiteConnection' [package \"RSQLite\"] with 8 slots" 
+#> [2] "  ..@ ptr                :<externalptr> "                           
+#> [3] "  ..@ dbname             : chr \"/tmp/Rtmp8RYD0H/file38fb134e0316\""
+#> [4] "  ..@ loadable.extensions: logi TRUE"                               
+#> [5] "  ..@ flags              : int 70"                                  
+#> [6] "  ..@ vfs                : chr \"\""                                
+#> [7] "  ..@ ref                :<environment: 0x56e93a2ecdf8> "           
+#> [8] "  ..@ bigint             : chr \"integer64\""                       
 #> [9] "  ..@ extended_types     : logi FALSE"

Disconnect from the database everywhere, and set the number of daemons to zero to reset.

@@ -518,7 +519,7 @@

Connecting to Remote Daemons Dire #> [1] 0 #> #> $daemons -#> [1] "tcp://hostname:43129" +#> [1] "tcp://hostname:42771"

The number of daemons connecting to the host URL is not limited and network resources may be added or removed at any time, with tasks automatically distributed to all connected daemons.

@@ -600,17 +601,18 @@

Cluster Resource Managers

remote_config() may be used to run a command to deploy daemons using a resource manager.

-

Taking Slurm as an example, the following uses srun to +

Taking Slurm as an example, the following uses sbatch to launch 2 daemons on the cluster, with some additional arguments to -srun specifying the resource allocation:

+sbatch specifying the resource allocation:

 daemons(
   n = 2,
   url = host_url(ws = TRUE),
   remote = remote_config(
-    command = "srun",
-    args = c("--mem 512", "-n 1", "."),
-    rscript = file.path(R.home("bin"), "Rscript")
+    command = "sbatch",
+    args = c("--mem 512", "-n 1", "--wrap", "."),
+    rscript = file.path(R.home("bin"), "Rscript"),
+    quote = TRUE
   )
 )
@@ -626,10 +628,10 @@

Manual Deployment#> [1] 2 launch_remote(1:2) #> [1] -#> Rscript -e 'mirai::daemon("tcp://hostname:42325",rs=c(10407,-1828318451,501344874,1362761955,397178056,735030313,1295249878))' +#> Rscript -e 'mirai::daemon("tcp://hostname:45023",rs=c(10407,-1977357818,1537914063,-944747324,685206773,301969714,384986891))' #> #> [2] -#> Rscript -e 'mirai::daemon("tcp://hostname:32991",rs=c(10407,1324936348,1629295047,-90918194,1459337963,994896166,-326446377))' +#> Rscript -e 'mirai::daemon("tcp://hostname:35177",rs=c(10407,-2118558994,-1471211395,-214113251,1708896166,-1342731093,-35397167))' daemons(0) #> [1] 0

Note that daemons() should be set up on the host machine @@ -666,37 +668,37 @@

Zero-configuration
 launch_remote(1)
 #> [1]
-#> Rscript -e 'mirai::daemon("wss://hostname:45635/1",tls=c("-----BEGIN CERTIFICATE-----
+#> Rscript -e 'mirai::daemon("wss://hostname:33667/1",tls=c("-----BEGIN CERTIFICATE-----
 #> MIIFNzCCAx+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQDDAhrdW1h
 #> bW90bzERMA8GA1UECgwITmFub25leHQxCzAJBgNVBAYTAkpQMB4XDTAxMDEwMTAw
 #> MDAwMFoXDTMwMTIzMTIzNTk1OVowMzERMA8GA1UEAwwIa3VtYW1vdG8xETAPBgNV
 #> BAoMCE5hbm9uZXh0MQswCQYDVQQGEwJKUDCCAiIwDQYJKoZIhvcNAQEBBQADggIP
-#> ADCCAgoCggIBANw6CDKpRtGQg8fAUO5o04BdaH9kfWWpqniBpD0UMNGUJu93NtYd
-#> TSdPhmB8pbJB4MtdVHFgB4kaFMQZ27N9bu9Bc7vXnzfs7Kh0felYQbjforNePWDz
-#> 9mxcy0ASNxn6iLUMsVXd9ZiyfKkUjkPirGaqfJ5sQE40yt49voeD6H5RMJTf5jts
-#> 15Pyo57XnWIh2V60yQ+VGuiqbVfAAY6cA812KQnvxr3MmQxJ8W+C8kFn8x9WK9zI
-#> 43rx9oQEVMVeJ2Bz4WKFuTohX0+PuGDJgMBQS4vns9uGmSQnuwzEVzq6KxtdE5fF
-#> S52DT0UAIWTjdO9ZDFa2aa3AMxpHfzGkl5YjKgP3cTbSaBKPpJaxY06r6DuX/UXN
-#> YUzZ6qL0+mmNqjBNI5+2fnBY5G18MXQC9hzXmosZMxJgiJSaRPz6lw9Q9oesDTXR
-#> tnCLbsp/F7ELxUqAWtxo+AIhE+4BZHVWRCOMif+kbvhm3iimHOtcErCa5Q/QtFS9
-#> VYYKT2wsfxL8WP5n5nWPDq9qfy9WSo6kiFnUDJhilHBMDO4yVrocE4j7fEatckqu
-#> nXlPwRAMb8vwxHVJiQbTIpIQVBJOU0tUxHSH9KZGe5UBnV+isJJEsRtj4WPvTsx9
-#> /tK7O61Z5fCNIEeULdXRgSLB2iTeRgBYA6QNqenDz17HRA6zmX4S9HqHAgMBAAGj
-#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFH05owUC7+3jOv7Sh3WF
-#> Hn/TwV6KMB8GA1UdIwQYMBaAFH05owUC7+3jOv7Sh3WFHn/TwV6KMA0GCSqGSIb3
-#> DQEBCwUAA4ICAQAzCWeL/W0HC9vZrO39FFyITkgY5TicGyaP51Y6VKxAlJX1PRli
-#> 4vjetgajqmsRyNqZ+YByWvc+TUce4Mo76pYgtwlPiVhbn3kfcxSKA9mHGCNIcxpj
-#> BO0tJlLQnNjLCtZNze4WMnNynE6H7ePki2DrCECGeJTtXfiyHnnFgu+gRFhRGlTe
-#> ZCTrzjm57F3ka+UIdGZUXuzWkSQgNwRBZg1279B1yqNQahBie+QTh7hEv55gPiiu
-#> 77snyPJGsumZ0QaOPj4TyPsCfHYys5DSd7Xs87a13PzPEfiYF2iWz6M69aiSKj2s
-#> 2Cd2R2kvUqjs/krc464QuRZBNaNhnyR3uOzt5wcuZmJC67AeVlU77IQkvTy2xlQF
-#> UnxSpGNxGSu1JruOjBeirSx7pM3Qlv6YROYwm/ru01xdxcTiN7+3ytn46i5puWAe
-#> gUYOIL1MoPwhiYsiCNcjiw50PUpIAE/aZ49UNg857CC8EEs5EvnbHgx0oJgws1Xx
-#> 4vz2CjaI2e4KzLKorN0WVTIBWWrjzD4hqbas2PxWD8sXqknsi8n/YChEXqa3xuzn
-#> TTJxcbG9qViOXFaND2Nu7wpYdbAOwAAuryQ0ic+1nYe5tibYUPtD43MSs7+nlhN3
-#> s0n8cCVrhYG+CJekq/2gRfzP48xmz1pBb3gJpvcHZika8FGPv9+IYsDX8w==
+#> ADCCAgoCggIBAMwra+feTABrzKCo8WrHmZg/rVUSz1oGNS8V1xNyrGMtdks7+PSu
+#> Wo8Zl4BYBUHKEWKMePsa6Xs5fjmxxoBj4GMbGP7y+YLWmZ5GEkjKCj4XqSX3vcgM
+#> Iw5iO+KsPnZAFfvYmXt1vXDwklcoHiFYO3A1YKRNrKP0RpIHCnsn2aLlinBw5bpf
+#> ZbnaJUcLvg3YM+K5KYvH5JTpE5dnrFMD00por0OGwhT/nUGng5ZAj9Cc/dzFjkDd
+#> s+UrHxwFbUfjaeiw4VV3vHr6mJEDWNYCdBrBr0q6kaiZUOJN+5DcUriXx059WfEw
+#> oDtJgY+9urVpCMsLlKWg4A14jnAVDGLexp/LxxsgEHIfG+Co95QBKCFLslQgS9PO
+#> HVE9bpEtXOOwAHnYzImfu3lR4uroiMWDSN/9XkLG01iIRYeAgSch7k2jDTvR/L6v
+#> AywWigfifHX3qGZrqr9uhpPukhXQ2lhN4PG+vLkmowovscIY0l7vDXOsVW+B2APH
+#> nZUjmd1tyKI7vvqzMb4gqjLIcETfV8a32ATB9zihRnwq4dzv4PTbv8X3ChYAaVvQ
+#> PL3CKXw4FIQGctwbTiSopsp5CfoJ9Dy6vregoe4+33KsMwYV3nGATpGZ+YMb4uf2
+#> xEqN/0jrSJmEQqrIfhUxjwBQDuKbEsnmE/jQJTbJTdPneMhkf4wcOBerAgMBAAGj
+#> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHduHiXzvsvbh82mVQTA
+#> m3mgEwAsMB8GA1UdIwQYMBaAFHduHiXzvsvbh82mVQTAm3mgEwAsMA0GCSqGSIb3
+#> DQEBCwUAA4ICAQA9ONCt0Ms6T9pz8jOiS++aEfYd9rOxAPHjqV0WtEIt5XqgbLmy
+#> UBqEho3gW6gPXT99AOzAAF93QHIoSHeaLLskbRsO5AtvhaFUx0D9tkcZCk46uT4i
+#> E4vk+H64r/EAaedkowA7c54vwHP+0XnDp8FKWDQaj9UWFC7WPnoYiAqu274YHpP2
+#> HDneTf2qb0+ALI52/8EL7YKbsAOpi42x35die9qS2zYbA8CNThDvWsMFCviC8bgM
+#> YAnH/5bqrzh9VyFn6EasDk7NXt8Gyadn7avZRWDjbKvEwrBbvmoWUJh0YBShMZSy
+#> I+88xHid9WJWJYGNL7fBvVxzfnew8cVU0D20YWK2WN/JMrNKg570fhIGSXW2rNRw
+#> x/hZ7a3JdADmUSMV8YRZmLIx48P/X7CgEyhZZVWWxPez0vz6srOOxLvxTod84CDB
+#> ApyWcm5GylpkRtMZdX4fkvZtc22Hi21cjZTjL1OU8olRgjycRuxqJI+zBMlepUk+
+#> jYGcMp4mczoK+KCqZI9BR5M4hs4nurE2QDaeEW8gC6tM4KlP5O+dI+CjXb8QV6IZ
+#> VgfxXVSch+9LpCynaQf+JdomygH35d+XYsps7KjkASJeig9poT7QdDqpWSAyID21
+#> RAWLnGaioVrEqp5SDiVEGyh0bX4fmSFWXv1TorfLDNbB1GAm6pGXn8yLjQ==
 #> -----END CERTIFICATE-----
-#> ",""),rs=c(10407,1672045068,-894820451,225659322,-372605965,-77358952,-166139463))'
+#> ",""),rs=c(10407,-190365515,-1460862478,1356191947,1879828880,1039554641,2031211998))'

The printed value may be deployed directly on a remote machine.

« Back to ToC

@@ -1005,10 +1007,10 @@

Key advantages:vec <- c(1, 1, 4, 4, 1, 1, 1, 1) system.time(mirai_map(vec, Sys.sleep)[]) #> user system elapsed -#> 0.004 0.004 4.007 +#> 0.003 0.003 4.007 system.time(parLapply(cl, vec, Sys.sleep)) #> user system elapsed -#> 0.011 0.003 8.011 +#> 0.010 0.004 8.013

.args is used to specify further constant arguments to .f - the ‘mean’ and ‘sd’ in the example below:

@@ -1017,13 +1019,13 @@ 
Key advantages:  mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[]
 )
 #> [[1]]
-#> [1] 21.78985
+#> [1] 20.27145
 #> 
 #> [[2]]
-#> [1] 20.14001 17.18939
+#> [1] 20.55287 22.25358
 #> 
 #> [[3]]
-#> [1] 20.35270 20.68685 22.75893
+#> [1] 20.18246 20.24844 19.74873

Use ... to further specify objects referenced but not defined in .f - the ‘do’ in the anonymous function below:

@@ -1035,16 +1037,16 @@

Key advantages:) #> Warning: mirai is launching one local daemon for a map operation as none previously set ml -#> < mirai map [2/3] > +#> < mirai map [3/3] > ml[] #> $a -#> [1] "23" +#> [1] "c8" #> #> $b -#> [1] 27 32 +#> [1] 96 66 #> #> $c -#> [1] "2dd1bf" +#> [1] "6b20bb"

Use of mirai_map() assumes that daemons() have previously been set. If not then one (non-dispatcher) daemon is set to allow the function to proceed. This ensures safe behaviour, but is diff --git a/pkgdown.yml b/pkgdown.yml index b2052d2b2..b75bd5c2d 100644 --- a/pkgdown.yml +++ b/pkgdown.yml @@ -9,7 +9,7 @@ articles: promises: promises.html shiny: shiny.html torch: torch.html -last_built: 2024-10-30T14:45Z +last_built: 2024-10-30T14:52Z urls: reference: https://shikokuchuo.net/mirai/reference article: https://shikokuchuo.net/mirai/articles diff --git a/reference/host_url.html b/reference/host_url.html index 8bdafaddd..e171d3b95 100644 --- a/reference/host_url.html +++ b/reference/host_url.html @@ -109,16 +109,16 @@

Details

Examples

host_url()
-#> [1] "tcp://fv-az569-105:0"
+#> [1] "tcp://fv-az661-991:0"
 host_url(ws = TRUE)
-#> [1] "ws://fv-az569-105:0"
+#> [1] "ws://fv-az661-991:0"
 host_url(tls = TRUE)
-#> [1] "tls+tcp://fv-az569-105:0"
+#> [1] "tls+tcp://fv-az661-991:0"
 host_url(ws = TRUE, tls = TRUE, port = 5555)
-#> [1] "wss://fv-az569-105:5555"
+#> [1] "wss://fv-az661-991:5555"
 
 local_url()
-#> [1] "abstract://62806b3c386ff17f3d95d156"
+#> [1] "abstract://b303001ab17932607922a093"
diff --git a/reference/remote_config.html b/reference/remote_config.html index e6a2b36b5..fc5de4138 100644 --- a/reference/remote_config.html +++ b/reference/remote_config.html @@ -111,7 +111,7 @@

Argumentsquote

[default FALSE] logical value whether or not to quote the daemon launch command (not required for Slurm ‘srun’ for example, but -required for ‘ssh’).

+required for ‘ssh’ or Slurm ‘sbatch’).

remotes
diff --git a/reference/serial_config.html b/reference/serial_config.html index 3cd660b81..f02cbf336 100644 --- a/reference/serial_config.html +++ b/reference/serial_config.html @@ -118,7 +118,7 @@

Examples#> [[2]] #> function (x) #> serialize(x, NULL) -#> <environment: 0x558a392626c8> +#> <environment: 0x560b05fed078> #> #> [[3]] #> function (connection, refhook = NULL) @@ -128,7 +128,7 @@

Examples#> stop("'connection' must be a connection") #> .Internal(unserialize(connection, refhook)) #> } -#> <bytecode: 0x558a392718f0> +#> <bytecode: 0x560b05ff8ef0> #> <environment: namespace:base> #> #> [[4]] diff --git a/search.json b/search.json index 94ce2e7b7..f82d2e5ea 100644 --- a/search.json +++ b/search.json @@ -1 +1 @@ -[{"path":[]},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"our-pledge","dir":"","previous_headings":"","what":"Our Pledge","title":"Contributor Covenant Code of Conduct","text":"members, contributors, leaders pledge make participation community harassment-free experience everyone, regardless age, body size, visible invisible disability, ethnicity, sex characteristics, gender identity expression, level experience, education, socio-economic status, nationality, personal appearance, race, religion, sexual identity orientation. pledge act interact ways contribute open, welcoming, diverse, inclusive, healthy community.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"our-standards","dir":"","previous_headings":"","what":"Our Standards","title":"Contributor Covenant Code of Conduct","text":"Examples behavior contributes positive environment community include: Demonstrating empathy kindness toward people respectful differing opinions, viewpoints, experiences Giving gracefully accepting constructive feedback Accepting responsibility apologizing affected mistakes, learning experience Focusing best just us individuals, overall community Examples unacceptable behavior include: use sexualized language imagery, sexual attention advances kind Trolling, insulting derogatory comments, personal political attacks Public private harassment Publishing others’ private information, physical email address, without explicit permission conduct reasonably considered inappropriate professional setting","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"enforcement-responsibilities","dir":"","previous_headings":"","what":"Enforcement Responsibilities","title":"Contributor Covenant Code of Conduct","text":"Community leaders responsible clarifying enforcing standards acceptable behavior take appropriate fair corrective action response behavior deem inappropriate, threatening, offensive, harmful. Community leaders right responsibility remove, edit, reject comments, commits, code, wiki edits, issues, contributions aligned Code Conduct, communicate reasons moderation decisions appropriate.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"scope","dir":"","previous_headings":"","what":"Scope","title":"Contributor Covenant Code of Conduct","text":"Code Conduct applies within community spaces, also applies individual officially representing community public spaces. Examples representing community include using official e-mail address, posting via official social media account, acting appointed representative online offline event.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"enforcement","dir":"","previous_headings":"","what":"Enforcement","title":"Contributor Covenant Code of Conduct","text":"Instances abusive, harassing, otherwise unacceptable behavior may reported community leaders responsible enforcement code@hibiki-ai.com. complaints reviewed investigated promptly fairly. community leaders obligated respect privacy security reporter incident.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"enforcement-guidelines","dir":"","previous_headings":"","what":"Enforcement Guidelines","title":"Contributor Covenant Code of Conduct","text":"Community leaders follow Community Impact Guidelines determining consequences action deem violation Code Conduct:","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_1-correction","dir":"","previous_headings":"Enforcement Guidelines","what":"1. Correction","title":"Contributor Covenant Code of Conduct","text":"Community Impact: Use inappropriate language behavior deemed unprofessional unwelcome community. Consequence: private, written warning community leaders, providing clarity around nature violation explanation behavior inappropriate. public apology may requested.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_2-warning","dir":"","previous_headings":"Enforcement Guidelines","what":"2. Warning","title":"Contributor Covenant Code of Conduct","text":"Community Impact: violation single incident series actions. Consequence: warning consequences continued behavior. interaction people involved, including unsolicited interaction enforcing Code Conduct, specified period time. includes avoiding interactions community spaces well external channels like social media. Violating terms may lead temporary permanent ban.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_3-temporary-ban","dir":"","previous_headings":"Enforcement Guidelines","what":"3. Temporary Ban","title":"Contributor Covenant Code of Conduct","text":"Community Impact: serious violation community standards, including sustained inappropriate behavior. Consequence: temporary ban sort interaction public communication community specified period time. public private interaction people involved, including unsolicited interaction enforcing Code Conduct, allowed period. Violating terms may lead permanent ban.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_4-permanent-ban","dir":"","previous_headings":"Enforcement Guidelines","what":"4. Permanent Ban","title":"Contributor Covenant Code of Conduct","text":"Community Impact: Demonstrating pattern violation community standards, including sustained inappropriate behavior, harassment individual, aggression toward disparagement classes individuals. Consequence: permanent ban sort public interaction within community.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"attribution","dir":"","previous_headings":"","what":"Attribution","title":"Contributor Covenant Code of Conduct","text":"Code Conduct adapted Contributor Covenant, version 2.0, available https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. Community Impact Guidelines inspired Mozilla’s code conduct enforcement ladder. answers common questions code conduct, see FAQ https://www.contributor-covenant.org/faq. Translations available https://www.contributor-covenant.org/translations.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"GNU General Public License","title":"GNU General Public License","text":"Version 3, 29 June 2007Copyright © 2007 Free Software Foundation, Inc.  Everyone permitted copy distribute verbatim copies license document, changing allowed.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"preamble","dir":"","previous_headings":"","what":"Preamble","title":"GNU General Public License","text":"GNU General Public License free, copyleft license software kinds works. licenses software practical works designed take away freedom share change works. contrast, GNU General Public License intended guarantee freedom share change versions program–make sure remains free software users. , Free Software Foundation, use GNU General Public License software; applies also work released way authors. can apply programs, . speak free software, referring freedom, price. General Public Licenses designed make sure freedom distribute copies free software (charge wish), receive source code can get want , can change software use pieces new free programs, know can things. protect rights, need prevent others denying rights asking surrender rights. Therefore, certain responsibilities distribute copies software, modify : responsibilities respect freedom others. example, distribute copies program, whether gratis fee, must pass recipients freedoms received. must make sure , , receive can get source code. must show terms know rights. Developers use GNU GPL protect rights two steps: (1) assert copyright software, (2) offer License giving legal permission copy, distribute /modify . developers’ authors’ protection, GPL clearly explains warranty free software. users’ authors’ sake, GPL requires modified versions marked changed, problems attributed erroneously authors previous versions. devices designed deny users access install run modified versions software inside , although manufacturer can . fundamentally incompatible aim protecting users’ freedom change software. systematic pattern abuse occurs area products individuals use, precisely unacceptable. Therefore, designed version GPL prohibit practice products. problems arise substantially domains, stand ready extend provision domains future versions GPL, needed protect freedom users. Finally, every program threatened constantly software patents. States allow patents restrict development use software general-purpose computers, , wish avoid special danger patents applied free program make effectively proprietary. prevent , GPL assures patents used render program non-free. precise terms conditions copying, distribution modification follow.","code":""},{"path":[]},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_0-definitions","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"0. Definitions","title":"GNU General Public License","text":"“License” refers version 3 GNU General Public License. “Copyright” also means copyright-like laws apply kinds works, semiconductor masks. “Program” refers copyrightable work licensed License. licensee addressed “”. “Licensees” “recipients” may individuals organizations. “modify” work means copy adapt part work fashion requiring copyright permission, making exact copy. resulting work called “modified version” earlier work work “based ” earlier work. “covered work” means either unmodified Program work based Program. “propagate” work means anything , without permission, make directly secondarily liable infringement applicable copyright law, except executing computer modifying private copy. Propagation includes copying, distribution (without modification), making available public, countries activities well. “convey” work means kind propagation enables parties make receive copies. Mere interaction user computer network, transfer copy, conveying. interactive user interface displays “Appropriate Legal Notices” extent includes convenient prominently visible feature (1) displays appropriate copyright notice, (2) tells user warranty work (except extent warranties provided), licensees may convey work License, view copy License. interface presents list user commands options, menu, prominent item list meets criterion.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_1-source-code","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"1. Source Code","title":"GNU General Public License","text":"“source code” work means preferred form work making modifications . “Object code” means non-source form work. “Standard Interface” means interface either official standard defined recognized standards body, , case interfaces specified particular programming language, one widely used among developers working language. “System Libraries” executable work include anything, work whole, () included normal form packaging Major Component, part Major Component, (b) serves enable use work Major Component, implement Standard Interface implementation available public source code form. “Major Component”, context, means major essential component (kernel, window system, ) specific operating system () executable work runs, compiler used produce work, object code interpreter used run . “Corresponding Source” work object code form means source code needed generate, install, (executable work) run object code modify work, including scripts control activities. However, include work’s System Libraries, general-purpose tools generally available free programs used unmodified performing activities part work. example, Corresponding Source includes interface definition files associated source files work, source code shared libraries dynamically linked subprograms work specifically designed require, intimate data communication control flow subprograms parts work. Corresponding Source need include anything users can regenerate automatically parts Corresponding Source. Corresponding Source work source code form work.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_2-basic-permissions","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"2. Basic Permissions","title":"GNU General Public License","text":"rights granted License granted term copyright Program, irrevocable provided stated conditions met. License explicitly affirms unlimited permission run unmodified Program. output running covered work covered License output, given content, constitutes covered work. License acknowledges rights fair use equivalent, provided copyright law. may make, run propagate covered works convey, without conditions long license otherwise remains force. may convey covered works others sole purpose make modifications exclusively , provide facilities running works, provided comply terms License conveying material control copyright. thus making running covered works must exclusively behalf, direction control, terms prohibit making copies copyrighted material outside relationship . Conveying circumstances permitted solely conditions stated . Sublicensing allowed; section 10 makes unnecessary.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_3-protecting-users-legal-rights-from-anti-circumvention-law","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"3. Protecting Users’ Legal Rights From Anti-Circumvention Law","title":"GNU General Public License","text":"covered work shall deemed part effective technological measure applicable law fulfilling obligations article 11 WIPO copyright treaty adopted 20 December 1996, similar laws prohibiting restricting circumvention measures. convey covered work, waive legal power forbid circumvention technological measures extent circumvention effected exercising rights License respect covered work, disclaim intention limit operation modification work means enforcing, work’s users, third parties’ legal rights forbid circumvention technological measures.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_4-conveying-verbatim-copies","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"4. Conveying Verbatim Copies","title":"GNU General Public License","text":"may convey verbatim copies Program’s source code receive , medium, provided conspicuously appropriately publish copy appropriate copyright notice; keep intact notices stating License non-permissive terms added accord section 7 apply code; keep intact notices absence warranty; give recipients copy License along Program. may charge price price copy convey, may offer support warranty protection fee.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_5-conveying-modified-source-versions","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"5. Conveying Modified Source Versions","title":"GNU General Public License","text":"may convey work based Program, modifications produce Program, form source code terms section 4, provided also meet conditions: ) work must carry prominent notices stating modified , giving relevant date. b) work must carry prominent notices stating released License conditions added section 7. requirement modifies requirement section 4 “keep intact notices”. c) must license entire work, whole, License anyone comes possession copy. License therefore apply, along applicable section 7 additional terms, whole work, parts, regardless packaged. License gives permission license work way, invalidate permission separately received . d) work interactive user interfaces, must display Appropriate Legal Notices; however, Program interactive interfaces display Appropriate Legal Notices, work need make . compilation covered work separate independent works, nature extensions covered work, combined form larger program, volume storage distribution medium, called “aggregate” compilation resulting copyright used limit access legal rights compilation’s users beyond individual works permit. Inclusion covered work aggregate cause License apply parts aggregate.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_6-conveying-non-source-forms","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"6. Conveying Non-Source Forms","title":"GNU General Public License","text":"may convey covered work object code form terms sections 4 5, provided also convey machine-readable Corresponding Source terms License, one ways: ) Convey object code , embodied , physical product (including physical distribution medium), accompanied Corresponding Source fixed durable physical medium customarily used software interchange. b) Convey object code , embodied , physical product (including physical distribution medium), accompanied written offer, valid least three years valid long offer spare parts customer support product model, give anyone possesses object code either (1) copy Corresponding Source software product covered License, durable physical medium customarily used software interchange, price reasonable cost physically performing conveying source, (2) access copy Corresponding Source network server charge. c) Convey individual copies object code copy written offer provide Corresponding Source. alternative allowed occasionally noncommercially, received object code offer, accord subsection 6b. d) Convey object code offering access designated place (gratis charge), offer equivalent access Corresponding Source way place charge. need require recipients copy Corresponding Source along object code. place copy object code network server, Corresponding Source may different server (operated third party) supports equivalent copying facilities, provided maintain clear directions next object code saying find Corresponding Source. Regardless server hosts Corresponding Source, remain obligated ensure available long needed satisfy requirements. e) Convey object code using peer--peer transmission, provided inform peers object code Corresponding Source work offered general public charge subsection 6d. separable portion object code, whose source code excluded Corresponding Source System Library, need included conveying object code work. “User Product” either (1) “consumer product”, means tangible personal property normally used personal, family, household purposes, (2) anything designed sold incorporation dwelling. determining whether product consumer product, doubtful cases shall resolved favor coverage. particular product received particular user, “normally used” refers typical common use class product, regardless status particular user way particular user actually uses, expects expected use, product. product consumer product regardless whether product substantial commercial, industrial non-consumer uses, unless uses represent significant mode use product. “Installation Information” User Product means methods, procedures, authorization keys, information required install execute modified versions covered work User Product modified version Corresponding Source. information must suffice ensure continued functioning modified object code case prevented interfered solely modification made. convey object code work section , , specifically use , User Product, conveying occurs part transaction right possession use User Product transferred recipient perpetuity fixed term (regardless transaction characterized), Corresponding Source conveyed section must accompanied Installation Information. requirement apply neither third party retains ability install modified object code User Product (example, work installed ROM). requirement provide Installation Information include requirement continue provide support service, warranty, updates work modified installed recipient, User Product modified installed. Access network may denied modification materially adversely affects operation network violates rules protocols communication across network. Corresponding Source conveyed, Installation Information provided, accord section must format publicly documented (implementation available public source code form), must require special password key unpacking, reading copying.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_7-additional-terms","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"7. Additional Terms","title":"GNU General Public License","text":"“Additional permissions” terms supplement terms License making exceptions one conditions. Additional permissions applicable entire Program shall treated though included License, extent valid applicable law. additional permissions apply part Program, part may used separately permissions, entire Program remains governed License without regard additional permissions. convey copy covered work, may option remove additional permissions copy, part . (Additional permissions may written require removal certain cases modify work.) may place additional permissions material, added covered work, can give appropriate copyright permission. Notwithstanding provision License, material add covered work, may (authorized copyright holders material) supplement terms License terms: ) Disclaiming warranty limiting liability differently terms sections 15 16 License; b) Requiring preservation specified reasonable legal notices author attributions material Appropriate Legal Notices displayed works containing ; c) Prohibiting misrepresentation origin material, requiring modified versions material marked reasonable ways different original version; d) Limiting use publicity purposes names licensors authors material; e) Declining grant rights trademark law use trade names, trademarks, service marks; f) Requiring indemnification licensors authors material anyone conveys material (modified versions ) contractual assumptions liability recipient, liability contractual assumptions directly impose licensors authors. non-permissive additional terms considered “restrictions” within meaning section 10. Program received , part , contains notice stating governed License along term restriction, may remove term. license document contains restriction permits relicensing conveying License, may add covered work material governed terms license document, provided restriction survive relicensing conveying. add terms covered work accord section, must place, relevant source files, statement additional terms apply files, notice indicating find applicable terms. Additional terms, permissive non-permissive, may stated form separately written license, stated exceptions; requirements apply either way.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_8-termination","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"8. Termination","title":"GNU General Public License","text":"may propagate modify covered work except expressly provided License. attempt otherwise propagate modify void, automatically terminate rights License (including patent licenses granted third paragraph section 11). However, cease violation License, license particular copyright holder reinstated () provisionally, unless copyright holder explicitly finally terminates license, (b) permanently, copyright holder fails notify violation reasonable means prior 60 days cessation. Moreover, license particular copyright holder reinstated permanently copyright holder notifies violation reasonable means, first time received notice violation License (work) copyright holder, cure violation prior 30 days receipt notice. Termination rights section terminate licenses parties received copies rights License. rights terminated permanently reinstated, qualify receive new licenses material section 10.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_9-acceptance-not-required-for-having-copies","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"9. Acceptance Not Required for Having Copies","title":"GNU General Public License","text":"required accept License order receive run copy Program. Ancillary propagation covered work occurring solely consequence using peer--peer transmission receive copy likewise require acceptance. However, nothing License grants permission propagate modify covered work. actions infringe copyright accept License. Therefore, modifying propagating covered work, indicate acceptance License .","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_10-automatic-licensing-of-downstream-recipients","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"10. Automatic Licensing of Downstream Recipients","title":"GNU General Public License","text":"time convey covered work, recipient automatically receives license original licensors, run, modify propagate work, subject License. responsible enforcing compliance third parties License. “entity transaction” transaction transferring control organization, substantially assets one, subdividing organization, merging organizations. propagation covered work results entity transaction, party transaction receives copy work also receives whatever licenses work party’s predecessor interest give previous paragraph, plus right possession Corresponding Source work predecessor interest, predecessor can get reasonable efforts. may impose restrictions exercise rights granted affirmed License. example, may impose license fee, royalty, charge exercise rights granted License, may initiate litigation (including cross-claim counterclaim lawsuit) alleging patent claim infringed making, using, selling, offering sale, importing Program portion .","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_11-patents","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"11. Patents","title":"GNU General Public License","text":"“contributor” copyright holder authorizes use License Program work Program based. work thus licensed called contributor’s “contributor version”. contributor’s “essential patent claims” patent claims owned controlled contributor, whether already acquired hereafter acquired, infringed manner, permitted License, making, using, selling contributor version, include claims infringed consequence modification contributor version. purposes definition, “control” includes right grant patent sublicenses manner consistent requirements License. contributor grants non-exclusive, worldwide, royalty-free patent license contributor’s essential patent claims, make, use, sell, offer sale, import otherwise run, modify propagate contents contributor version. following three paragraphs, “patent license” express agreement commitment, however denominated, enforce patent (express permission practice patent covenant sue patent infringement). “grant” patent license party means make agreement commitment enforce patent party. convey covered work, knowingly relying patent license, Corresponding Source work available anyone copy, free charge terms License, publicly available network server readily accessible means, must either (1) cause Corresponding Source available, (2) arrange deprive benefit patent license particular work, (3) arrange, manner consistent requirements License, extend patent license downstream recipients. “Knowingly relying” means actual knowledge , patent license, conveying covered work country, recipient’s use covered work country, infringe one identifiable patents country reason believe valid. , pursuant connection single transaction arrangement, convey, propagate procuring conveyance , covered work, grant patent license parties receiving covered work authorizing use, propagate, modify convey specific copy covered work, patent license grant automatically extended recipients covered work works based . patent license “discriminatory” include within scope coverage, prohibits exercise , conditioned non-exercise one rights specifically granted License. may convey covered work party arrangement third party business distributing software, make payment third party based extent activity conveying work, third party grants, parties receive covered work , discriminatory patent license () connection copies covered work conveyed (copies made copies), (b) primarily connection specific products compilations contain covered work, unless entered arrangement, patent license granted, prior 28 March 2007. Nothing License shall construed excluding limiting implied license defenses infringement may otherwise available applicable patent law.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_12-no-surrender-of-others-freedom","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"12. No Surrender of Others’ Freedom","title":"GNU General Public License","text":"conditions imposed (whether court order, agreement otherwise) contradict conditions License, excuse conditions License. convey covered work satisfy simultaneously obligations License pertinent obligations, consequence may convey . example, agree terms obligate collect royalty conveying convey Program, way satisfy terms License refrain entirely conveying Program.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_13-use-with-the-gnu-affero-general-public-license","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"13. Use with the GNU Affero General Public License","title":"GNU General Public License","text":"Notwithstanding provision License, permission link combine covered work work licensed version 3 GNU Affero General Public License single combined work, convey resulting work. terms License continue apply part covered work, special requirements GNU Affero General Public License, section 13, concerning interaction network apply combination .","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_14-revised-versions-of-this-license","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"14. Revised Versions of this License","title":"GNU General Public License","text":"Free Software Foundation may publish revised /new versions GNU General Public License time time. new versions similar spirit present version, may differ detail address new problems concerns. version given distinguishing version number. Program specifies certain numbered version GNU General Public License “later version” applies , option following terms conditions either numbered version later version published Free Software Foundation. Program specify version number GNU General Public License, may choose version ever published Free Software Foundation. Program specifies proxy can decide future versions GNU General Public License can used, proxy’s public statement acceptance version permanently authorizes choose version Program. Later license versions may give additional different permissions. However, additional obligations imposed author copyright holder result choosing follow later version.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_15-disclaimer-of-warranty","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"15. Disclaimer of Warranty","title":"GNU General Public License","text":"WARRANTY PROGRAM, EXTENT PERMITTED APPLICABLE LAW. EXCEPT OTHERWISE STATED WRITING COPYRIGHT HOLDERS /PARTIES PROVIDE PROGRAM “” WITHOUT WARRANTY KIND, EITHER EXPRESSED IMPLIED, INCLUDING, LIMITED , IMPLIED WARRANTIES MERCHANTABILITY FITNESS PARTICULAR PURPOSE. ENTIRE RISK QUALITY PERFORMANCE PROGRAM . PROGRAM PROVE DEFECTIVE, ASSUME COST NECESSARY SERVICING, REPAIR CORRECTION.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_16-limitation-of-liability","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"16. Limitation of Liability","title":"GNU General Public License","text":"EVENT UNLESS REQUIRED APPLICABLE LAW AGREED WRITING COPYRIGHT HOLDER, PARTY MODIFIES /CONVEYS PROGRAM PERMITTED , LIABLE DAMAGES, INCLUDING GENERAL, SPECIAL, INCIDENTAL CONSEQUENTIAL DAMAGES ARISING USE INABILITY USE PROGRAM (INCLUDING LIMITED LOSS DATA DATA RENDERED INACCURATE LOSSES SUSTAINED THIRD PARTIES FAILURE PROGRAM OPERATE PROGRAMS), EVEN HOLDER PARTY ADVISED POSSIBILITY DAMAGES.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_17-interpretation-of-sections-15-and-16","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"17. Interpretation of Sections 15 and 16","title":"GNU General Public License","text":"disclaimer warranty limitation liability provided given local legal effect according terms, reviewing courts shall apply local law closely approximates absolute waiver civil liability connection Program, unless warranty assumption liability accompanies copy Program return fee. END TERMS CONDITIONS","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"how-to-apply-these-terms-to-your-new-programs","dir":"","previous_headings":"","what":"How to Apply These Terms to Your New Programs","title":"GNU General Public License","text":"develop new program, want greatest possible use public, best way achieve make free software everyone can redistribute change terms. , attach following notices program. safest attach start source file effectively state exclusion warranty; file least “copyright” line pointer full notice found. Also add information contact electronic paper mail. program terminal interaction, make output short notice like starts interactive mode: hypothetical commands show w show c show appropriate parts General Public License. course, program’s commands might different; GUI interface, use “box”. also get employer (work programmer) school, , sign “copyright disclaimer” program, necessary. information , apply follow GNU GPL, see . GNU General Public License permit incorporating program proprietary programs. program subroutine library, may consider useful permit linking proprietary applications library. want , use GNU Lesser General Public License instead License. first, please read .","code":" Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type 'show w'. This is free software, and you are welcome to redistribute it under certain conditions; type 'show c' for details."},{"path":"https://shikokuchuo.net/mirai/articles/databases.html","id":"database-hosting---the-basics","dir":"Articles","previous_headings":"","what":"Database Hosting - The Basics","title":"mirai - Databases and Arrow","text":"mirai supports hosting multiple database connections across processes local machine remote server. everywhere() easily sets identical database connections daemon process. following represents simple example, sets 2 local daemons, opens connection SQLite file database daemon. mirai() calls may used write query database, may executed either daemon. everywhere() can used cleanly tear databases, resetting daemons.","code":"file <- tempfile() library(mirai) daemons(2) #> [1] 2 everywhere({ library(DBI) con <<- dbConnect(RSQLite::SQLite(), file) }, file = file) m <- mirai(dbWriteTable(con, \"iris\", iris)) m[] #> [1] TRUE m <- mirai(dbListTables(con)) m[] #> [1] \"iris\" m <- mirai(dbGetQuery(con, 'SELECT * FROM iris WHERE \"Sepal.Length\" < 4.6')) m[] #> Sepal.Length Sepal.Width Petal.Length Petal.Width Species #> 1 4.4 2.9 1.4 0.2 setosa #> 2 4.3 3.0 1.1 0.1 setosa #> 3 4.4 3.0 1.3 0.2 setosa #> 4 4.5 2.3 1.3 0.3 setosa #> 5 4.4 3.2 1.3 0.2 setosa everywhere(dbDisconnect(con)) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/databases.html","id":"database-hosting---using-arrow-database-connectivity","dir":"Articles","previous_headings":"","what":"Database Hosting - Using Arrow Database Connectivity","title":"mirai - Databases and Arrow","text":"possible using DBI interface access manipulate data Apache Arrow data format efficiently ABDC (Arrow Database Connectivity). example creates -memory SQLite connection using adbcsqlite backend. Serialization set relevant serialization functions arrow package part everywhere() call. Note specified class ‘nanoarrow_array_stream’ nanoarrow backend queries made DBI db*Arrow() functions. mirai() calls may used write query database Arrow format. Due tight integration mirai serialization mechanism R’s ‘refhook’ system, can easily return complex / nested objects containing multiple queries Arrow format: , everywhere() can used cleanly tear databases, resetting daemons.","code":"library(mirai) daemons(1) #> [1] 1 cfg <- serial_config( class = \"nanoarrow_array_stream\", sfunc = arrow::write_to_raw, ufunc = function(x) arrow::read_ipc_stream(x, as_data_frame = FALSE) ) everywhere( { library(DBI) # `adbi` and `adbcsqlite` packages must also be installed con <<- dbConnect(adbi::adbi(\"adbcsqlite\"), uri = \":memory:\") }, .serial = cfg ) m <- mirai(dbWriteTableArrow(con, \"iris\", iris)) m[] #> [1] TRUE m <- mirai(dbReadTableArrow(con, \"iris\")) m[] #> Table #> 150 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species m <- mirai(dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Sepal.Length\" < 4.6')) m[] #> Table #> 5 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species m <- mirai({ a <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Sepal.Length\" < 4.6') b <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Sepal.Width\" < 2.6') x <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Petal.Length\" < 1.5') y <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Petal.Width\" < 0.2') list(sepal = list(length = a, width = b), petal = list(length = x, width = y)) }) m[] #> $sepal #> $sepal$length #> Table #> 5 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species #> #> $sepal$width #> Table #> 19 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species #> #> #> $petal #> $petal$length #> Table #> 24 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species #> #> $petal$width #> Table #> 5 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species everywhere(dbDisconnect(con)) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/databases.html","id":"shiny-mirai-dbi-adbc-integrated-example","dir":"Articles","previous_headings":"","what":"Shiny / mirai / DBI / ADBC Integrated Example","title":"mirai - Databases and Arrow","text":"following example database connections hosted mirai daemons may used power Shiny app. one-time serialization() setup ensures seamless transport Apache Arrow data, occurs global environment outside server(). new database connection created new daemon process every new Shiny session. resources freed sesssion ends. logic defined within server(). unique ID used identify session, specified ‘compute profile’ daemons. Non-dispatcher daemons created scheduling required (queries expected take roughly time, case session uses one daemon anyway). Shiny ExtendedTask used perform query via mirai() call, using session-specific compute profile.","code":"library(mirai) library(secretbase) library(shiny) library(bslib) # create an Arrow serialization configuration cfg <- serial_config( class = \"nanoarrow_array_stream\", sfunc = arrow::write_to_raw, ufunc = nanoarrow::read_nanoarrow ) # write 'iris' dataset to temp database file (for this demonstration) file <- tempfile() con <- DBI::dbConnect(adbi::adbi(\"adbcsqlite\"), uri = file) DBI::dbWriteTableArrow(con, \"iris\", iris) DBI::dbDisconnect(con) # common input parameters slmin <- min(iris$Sepal.Length) slmax <- max(iris$Sepal.Length) ui <- page_fluid( p(\"The time is \", textOutput(\"current_time\", inline = TRUE)), hr(), h3(\"Shiny / mirai / DBI / ADBC demonstration\"), p(\"New daemon-hosted database connection is created for every Shiny session\"), sliderInput( \"sl\", \"Query iris dataset based on Sepal Length\", min = slmin, max = slmax, value = c(slmin, slmax), width = \"75%\" ), input_task_button(\"btn\", \"Return query\"), tableOutput(\"table\") ) # uses Shiny ExtendedTask with mirai server <- function(input, output, session) { # create unique session id by hashing current time with a random key id <- secretbase::siphash13(Sys.time(), key = nanonext::random(4L)) # create new (non-dispatcher) daemon for each session daemons(1L, dispatcher = \"none\", .compute = id) # tear down daemon when session ends session$onEnded(function() daemons(0L, .compute = id)) # everywhere() loads DBI and creates ADBC connection in each daemon # and sets up serialization everywhere( { library(DBI) # `adbi` and `adbcsqlite` packages must also be installed con <<- dbConnect(adbi::adbi(\"adbcsqlite\"), uri = file) }, file = file, .serial = cfg, .compute = id ) output$current_time <- renderText({ invalidateLater(1000) format(Sys.time(), \"%H:%M:%S %p\") }) task <- ExtendedTask$new( function(...) mirai( dbGetQueryArrow( con, sprintf( \"SELECT * FROM iris WHERE \\\"Sepal.Length\\\" BETWEEN %.2f AND %.2f\", sl[1L], sl[2L] ) ), ..., .compute = id ) ) |> bind_task_button(\"btn\") observeEvent(input$btn, task$invoke(sl = input$sl)) output$table <- renderTable(task$result()) } # run Shiny app shinyApp(ui = ui, server = server) # deletes temp database file (for this demonstration) unlink(file)"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"table-of-contents","dir":"Articles","previous_headings":"","what":"Table of Contents","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Example 1: Compute-intensive Operations Example 2: /O-bound Operations Example 3: Resilient Pipelines Daemons: Local Persistent Processes Distributed Computing: Remote Daemons Distributed Computing: Launching Daemons Distributed Computing: TLS Secure Connections Compute Profiles Errors, Interrupts Timeouts Serialization - Arrow, polars beyond Asynchronous Parallel Map Using mirai Package","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"example-1-compute-intensive-operations","dir":"Articles","previous_headings":"","what":"Example 1: Compute-intensive Operations","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use case: minimise execution times performing long-running tasks concurrently separate processes. Multiple long computes (model fits etc.) can performed parallel available computing cores. Use mirai() evaluate expression asynchronously separate, clean R process. following mimics expensive calculation eventually returns random value. mirai expression evaluated another process hence must self-contained, referring variables already exist . , variables time mean passed part mirai() call. ‘mirai’ object returned immediately - creating mirai never blocks session. Whilst async operation ongoing, attempting access mirai’s data yields ‘unresolved’ logical NA. check whether mirai remains unresolved (yet complete): wait collect return value, use mirai’s [] method: mirai represents async operation, never necessary wait . code can continue run. completes, return value automatically becomes available $data. easy programmatic use mirai(), ‘.expr’ accepts pre-constructed language object, also list named arguments passed via ‘.args’. , following equivalent : « Back ToC","code":"library(mirai) x <- list(time = 2L, mean = 4) m <- mirai({Sys.sleep(time); rnorm(5L, mean)}, time = x$time, mean = x$mean) m #> < mirai [] > m$data #> 'unresolved' logi NA unresolved(m) #> [1] TRUE m[] #> [1] 3.809949 3.009536 3.580907 4.174612 3.275564 m #> < mirai [$data] > m$data #> [1] 3.809949 3.009536 3.580907 4.174612 3.275564 expr <- quote({Sys.sleep(time); rnorm(5L, mean)}) args <- list(time = x$time, mean = x$mean) m <- mirai(.expr = expr, .args = args) m[] #> [1] 2.857742 4.677513 4.285234 3.773817 2.754848"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"example-2-io-bound-operations","dir":"Articles","previous_headings":"","what":"Example 2: I/O-bound Operations","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use case: ensure execution flow main process blocked. High-frequency real-time data written file/database synchronously without disrupting execution flow. Cache data memory use mirai() perform periodic write operations concurrently separate process. , ‘.args’ used pass environment(), calling environment. provides convenient method passing existing objects. ‘mirai’ object returned immediately. unresolved() may used control flow statements perform actions depend resolution ‘mirai’, . means need actually wait (block) ‘mirai’ resolve, example demonstrates. Now actions depend resolution may processed, example next write. « Back ToC","code":"library(mirai) x <- rnorm(1e6) file <- tempfile() m <- mirai(write.csv(x, file = file), .args = environment()) # unresolved() queries for resolution itself so no need to use it again within the while loop while (unresolved(m)) { cat(\"while unresolved\\n\") Sys.sleep(0.5) } #> while unresolved #> while unresolved cat(\"Write complete:\", is.null(m$data)) #> Write complete: TRUE"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"example-3-resilient-pipelines","dir":"Articles","previous_headings":"","what":"Example 3: Resilient Pipelines","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use case: isolating code can potentially fail separate process ensure continued uptime. part data science / machine learning pipeline, iterations model training may periodically fail stochastic uncontrollable reasons (e.g. buggy memory management graphics cards). Running iteration ‘mirai’ isolates potentially-problematic code even fail, bring entire pipeline. , testing return value ‘mirai’ errors, error-handling code able automate recovery re-attempts, example. details error handling can found section . end result resilient fault-tolerant pipeline minimises downtime eliminating interruptions long computes. « Back ToC","code":"library(mirai) run_iteration <- function(i) { if (runif(1) < 0.1) stop(\"random error\\n\", call. = FALSE) # simulates a stochastic error rate sprintf(\"iteration %d successful\\n\", i) } for (i in 1:10) { m <- mirai(run_iteration(i), environment()) while (is_error_value(call_mirai(m)$data)) { cat(m$data) m <- mirai(run_iteration(i), environment()) } cat(m$data) } #> iteration 1 successful #> iteration 2 successful #> iteration 3 successful #> iteration 4 successful #> iteration 5 successful #> iteration 6 successful #> iteration 7 successful #> iteration 8 successful #> iteration 9 successful #> Error: random error #> iteration 10 successful"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"daemons-local-persistent-processes","dir":"Articles","previous_headings":"","what":"Daemons: Local Persistent Processes","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Daemons, persistent background processes, may set receive ‘mirai’ requests. potentially efficient new processes longer need created ad hoc basis. Daemons inherit default system configuration read relevant ‘.Renviron’ ‘.Rprofile’ etc. startup. also load default packages. instead load base package (cuts half R’s startup time), environment variable R_SCRIPT_DEFAULT_PACKAGES=NULL may set prior launching daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"with-dispatcher-default","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"With Dispatcher (default)","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Call daemons() specifying number daemons launch. view current status, status() provides number active connections along matrix statistics daemon. default dispatcher = \"process\" creates dispatcher() background process connects individual daemon processes local machine. ensures tasks dispatched efficiently first-first-(FIFO) basis daemons processing. Tasks queued dispatcher sent daemon soon can accept task immediate execution. Dispatcher uses synchronisation primitives nanonext, waiting upon rather polling tasks, efficient terms consuming resources waiting, also fully synchronised events (latency). Alternatively, specifying dispatcher = \"thread\" runs dispatcher logic thread, faster efficient alternative separate background process. new feature considered experimental. Set number daemons zero reset. reverts default creating new background process ‘mirai’ request.","code":"daemons(6) #> [1] 6 status() #> $connections #> [1] 1 #> #> $daemons #> i online instance assigned complete #> abstract://a35ccba82d7cec7e897e95a4 1 1 1 0 0 #> abstract://e258258b1853578cddbecfc6 2 1 1 0 0 #> abstract://854e181275167b1678b4a7fb 3 1 1 0 0 #> abstract://40527295f9a12505b3f83b6b 4 1 1 0 0 #> abstract://c67f46434dc534e30f48019e 5 1 1 0 0 #> abstract://cf67ef9e1563adc905199a23 6 1 1 0 0 daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"without-dispatcher","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"Without Dispatcher","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Alternatively, specifying dispatcher = 'none, background daemons connect directly host process. Requesting status now shows 6 connections, along host URL $daemons. implementation sends tasks immediately, ensures tasks evenly-distributed amongst daemons. means optimal scheduling guaranteed duration tasks known priori. example, tasks queued daemon behind long-running task, whilst daemons idle already completed tasks. advantage approach low-level require additional dispatcher process. well-suited working similar-length tasks, number concurrent tasks typically exceed available daemons.","code":"daemons(6, dispatcher = \"none\") #> [1] 6 status() #> $connections #> [1] 6 #> #> $daemons #> [1] \"abstract://fe6011ba0b3913df8a01fb18\""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"everywhere","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"Everywhere","title":"mirai - Minimalist Async Evaluation Framework for R","text":"everywhere() may used evaluate expression connected daemons persist resultant state, regardless daemon’s ‘cleanup’ setting. keeps DBI package loaded evaluations. types setup task may also performed, including making common resource available, database connection: super-assignment, conenction ‘con’ available global environment daemon instances. Subsequent mirai calls may make use ‘con’. Disconnect database everywhere, set number daemons zero reset.","code":"everywhere(library(DBI)) file <- tempfile() everywhere(con <<- dbConnect(RSQLite::SQLite(), file), file = file) m <- mirai(capture.output(str(con))) m[] #> [1] \"Formal class 'SQLiteConnection' [package \\\"RSQLite\\\"] with 8 slots\" #> [2] \" ..@ ptr : \" #> [3] \" ..@ dbname : chr \\\"/tmp/RtmpzXpeSH/file34b5234a60eea\\\"\" #> [4] \" ..@ loadable.extensions: logi TRUE\" #> [5] \" ..@ flags : int 70\" #> [6] \" ..@ vfs : chr \\\"\\\"\" #> [7] \" ..@ ref : \" #> [8] \" ..@ bigint : chr \\\"integer64\\\"\" #> [9] \" ..@ extended_types : logi FALSE\" everywhere(dbDisconnect(con)) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"with-method","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"With Method","title":"mirai - Minimalist Async Evaluation Framework for R","text":"daemons() () method, evaluates expression daemons created duration expression automatically torn upon completion. designed use case running Shiny app desired number daemons. Note: case, assumed app already created. Wrapping call shiny::shinyApp() work runApp() implicitly called app printed, however printing occurs () returned, hence app run outside scope () statement. « Back ToC","code":"with(daemons(4), shiny::runApp(app))"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"distributed-computing-remote-daemons","dir":"Articles","previous_headings":"","what":"Distributed Computing: Remote Daemons","title":"mirai - Minimalist Async Evaluation Framework for R","text":"daemons interface may also used send tasks computation remote daemon processes network. Call daemons() specifying ‘url’ character string : ‘tcp://10.75.32.70:5555’ daemon processes connect . Alternatively, use host_url() automatically construct valid URL. IPv6 addresses also supported must enclosed square brackets [] avoid confusion final colon separating port. example, port 5555 IPv6 address ::ffff:a6f:50d specified tcp://[::ffff:a6f:50d]:5555. options actually launching daemons, please see next section.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"connecting-to-remote-daemons-through-dispatcher","dir":"Articles","previous_headings":"Distributed Computing: Remote Daemons","what":"Connecting to Remote Daemons Through Dispatcher","title":"mirai - Minimalist Async Evaluation Framework for R","text":"default dispatcher = \"process\" alternatively dispatcher = \"thread\", dispatcher listens vector URLs remote daemon() processes dial , daemon unique URL. recommended use websocket URL starting ws:// instead TCP scenario (used interchangeably tcp://). websocket URL supports path port number, can made unique daemon. way dispatcher can connect arbitrary number daemons single port. Supplying vector URLs allows use arbitrary port numbers / paths. ‘n’ need specified can inferred length ‘url’ vector, example: Alternatively, single URL supplied, along n = 4 specify dispatcher listen 4 URLs. case, integer sequence automatically appended path /1 /4 produce URLs. Requesting status host machine: per local case, $connections shows single connection dispatcher, however $daemons now provides matrix statistics remote daemons. index number. online shows 1 active connection, else 0 daemon yet connect disconnected. instance increments 1 every time new connection URL. counter designed track new daemon instances connecting previous ones ended (due time-outs etc.). count becomes negative immediately URL regenerated saisei(), increments new daemon connects. assigned shows cumulative number tasks assigned daemon. complete shows cumulative number tasks completed daemon. Dispatcher automatically adjusts number daemons actually connected. Hence possible dynamically scale number daemons according requirements (limited ‘n’ URLs assigned). reset connections revert default behaviour: Closing connection causes dispatcher exit automatically, turn connected daemons respective connections dispatcher terminated.","code":"daemons(url = c(\"ws://10.75.32.70:5566/cpu\", \"ws://10.75.32.70:5566/gpu\", \"ws://10.75.32.70:7788/1\")) daemons(n = 4, url = host_url(port = 5555)) #> [1] 4 status() #> $connections #> [1] 1 #> #> $daemons #> i online instance assigned complete #> tcp://hostname:5555 1 0 0 0 0 #> tcp://hostname:5556 2 0 0 0 0 #> tcp://hostname:5557 3 0 0 0 0 #> tcp://hostname:5558 4 0 0 0 0 daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"connecting-to-remote-daemons-directly","dir":"Articles","previous_headings":"Distributed Computing: Remote Daemons","what":"Connecting to Remote Daemons Directly","title":"mirai - Minimalist Async Evaluation Framework for R","text":"specifying dispatcher = \"none\", remote daemons connect directly host process. host listens single URL, distributes tasks connected daemons. Note , calling host_url() without port value uses default ‘0’. wildcard value automatically cause free ephemeral port assigned. actual assigned port may queried time via status(): number daemons connecting host URL limited network resources may added removed time, tasks automatically distributed connected daemons. status query , $connections shows actual number connected daemons. reset connections revert default behaviour: causes connected daemons exit automatically. « Back ToC","code":"daemons(url = host_url(), dispatcher = \"none\") #> [1] 0 status() #> $connections #> [1] 0 #> #> $daemons #> [1] \"tcp://hostname:43129\" daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"distributed-computing-launching-daemons","dir":"Articles","previous_headings":"","what":"Distributed Computing: Launching Daemons","title":"mirai - Minimalist Async Evaluation Framework for R","text":"launch remote daemons, supply remote launch configuration ‘remote’ argument daemons() setting daemons, launch_remote() time afterwards. ssh_config() may used generate remote launch configuration SSH access remote machine, else remote_config() provides flexible method generating configuration involving custom resource manager / application.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"ssh-direct-connection","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"SSH Direct Connection","title":"mirai - Minimalist Async Evaluation Framework for R","text":"first example launches 4 daemons machine 10.75.32.90 (using default SSH port 22 specified), connecting back dispatcher URLs: second example launches one daemon 10.75.32.90 10.75.32.91 using custom SSH port 222: examples, remote daemons connect back directly, port 5555 local machine must open incoming connections remote addresses.","code":"daemons( n = 4, url = host_url(ws = TRUE, port = 5555), remote = ssh_config(remotes = \"ssh://10.75.32.90\") ) daemons( n = 2, url = host_url(ws = TRUE, port = 5555), remote = ssh_config(c(\"ssh://10.75.32.90:222\", \"ssh://10.75.32.91:222\")) )"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"ssh-tunnelling","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"SSH Tunnelling","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use SSH tunnelling provides convenient way launch remote daemons without requiring remote machine able access host. Often firewall configurations security policies may prevent opening port accept outside connections. cases SSH tunnelling offers solution creating tunnel initial SSH connection made. simplicity, SSH tunnelling implementation uses port side host corresponding node. SSH key-based authentication must also already place. Tunnelling requires hostname ‘url’ specified setting daemons either ‘127.0.0.1’ ‘localhost’. tunnel created 127.0.0.1:port equivalently localhost:port machine. host listens localhost:port remotes dial localhost:port respective machines. example launches 2 nodes remote machine 10.75.32.90 using SSH tunnelling port 5555 (‘url’ hostname specified ‘localhost’):","code":"daemons( url = \"tcp://localhost:5555\", remote = ssh_config( remotes = c(\"ssh://10.75.32.90\", \"ssh://10.75.32.90\"), tunnel = TRUE ) )"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"cluster-resource-managers","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"Cluster Resource Managers","title":"mirai - Minimalist Async Evaluation Framework for R","text":"remote_config() may used run command deploy daemons using resource manager. Taking Slurm example, following uses srun launch 2 daemons cluster, additional arguments srun specifying resource allocation:","code":"daemons( n = 2, url = host_url(ws = TRUE), remote = remote_config( command = \"srun\", args = c(\"--mem 512\", \"-n 1\", \".\"), rscript = file.path(R.home(\"bin\"), \"Rscript\") ) )"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"manual-deployment","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"Manual Deployment","title":"mirai - Minimalist Async Evaluation Framework for R","text":"alternative automated launches, calling launch_remote() without specifying ‘remote’ may used return shell commands deploying daemons manually. printed return values may copy / pasted directly remote machine. Note daemons() set host machine launching daemon() remote resources, otherwise daemon instances exit connection immediately available. Alternatively, specifying argument autoexit = FALSE allow daemons wait (indefinitely) connection become available. « Back ToC","code":"daemons(n = 2, url = host_url()) #> [1] 2 launch_remote(1:2) #> [1] #> Rscript -e 'mirai::daemon(\"tcp://hostname:42325\",rs=c(10407,-1828318451,501344874,1362761955,397178056,735030313,1295249878))' #> #> [2] #> Rscript -e 'mirai::daemon(\"tcp://hostname:32991\",rs=c(10407,1324936348,1629295047,-90918194,1459337963,994896166,-326446377))' daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"distributed-computing-tls-secure-connections","dir":"Articles","previous_headings":"","what":"Distributed Computing: TLS Secure Connections","title":"mirai - Minimalist Async Evaluation Framework for R","text":"TLS available option secure communications local machine remote daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"zero-configuration","dir":"Articles","previous_headings":"Distributed Computing: TLS Secure Connections","what":"Zero-configuration","title":"mirai - Minimalist Async Evaluation Framework for R","text":"automatic zero-configuration default implemented. Simply specify secure URL form wss:// tls+tcp:// setting daemons, use host_url(tls = TRUE), example: Single-use keys certificates automatically generated configured, without requiring intervention. private key always retained host machine never transmitted. generated self-signed certificate available via launch_remote(). function conveniently constructs full shell command launch daemon, including correctly specified ‘tls’ argument daemon(). printed value may deployed directly remote machine. « Back ToC","code":"daemons(n = 4, url = host_url(ws = TRUE, tls = TRUE)) #> [1] 4 launch_remote(1) #> [1] #> Rscript -e 'mirai::daemon(\"wss://hostname:45635/1\",tls=c(\"-----BEGIN CERTIFICATE----- #> MIIFNzCCAx+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQDDAhrdW1h #> bW90bzERMA8GA1UECgwITmFub25leHQxCzAJBgNVBAYTAkpQMB4XDTAxMDEwMTAw #> MDAwMFoXDTMwMTIzMTIzNTk1OVowMzERMA8GA1UEAwwIa3VtYW1vdG8xETAPBgNV #> BAoMCE5hbm9uZXh0MQswCQYDVQQGEwJKUDCCAiIwDQYJKoZIhvcNAQEBBQADggIP #> ADCCAgoCggIBANw6CDKpRtGQg8fAUO5o04BdaH9kfWWpqniBpD0UMNGUJu93NtYd #> TSdPhmB8pbJB4MtdVHFgB4kaFMQZ27N9bu9Bc7vXnzfs7Kh0felYQbjforNePWDz #> 9mxcy0ASNxn6iLUMsVXd9ZiyfKkUjkPirGaqfJ5sQE40yt49voeD6H5RMJTf5jts #> 15Pyo57XnWIh2V60yQ+VGuiqbVfAAY6cA812KQnvxr3MmQxJ8W+C8kFn8x9WK9zI #> 43rx9oQEVMVeJ2Bz4WKFuTohX0+PuGDJgMBQS4vns9uGmSQnuwzEVzq6KxtdE5fF #> S52DT0UAIWTjdO9ZDFa2aa3AMxpHfzGkl5YjKgP3cTbSaBKPpJaxY06r6DuX/UXN #> YUzZ6qL0+mmNqjBNI5+2fnBY5G18MXQC9hzXmosZMxJgiJSaRPz6lw9Q9oesDTXR #> tnCLbsp/F7ELxUqAWtxo+AIhE+4BZHVWRCOMif+kbvhm3iimHOtcErCa5Q/QtFS9 #> VYYKT2wsfxL8WP5n5nWPDq9qfy9WSo6kiFnUDJhilHBMDO4yVrocE4j7fEatckqu #> nXlPwRAMb8vwxHVJiQbTIpIQVBJOU0tUxHSH9KZGe5UBnV+isJJEsRtj4WPvTsx9 #> /tK7O61Z5fCNIEeULdXRgSLB2iTeRgBYA6QNqenDz17HRA6zmX4S9HqHAgMBAAGj #> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFH05owUC7+3jOv7Sh3WF #> Hn/TwV6KMB8GA1UdIwQYMBaAFH05owUC7+3jOv7Sh3WFHn/TwV6KMA0GCSqGSIb3 #> DQEBCwUAA4ICAQAzCWeL/W0HC9vZrO39FFyITkgY5TicGyaP51Y6VKxAlJX1PRli #> 4vjetgajqmsRyNqZ+YByWvc+TUce4Mo76pYgtwlPiVhbn3kfcxSKA9mHGCNIcxpj #> BO0tJlLQnNjLCtZNze4WMnNynE6H7ePki2DrCECGeJTtXfiyHnnFgu+gRFhRGlTe #> ZCTrzjm57F3ka+UIdGZUXuzWkSQgNwRBZg1279B1yqNQahBie+QTh7hEv55gPiiu #> 77snyPJGsumZ0QaOPj4TyPsCfHYys5DSd7Xs87a13PzPEfiYF2iWz6M69aiSKj2s #> 2Cd2R2kvUqjs/krc464QuRZBNaNhnyR3uOzt5wcuZmJC67AeVlU77IQkvTy2xlQF #> UnxSpGNxGSu1JruOjBeirSx7pM3Qlv6YROYwm/ru01xdxcTiN7+3ytn46i5puWAe #> gUYOIL1MoPwhiYsiCNcjiw50PUpIAE/aZ49UNg857CC8EEs5EvnbHgx0oJgws1Xx #> 4vz2CjaI2e4KzLKorN0WVTIBWWrjzD4hqbas2PxWD8sXqknsi8n/YChEXqa3xuzn #> TTJxcbG9qViOXFaND2Nu7wpYdbAOwAAuryQ0ic+1nYe5tibYUPtD43MSs7+nlhN3 #> s0n8cCVrhYG+CJekq/2gRfzP48xmz1pBb3gJpvcHZika8FGPv9+IYsDX8w== #> -----END CERTIFICATE----- #> \",\"\"),rs=c(10407,1672045068,-894820451,225659322,-372605965,-77358952,-166139463))'"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"ca-signed-certificates","dir":"Articles","previous_headings":"Distributed Computing: TLS Secure Connections","what":"CA Signed Certificates","title":"mirai - Minimalist Async Evaluation Framework for R","text":"alternative zero-configuration default, certificate may also generated via Certificate Signing Request (CSR) Certificate Authority (CA), may public CA CA internal organisation. Generate private key CSR. following resources describe : using Mbed TLS: https://mbed-tls.readthedocs.io/en/latest/kb/-/generate--certificate-request-csr/ using OpenSSL: https://www.feistyduck.com/library/openssl-cookbook/online/ (Chapter 1.2 Key Certificate Management) Provide generated CSR CA sign new TLS certificate. common name (CN) certificate must identical hostname IP address actually used connection. verified, fail . received certificate comprise block cipher text markers -----BEGIN CERTIFICATE----- -----END CERTIFICATE-----. Make sure request certificate PEM format. available formats, TLS library used usually provide conversion utilities. Check also private key block cipher text markers -----BEGIN PRIVATE KEY----- -----END PRIVATE KEY-----. setting daemons, TLS certificate private key provided ‘tls’ argument daemons(). certificate private key imported character strings cert key respectively, ‘tls’ argument may specified character vector c(cert, key). Alternatively, certificate may copied new text file, private key appended, case path/filename file may provided ‘tls’ argument. launching daemons, certificate chain CA supplied ‘tls’ argument daemon() launch_remote(). certificate chain comprise multiple certificates, -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- markers. first one newly-generated TLS certificate, supplied daemons(), final one CA root certificate. certificates required certificate signed directly CA. , intermediate certificates included certificate chain starts TLS certificate ends certificate CA. concatenated together single character string certchain, character vector comprising empty character string c(certchain, \"\") may supplied relevant ‘tls’ argument. Alternatively, written file (file replicated remote machines), ‘tls’ argument may also specified path/filename (assuming machine). « Back ToC","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"compute-profiles","dir":"Articles","previous_headings":"","what":"Compute Profiles","title":"mirai - Minimalist Async Evaluation Framework for R","text":"daemons() interface also allows specification compute profiles managing tasks heterogeneous compute requirements: send tasks different daemons clusters daemons appropriate specifications (terms CPUs / memory / GPU / accelerators etc.) split tasks local remote computation Simply specify argument .compute calling daemons() profile name (‘default’ default profile). daemons settings saved named profile. create ‘mirai’ task using specific compute profile, specify ‘.compute’ argument mirai(), defaults ‘default’ compute profile. Similarly, functions status(), launch_local() launch_remote() specified desired ‘.compute’ argument. « Back ToC","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"errors-interrupts-and-timeouts","dir":"Articles","previous_headings":"","what":"Errors, Interrupts and Timeouts","title":"mirai - Minimalist Async Evaluation Framework for R","text":"execution mirai fails, error message returned character string class ‘miraiError’ ‘errorValue’ facilitate debugging. is_mirai_error() may used test mirai execution errors. full stack trace evaluation within mirai recorded accessible $stack.trace error object. daemon instance sent user interrupt, mirai resolve object class ‘miraiInterrupt’ ‘errorValue’. is_mirai_interrupt() may used test interrupts. execution mirai surpasses timeout set via ‘.timeout’ argument, mirai resolve ‘errorValue’ 5L (timed ). can, amongst things, guard mirai processes potential hang never return. is_error_value() tests mirai execution errors, user interrupts timeouts. « Back ToC","code":"m1 <- mirai(stop(\"occurred with a custom message\", call. = FALSE)) m1[] #> 'miraiError' chr Error: occurred with a custom message m2 <- mirai(mirai::mirai()) m2[] #> 'miraiError' chr Error in mirai::mirai(): missing expression, perhaps wrap in {}? is_mirai_error(m2$data) #> [1] TRUE is_error_value(m2$data) #> [1] TRUE f <- function(x) if (x > 0) stop(\"positive\") m3 <- mirai({f(-1); f(1)}, f = f) m3[] #> 'miraiError' chr Error in f(1): positive m3$data$stack.trace #> [[1]] #> [1] \"stop(\\\"positive\\\")\" #> #> [[2]] #> [1] \"f(1)\" m4 <- mirai(rlang::interrupt()) # simulates a user interrupt is_mirai_interrupt(m4[]) #> [1] TRUE m4 <- mirai(nanonext::msleep(1000), .timeout = 500) m4[] #> 'errorValue' int 5 | Timed out is_mirai_error(m4$data) #> [1] FALSE is_mirai_interrupt(m4$data) #> [1] FALSE is_error_value(m4$data) #> [1] TRUE"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"serialization-arrow-polars-and-beyond","dir":"Articles","previous_headings":"","what":"Serialization: Arrow, polars and beyond","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Native R serialization used sending data host daemons. R objects nature serialized, accessed via external pointer. cases, performing ‘mirai’ operations normally error. Using arrow package example: However, serial_config() can used create custom serialization configurations, specifying functions hook R’s native serialization mechanism reference objects (‘refhooks’). configuration can specified part everywhere() call via ‘.serial’ argument. can seen time, arrow table seamlessly handled ‘mirai’ process. case even object deeply nested inside lists structures. change registered serialization functions, just call everywhere() supplying new functions. example, can switch using polars, ‘lightning fast’ dataframe library written Rust (requires polars >= 0.16.4). cancel serialization functions entirely, supply empty list ‘.serial’ argument everywhere(): ‘vec’ argument serialization() may specified TRUE serialization functions vectorized take lists objects, case safetensors, used serialization torch. Please refer torch vignette examples. « Back ToC","code":"library(arrow, warn.conflicts = FALSE) daemons(2) #> [1] 2 everywhere(library(arrow)) x <- as_arrow_table(iris) m <- mirai(list(a = head(x), b = \"some text\"), x = x) m[] #> 'miraiError' chr Error: Invalid , external pointer to null cfg <- serial_config( class = \"ArrowTabular\", sfunc = arrow::write_to_raw, ufunc = function(x) arrow::read_ipc_stream(x, as_data_frame = FALSE) ) daemons(2) #> [1] 2 everywhere(library(arrow), .serial = cfg) m <- mirai(list(a = head(x), b = \"some text\"), x = x) m[] #> $a #> Table #> 6 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species > #> #> See $metadata for additional Schema metadata #> #> $b #> [1] \"some text\" everywhere( {}, .serial = serial_config( class = \"RPolarsDataFrame\", sfunc = function(x) polars::as_polars_df(x)$to_raw_ipc(), ufunc = polars::pl$read_ipc ) ) x <- polars::as_polars_df(iris) m <- mirai(list(a = head(x), b = \"some text\"), x = x) m[] #> $a #> shape: (6, 5) #> ┌──────────────┬─────────────┬──────────────┬─────────────┬─────────┐ #> │ Sepal.Length ┆ Sepal.Width ┆ Petal.Length ┆ Petal.Width ┆ Species │ #> │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ #> │ f64 ┆ f64 ┆ f64 ┆ f64 ┆ cat │ #> ╞══════════════╪═════════════╪══════════════╪═════════════╪═════════╡ #> │ 5.1 ┆ 3.5 ┆ 1.4 ┆ 0.2 ┆ setosa │ #> │ 4.9 ┆ 3.0 ┆ 1.4 ┆ 0.2 ┆ setosa │ #> │ 4.7 ┆ 3.2 ┆ 1.3 ┆ 0.2 ┆ setosa │ #> │ 4.6 ┆ 3.1 ┆ 1.5 ┆ 0.2 ┆ setosa │ #> │ 5.0 ┆ 3.6 ┆ 1.4 ┆ 0.2 ┆ setosa │ #> │ 5.4 ┆ 3.9 ┆ 1.7 ┆ 0.4 ┆ setosa │ #> └──────────────┴─────────────┴──────────────┴─────────────┴─────────┘ #> #> $b #> [1] \"some text\" everywhere({}, .serial = list()) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"asynchronous-parallel-map","dir":"Articles","previous_headings":"","what":"Asynchronous Parallel Map","title":"mirai - Minimalist Async Evaluation Framework for R","text":"mirai_map() performs asynchronous parallel/distributed map using mirai. function similar purrr::map(), returns ‘mirai_map’ object. also advanced allows multiple map rows dataframe matrix. results mirai_map x may collected using x[]. waits asynchronous operations complete still progress.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"key-advantages","dir":"Articles","previous_headings":"Asynchronous Parallel Map","what":"Key advantages:","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Returns immediately evaluations taking place asynchronously. Printing ‘mirai map’ object shows current completion progress. ‘.promise’ argument allows promise registered mirai, can used perform side-effects. Returns evaluation errors ‘miraiError’ ‘errorValue’ case may , rather causing entire operation fail. allows efficient recovery partial failure. rely ‘chunking’ algorithm attempts split work batches according number available daemons, implemented example parallel package. Chunking take account varying unpredictable compute times indices. can optimal rely mirai scheduling instead. demonstrated example . .args used specify constant arguments .f - ‘mean’ ‘sd’ example : Use ... specify objects referenced defined .f - ‘’ anonymous function : Use mirai_map() assumes daemons() previously set. one (non-dispatcher) daemon set allow function proceed. ensures safe behaviour, unlikely optimal, please ensure daemons set beforehand.","code":"library(mirai) library(parallel) cl <- make_cluster(4) daemons(4) #> [1] 4 vec <- c(1, 1, 4, 4, 1, 1, 1, 1) system.time(mirai_map(vec, Sys.sleep)[]) #> user system elapsed #> 0.004 0.004 4.007 system.time(parLapply(cl, vec, Sys.sleep)) #> user system elapsed #> 0.011 0.003 8.011 with( daemons(3, dispatcher = \"none\"), mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[] ) #> [[1]] #> [1] 21.78985 #> #> [[2]] #> [1] 20.14001 17.18939 #> #> [[3]] #> [1] 20.35270 20.68685 22.75893 ml <- mirai_map( c(a = 1, b = 2, c = 3), function(x) do(x, as.logical(x %% 2)), do = nanonext::random ) #> Warning: mirai is launching one local daemon for a map operation as none previously set ml #> < mirai map [2/3] > ml[] #> $a #> [1] \"23\" #> #> $b #> [1] 27 32 #> #> $c #> [1] \"2dd1bf\""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"collecting-results","dir":"Articles","previous_headings":"Asynchronous Parallel Map","what":"Collecting Results","title":"mirai - Minimalist Async Evaluation Framework for R","text":"collecting results, optionally specify arguments []: x[.flat] collects flattens results, checking type avoid coercion. x[.progress] collects results whilst showing simple text progress indicator parts completed total. x[.progress_cli] alternative uses cli progress bars, available, showing completion percentage ETA. x[.stop] collects results applying early stopping, stops first failure cancels remaining computations (note: computations already progress continue completion, results collected). Combinations may supplied fashion x[.stop, .progress].","code":"tryCatch( mirai_map(list(a = 1, b = \"a\", c = 3), sum)[.stop], error = identity ) #> with( daemons(4, dispatcher = \"none\"), mirai_map(c(0.1, 0.2, 0.3), Sys.sleep)[.progress, .flat] ) #> NULL"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"multiple-map","dir":"Articles","previous_headings":"Asynchronous Parallel Map","what":"Multiple Map","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Multiple map performed rows dataframe matrix, often desired behaviour. allows map 2 arguments specifying dataframe. One may index value indexed map. dataframe often contains columns differing type, unusual want map columns, however possible simply transforming beforehand list using .list(). Similarly, behaviour lapply() purrr::map() matrix vector. mirai_map() hand take account fact matrix dimensions, maps rows, consistent behaviour dataframes. instead, mapping columns desired, simply take transpose matrix beforehand using t(). « Back ToC","code":"fruit <- c(\"melon\", \"grapes\", \"coconut\") # create a dataframe for indexed map: df <- data.frame(i = seq_along(fruit), fruit = fruit) with( daemons(3, dispatcher = \"none\"), mirai_map(df, sprintf, .args = list(fmt = \"%d. %s\"))[.flat] ) #> [1] \"1. melon\" \"2. grapes\" \"3. coconut\""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"using-mirai-in-a-package","dir":"Articles","previous_headings":"","what":"Using mirai in a Package","title":"mirai - Minimalist Async Evaluation Framework for R","text":"mirai framework designed support completely transparent inter-operable use within packages. core design precept relying global options environment variables minimises likelihood conflict use different packages. hence requirements package authors. following points may, however, useful: daemons() settings usually left end-users. Users may guided mirai documentation desired. however, package wishes set default settings, example, daemons() always called specifying force = FALSE. ensures prior user settings respected, daemons set elsewhere prematurely terminated. Calling package functions mirai requires namespacing call, alternatively exporting function, .e. shape contents status() daemons matrix must relied upon, user interface subject change time. developer interface nextget(), querying values ‘urls’ described function documentation. Note: specifically-documented values supported interfaces. recommended practice case, especially relevant package developers: functions unresolved(), is_error_value(), is_mirai_error(), is_mirai_interrupt() used test relevant state mirai value. characteristics currently implemented, e.g. logical NA ‘unresolvedValue’, relied upon, subject change. Testing CRAN respect ’s 2-core usage limit. practically means limiting tests using one daemon (dispatcher = \"none\") ensure one additional process used. Always reset daemons done allow least one-second sleep ensure background processes properly exited. limits apply tests run CRAN machines run elsewhere. « Back ToC","code":"mirai(mypkg::my_func()) mirai(my_func(), .args = list(myfunc = myfunc))"},{"path":"https://shikokuchuo.net/mirai/articles/parallel.html","id":"parallel-integration","dir":"Articles","previous_headings":"","what":"Parallel Integration","title":"mirai - Parallel Integration","text":"mirai provides alternative communications backend R. functionality developed fulfil request R Core R Project Sprint 2023. make_cluster() creates cluster object class ‘miraiCluster’, fully-compatible parallel cluster types. Specify ‘n’ launch nodes local machine. Specify ‘url’ receiving connections remote nodes. Optionally, specify ‘remote’ launch remote daemons using remote configuration generated remote_config() ssh_config(). Created clusters may used function parallel base package parallel::clusterApply() parallel::parLapply(), load-balanced versions parallel::parLapplyLB(). status() may called ’miraiCluster` query number connected nodes time. Making cluster specifying ‘url’ without ‘remote’ causes shell commands manual deployment nodes printed console.","code":"library(mirai) cl <- make_cluster(4) cl #> < miraiCluster | ID: `0` nodes: 4 active: TRUE > parallel::parLapply(cl, iris, mean) #> $Sepal.Length #> [1] 5.843333 #> #> $Sepal.Width #> [1] 3.057333 #> #> $Petal.Length #> [1] 3.758 #> #> $Petal.Width #> [1] 1.199333 #> #> $Species #> [1] NA status(cl) #> $connections #> [1] 4 #> #> $daemons #> [1] \"abstract://e755083adb3ac002d81d87c7\" stop_cluster(cl) cl <- make_cluster(n = 2, url = host_url()) #> Shell commands for deployment on nodes: #> #> [1] #> Rscript -e 'mirai::daemon(\"tcp://hostname:41567\",rs=c(10407,-1229812554,-355079233,-704283916,613800293,-1049741598,-1259508485))' #> #> [2] #> Rscript -e 'mirai::daemon(\"tcp://hostname:41567\",rs=c(10407,-936602309,482053993,1829342693,1529156475,2009204106,1430761999))' stop_cluster(cl)"},{"path":"https://shikokuchuo.net/mirai/articles/parallel.html","id":"foreach-integration","dir":"Articles","previous_headings":"","what":"Foreach Integration","title":"mirai - Parallel Integration","text":"‘miraiCluster’ may also registered doParallel use foreach package. Running parallel examples foreach() function:","code":"library(foreach) library(iterators) cl <- make_cluster(4) doParallel::registerDoParallel(cl) # normalize the rows of a matrix m <- matrix(rnorm(9), 3, 3) foreach(i = 1:nrow(m), .combine = rbind) %dopar% (m[i, ] / mean(m[i, ])) #> [,1] [,2] [,3] #> result.1 0.6168828 1.1209518 1.2621654 #> result.2 1.9631338 0.3970546 0.6398116 #> result.3 1.8112756 8.5974163 -7.4086918 # simple parallel matrix multiply a <- matrix(1:16, 4, 4) b <- t(a) foreach(b = iter(b, by='col'), .combine = cbind) %dopar% (a %*% b) #> [,1] [,2] [,3] [,4] #> [1,] 276 304 332 360 #> [2,] 304 336 368 400 #> [3,] 332 368 404 440 #> [4,] 360 400 440 480"},{"path":"https://shikokuchuo.net/mirai/articles/plumber.html","id":"plumber-integration","dir":"Articles","previous_headings":"","what":"Plumber Integration","title":"mirai - Plumber Integration","text":"mirai may used asynchronous backend plumber pipelines. Example usage provided different types endpoint.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/plumber.html","id":"example-get-endpoint","dir":"Articles","previous_headings":"Plumber Integration","what":"Example GET Endpoint","title":"mirai - Plumber Integration","text":"plumber router code run daemon process block interactive process. /echo endpoint takes GET request, sleeps 1 second (simulating expensive computation) simply returns ‘msg’ request header together timestamp process ID process run . API can queried using async HTTP client nanonext::ncurl_aio(). , 8 requests submitted , note responses differing timestamps 4 can processed one time (limited number daemons set).","code":"library(mirai) # supply SIGINT so the plumber server is interrupted and exits cleanly when finished daemons(1L, dispatcher = \"none\", autoexit = tools::SIGINT) #> [1] 1 m <- mirai({ library(plumber) library(promises) # to provide the promise pipe library(mirai) # more efficient not to use dispatcher if all requests are similar length daemons(4L, dispatcher = \"none\") # handles 4 requests simultaneously pr() |> pr_get( \"/echo\", function(req, res) { mirai( { Sys.sleep(1L) list( status = 200L, body = list( time = format(Sys.time()), msg = msg, pid = Sys.getpid() ) ) }, msg = req[[\"HEADERS\"]][[\"msg\"]] ) %...>% (function(x) { res$status <- x$status res$body <- x$body }) } ) |> pr_run(host = \"127.0.0.1\", port = 8985) }) library(nanonext) res <- lapply( 1:8, function(i) ncurl_aio( \"http://127.0.0.1:8985/echo\", headers = c(msg = as.character(i)) ) ) collect_aio(res) #> [[1]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"1\\\"],\\\"pid\\\":[81109]}\" #> #> [[2]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"2\\\"],\\\"pid\\\":[81111]}\" #> #> [[3]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"3\\\"],\\\"pid\\\":[81107]}\" #> #> [[4]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"4\\\"],\\\"pid\\\":[81115]}\" #> #> [[5]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"5\\\"],\\\"pid\\\":[81111]}\" #> #> [[6]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"6\\\"],\\\"pid\\\":[81115]}\" #> #> [[7]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"7\\\"],\\\"pid\\\":[81109]}\" #> #> [[8]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"8\\\"],\\\"pid\\\":[81107]}\" daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/plumber.html","id":"example-post-endpoint","dir":"Articles","previous_headings":"Plumber Integration","what":"Example POST Endpoint","title":"mirai - Plumber Integration","text":"demonstration equivalent using POST endpoint, accepting JSON instruction sent request data. Note req$postBody always accessed router process passed argument ‘mirai’, retrieved using connection serializable. Querying endpoint produces set outputs previous example.","code":"library(mirai) # supply SIGINT so the plumber server is interrupted and exits cleanly when finished daemons(1L, dispatcher = \"none\", autoexit = tools::SIGINT) #> [1] 1 m <- mirai({ library(plumber) library(promises) # to provide the promise pipe library(mirai) # uses dispatcher - suitable when requests take differing times to complete daemons(4L, dispatcher = \"thread\") # handles 4 requests simultaneously pr() |> pr_post( \"/echo\", function(req, res) { mirai( { Sys.sleep(1L) # simulate expensive computation list( status = 200L, body = list( time = format(Sys.time()), msg = jsonlite::fromJSON(data)[[\"msg\"]], pid = Sys.getpid() ) ) }, data = req$postBody ) %...>% (function(x) { res$status <- x$status res$body <- x$body }) } ) |> pr_run(host = \"127.0.0.1\", port = 8986) }) library(nanonext) res <- lapply( 1:8, function(i) ncurl_aio( \"http://127.0.0.1:8986/echo\", method = \"POST\", data = sprintf('{\"msg\":\"%d\"}', i) ) ) collect_aio(res) #> [[1]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"1\\\"],\\\"pid\\\":[81336]}\" #> #> [[2]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"2\\\"],\\\"pid\\\":[81338]}\" #> #> [[3]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"3\\\"],\\\"pid\\\":[81344]}\" #> #> [[4]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"4\\\"],\\\"pid\\\":[81338]}\" #> #> [[5]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"5\\\"],\\\"pid\\\":[81336]}\" #> #> [[6]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"6\\\"],\\\"pid\\\":[81340]}\" #> #> [[7]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"7\\\"],\\\"pid\\\":[81340]}\" #> #> [[8]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"8\\\"],\\\"pid\\\":[81344]}\" daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/promises.html","id":"promises-integration","dir":"Articles","previous_headings":"","what":"Promises Integration","title":"mirai - Promises Integration","text":"mirai supplies .promise() method, allowing used promise promises package. next-generation, event-driven promises, developed collaboration Joe Cheng. require promise polled completion later loop like promises. Instead, promise actions automatically queued execution soon ‘mirai’ resolves. Allows much higher responsiveness (lower latency) massive scalability (situations thousand promises ). ‘mirai’ may piped directly using promise pipe &...>%, implicitly calls .promise() ‘mirai’. Similarly promise-aware functions promises::() shiny::ExtendedTask$new() take promise can also take ‘mirai’ (using promises >= 1.3.0). Alternatively, ‘mirai’ may explicitly converted promise .promise(), allows using methods $(), $finally() etc. following example outputs “hello” console one second ‘mirai’ resolves. possible access ‘mirai’ value $data use promise enacting side effect (assigning value environment example ). returning top level prompt:","code":"library(mirai) library(promises) p <- mirai({Sys.sleep(1); \"hello\"}) %...>% cat() p #> env <- new.env() m <- mirai({ Sys.sleep(1) \"hello\" }) promises::then(m, function(x) env$res <- x) m[] #> [1] \"hello\" env$res #> [1] \"hello\""},{"path":"https://shikokuchuo.net/mirai/articles/promises.html","id":"the-one-million-promises-challenge","dir":"Articles","previous_headings":"","what":"The One Million Promises Challenge","title":"mirai - Promises Integration","text":"code taken challenge launch collect one million promises. illustration, example scaled ten thousand. one million promises challenge took 6 mins 25 secs complete using Intel i7 11th gen mobile processor 16GB RAM.","code":"library(mirai) daemons(8, dispatcher = \"none\") #> [1] 8 r <- 0 start <- Sys.time() m <- mirai_map(1:10000, \\(x) x, .promise = \\(x) r <<- r + x) Sys.time() - start #> Time difference of 2.722618 secs later::run_now() r #> [1] 50005000 daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"shiny-integration","dir":"Articles","previous_headings":"","what":"Shiny Integration","title":"mirai - Shiny Integration","text":"mirai may used asynchronous backend scale Shiny applications. Depending options suppled daemons(), mirai tasks may distributed across local background processes multiple networked servers efficient performant manner. use Shiny, mirai implements truly event-driven promises, developed collaboration Joe Cheng (creator Shiny). promise polled loop completion (types promise). Instead, promise actions triggered execution soon ‘mirai’ completes (asynchronously). Allows much higher responsiveness (lower latency) massive scalability (situations thousand promises ). mirai natively supports Shiny ExtendedTask create scalable Shiny apps, remain responsive intra-session user, well inter-session multiple concurrent users. ‘mirai’ may used anywhere ‘promise’ ‘future_promise’ accepted (promises >= 1.3.0).","code":""},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"shiny-extendedtask-example-plot-with-clock","dir":"Articles","previous_headings":"Shiny Integration","what":"Shiny ExtendedTask Example: Plot with Clock","title":"mirai - Shiny Integration","text":"example , app remains responsive, clock continuing tick whilst simulated expensive computation running asynchronously parallel process. Also button disabled plot greyed computation complete. wrapping runApp() call (daemons(...), ...) daemons set duration app, exiting automatically app stopped. Thanks Joe Cheng providing examples based. key components using ExtendedTask : UI, use bslib::input_task_button(). button disabled computation prevent additional clicks. server, create ExtendedTask object calling ExtendedTask$new() anonymous function passing ... arguments mirai(), bind button created (1). server, create observer input button, invokes ExtendedTask, passing named arguments anonymous function (hence mirai) . server, create render function output, consumes result ExtendedTask.","code":"library(shiny) library(bslib) library(mirai) ui <- page_fluid( p(\"The time is \", textOutput(\"current_time\", inline = TRUE)), hr(), numericInput(\"n\", \"Sample size (n)\", 100), numericInput(\"delay\", \"Seconds to take for plot\", 5), input_task_button(\"btn\", \"Plot uniform distribution\"), plotOutput(\"plot\") ) server <- function(input, output, session) { output$current_time <- renderText({ invalidateLater(1000) format(Sys.time(), \"%H:%M:%S %p\") }) extended_task <- ExtendedTask$new( function(...) mirai({Sys.sleep(y); runif(x)}, ...) ) |> bind_task_button(\"btn\") observeEvent(input$btn, extended_task$invoke(x = input$n, y = input$delay)) output$plot <- renderPlot(hist(extended_task$result())) } app <- shinyApp(ui = ui, server = server) # run app using 2 local daemons with(daemons(2), runApp(app)) input_task_button(\"btn\", \"Plot uniform distribution\") extended_task <- ExtendedTask$new( function(...) mirai({Sys.sleep(y); runif(x)}, ...) ) |> bind_task_button(\"btn\") observeEvent(input$btn, extended_task$invoke(x = input$n, y = input$delay)) output$plot <- renderPlot(hist(extended_task$result()))"},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"shiny-extendedtask-example-generative-art","dir":"Articles","previous_headings":"Shiny Integration","what":"Shiny ExtendedTask Example: Generative Art","title":"mirai - Shiny Integration","text":"following app produces pretty spiral patterns. user can add multiple plots, making use Shiny modules, different calculation time. plots generated asynchronously, easy see practical limitations number daemons set. example, updating 4 plots, 3 daemons, 4th plot start generated one plots finished. example builds original code Joe Cheng, Daniel Woodie William Landau. uses environment() instead ... alternative equivalent way passing variables present calling environment mirai. key components using ExtendedTask example : UI, use bslib::input_task_button(). button disabled computation prevent additional clicks. server, create ExtendedTask object calling ExtendedTask$new() anonymous function passing named arguments mirai(), bind button created (1). passed mirai use environment(). server, create observer input button, invokes ExtendedTask, supplying arguments anonymous function . server, create render function output, consumes result ExtendedTask.","code":"library(shiny) library(mirai) library(bslib) library(ggplot2) library(aRtsy) # function definitions run_task <- function(calc_time) { Sys.sleep(calc_time) list( colors = aRtsy::colorPalette(name = \"random\", n = 3), angle = runif(n = 1, min = - 2 * pi, max = 2 * pi), size = 1, p = 1 ) } plot_result <- function(result) { do.call(what = canvas_phyllotaxis, args = result) } # modules for individual plots plotUI <- function(id, calc_time) { ns <- NS(id) card( strong(paste0(\"Plot (calc time = \", calc_time, \" secs)\")), input_task_button(ns(\"resample\"), \"Resample\"), plotOutput(ns(\"plot\"), height=\"400px\", width=\"400px\") ) } plotServer <- function(id, calc_time) { force(id) force(calc_time) moduleServer( id, function(input, output, session) { extended_task <- ExtendedTask$new( function(time, run) mirai(run(time), environment()) ) |> bind_task_button(\"resample\") observeEvent(input$resample, extended_task$invoke(calc_time, run_task)) output$plot <- renderPlot(plot_result(extended_task$result())) } ) } # ui and server ui <- page_sidebar(fillable = FALSE, sidebar = sidebar( numericInput(\"calc_time\", \"Calculation time (secs)\", 5), actionButton(\"add\", \"Add\", class=\"btn-primary\"), ), layout_column_wrap(id = \"results\", width = \"400px\", fillable = FALSE) ) server <- function(input, output, session) { observeEvent(input$add, { id <- nanonext::random(4) insertUI(\"#results\", where = \"beforeEnd\", ui = plotUI(id, input$calc_time)) plotServer(id, input$calc_time) }) } app <- shinyApp(ui, server) # run app using 3 local daemons with(daemons(3), runApp(app)) input_task_button(ns(\"resample\"), \"Resample\") extended_task <- ExtendedTask$new( function(time, run) mirai(run(time), environment()) ) |> bind_task_button(\"resample\") observeEvent(input$resample, extended_task$invoke(calc_time, run_task)) output$plot <- renderPlot(plot_result(extended_task$result()))"},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"advanced-promises-example-coin-flips","dir":"Articles","previous_headings":"Shiny Integration","what":"Advanced Promises Example: Coin Flips","title":"mirai - Shiny Integration","text":"example demonstrates integrate mirai_map() operation Shiny app. specifying ‘.promise’ argument, registers promise action mapped operation. can used update reactive values otherwise interact Shiny app. adaptation original example provided Landau use crew Shiny. Please see https://wlandau.github.io/crew/articles/shiny.html.","code":"library(shiny) library(mirai) flip_coin <- function(...) { Sys.sleep(0.1) rbinom(n = 1, size = 1, prob = 0.501) } ui <- fluidPage( div(\"Is the coin fair?\"), actionButton(\"task\", \"Flip 1000 coins\"), textOutput(\"status\"), textOutput(\"outcomes\") ) server <- function(input, output, session) { # Keep running totals of heads, tails, and task errors flips <- reactiveValues(heads = 0, tails = 0, flips = 0) # Button to submit a batch of coin flips observeEvent(input$task, { flips$flips <- flips$flips + 1000 m <- mirai_map(1:1000, flip_coin, .promise = \\(x) if (x) flips$heads <- flips$heads + 1 else flips$tails <- flips$tails + 1) }) # Print time and task status output$status <- renderText({ input$task invalidateLater(millis = 1000) time <- format(Sys.time(), \"%H:%M:%S\") sprintf(\"%s %s flips submitted\", time, flips$flips) }) # Print number of heads and tails output$outcomes <- renderText( sprintf(\"%s heads %s tails\", flips$heads, flips$tails) ) } app <- shinyApp(ui = ui, server = server) # run app using 8 local non-dispatcher daemons (tasks are the same length) with(daemons(8, dispatcher = \"none\"), { # pre-load flip_coin function on all daemons for efficiency everywhere({}, flip_coin = flip_coin) runApp(app) })"},{"path":"https://shikokuchuo.net/mirai/articles/torch.html","id":"torch-integration","dir":"Articles","previous_headings":"","what":"Torch Integration","title":"mirai - Torch Integration","text":"Custom serialization functions may registered handle external pointer type reference objects. allows tensors torch package used seamlessly ‘mirai’ computations.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/torch.html","id":"setup-steps","dir":"Articles","previous_headings":"Torch Integration","what":"Setup Steps","title":"mirai - Torch Integration","text":"Set dameons. Create serialization configuration, specifying ‘class’ ‘torch_tensor’ ‘vec’ TRUE. Use everywhere(), supplying configuration ‘.serial’ argument, (optionally) making torch package available daemons convenience.","code":"library(mirai) library(torch) daemons(1) #> [1] 1 cfg <- serial_config( class = \"torch_tensor\", sfunc = torch:::torch_serialize, ufunc = torch::torch_load, vec = TRUE ) everywhere(library(torch), .serial = cfg)"},{"path":"https://shikokuchuo.net/mirai/articles/torch.html","id":"example-usage","dir":"Articles","previous_headings":"Torch Integration","what":"Example Usage","title":"mirai - Torch Integration","text":"example creates convolutional neural network using torch::nn_module(). set model parameters also specified. model specification parameters passed initialized within ‘mirai’. returned model object containing many tensor elements. usual model parameters passed optimiser. can also initialized within ‘mirai’ process. , tensors complex objects containing tensors passed seamlessly host daemon processes, way R object. custom serialization mirai leverages R’s native ‘refhook’ mechanism allow completely transparent usage. Designed fast efficient, data copies minimised ‘official’ serialization methods torch package used directly.","code":"model <- nn_module( initialize = function(in_size, out_size) { self$conv1 <- nn_conv2d(in_size, out_size, 5) self$conv2 <- nn_conv2d(in_size, out_size, 5) }, forward = function(x) { x <- self$conv1(x) x <- nnf_relu(x) x <- self$conv2(x) x <- nnf_relu(x) x } ) params <- list(in_size = 1, out_size = 20) m <- mirai(do.call(model, params), model = model, params = params) m[] #> An `nn_module` containing 1,040 parameters. #> #> ── Modules ──────────────────────────────────────────────────────────────────────────────────────────────────────────── #> • conv1: #520 parameters #> • conv2: #520 parameters m$data$parameters$conv1.weight #> torch_tensor #> (1,1,.,.) = #> 0.1090 0.0691 -0.0591 -0.0461 -0.0532 #> -0.0153 0.1228 0.1182 -0.0665 0.0505 #> -0.0303 0.0163 -0.0647 -0.1798 -0.0441 #> -0.0588 0.0846 0.0857 0.0327 0.1972 #> 0.1442 0.0955 -0.1682 -0.0183 0.0960 #> #> (2,1,.,.) = #> -0.0650 0.0633 0.0921 -0.0372 0.1392 #> -0.0493 -0.0742 -0.1552 -0.0638 -0.0708 #> -0.0113 -0.1114 -0.0013 -0.0260 -0.0838 #> -0.0292 0.0165 -0.1340 -0.0556 0.0925 #> -0.0394 0.0905 0.1140 -0.1017 0.0363 #> #> (3,1,.,.) = #> -0.1762 0.0509 -0.1795 0.1617 -0.1282 #> 0.1735 -0.1951 -0.1044 0.1623 -0.1978 #> 0.1982 -0.1127 -0.1133 -0.0947 -0.0160 #> 0.1135 0.0198 -0.0254 0.0281 -0.0520 #> 0.0794 -0.0114 -0.1520 0.0267 -0.1980 #> #> (4,1,.,.) = #> 0.0412 0.0476 0.1843 -0.0444 0.0996 #> 0.0813 0.1186 0.1490 -0.1211 -0.0169 #> 0.0239 0.0793 -0.0484 0.0478 0.1343 #> 0.1461 0.1949 0.0382 0.0634 0.1292 #> 0.0958 0.0273 0.1933 0.0691 0.0905 #> #> (5,1,.,.) = #> -0.0724 0.1578 -0.0650 -0.0328 0.1338 #> ... [the output was truncated (use n=-1 to disable)] #> [ CPUFloatType{20,1,5,5} ][ requires_grad = TRUE ] optim <- mirai(optim_rmsprop(params = params), params = m$data$parameters) optim[] #> #> Inherits from: #> Public: #> add_param_group: function (param_group) #> clone: function (deep = FALSE) #> defaults: list #> initialize: function (params, lr = 0.01, alpha = 0.99, eps = 1e-08, weight_decay = 0, #> load_state_dict: function (state_dict, ..., .refer_to_state_dict = FALSE) #> param_groups: list #> state: State, R6 #> state_dict: function () #> step: function (closure = NULL) #> zero_grad: function () #> Private: #> step_helper: function (closure, loop_fun) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Charlie Gao. Author, maintainer. Joe Cheng. Contributor. Hibiki AI Limited. Copyright holder.","code":""},{"path":"https://shikokuchuo.net/mirai/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Gao C (2024). mirai: Minimalist Async Evaluation Framework R. R package version 1.3.0.9000, https://doi.org/10.5281/zenodo.7912722.","code":"@Manual{, title = {mirai: Minimalist Async Evaluation Framework for R}, author = {Charlie Gao}, year = {2024}, note = {R package version 1.3.0.9000}, url = {https://doi.org/10.5281/zenodo.7912722}, }"},{"path":[]},{"path":"https://shikokuchuo.net/mirai/index.html","id":"ミライ","dir":"","previous_headings":"","what":"ミライ","title":"Minimalist Async Evaluation Framework for R","text":"みらい 未来 Minimalist Async Evaluation Framework R Designed simplicity, ‘mirai’ evaluates R expression asynchronously parallel process, locally distributed network, result automatically available upon completion. Modern networking concurrency built nanonext NNG (Nanomsg Next Gen) ensure reliable efficient scheduling, fast inter-process communications TCP/IP secured TLS. Advantages include inherently queued thus handling many tasks available processes, storage file system, support otherwise non-exportable reference objects, event-driven promises implementation, built-asynchronous parallel map.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"quick-start","dir":"","previous_headings":"","what":"Quick Start","title":"Minimalist Async Evaluation Framework for R","text":"Use mirai() evaluate expression asynchronously separate, clean R process. following mimics expensive calculation eventually returns vector random values. mirai expression evaluated another process hence must self-contained, referring variables already exist . , variables n mean passed part mirai() call. ‘mirai’ object returned immediately - creating mirai never blocks session. Whilst async operation ongoing, attempting access mirai’s data yields ‘unresolved’ logical NA. check whether mirai remains unresolved (yet complete): wait collect return value, use mirai’s [] method: mirai represents async operation, never necessary wait . code can continue run. completes, return value automatically becomes available $data.","code":"library(mirai) m <- mirai({Sys.sleep(n); rnorm(n, mean)}, n = 5L, mean = 7) m #> < mirai [] > m$data #> 'unresolved' logi NA unresolved(m) #> [1] TRUE m[] #> [1] 6.558945 6.774610 6.776656 7.419766 6.670370 while (unresolved(m)) { # do work here that does not depend on 'm' } m #> < mirai [$data] > m$data #> [1] 6.558945 6.774610 6.776656 7.419766 6.670370"},{"path":"https://shikokuchuo.net/mirai/index.html","id":"daemons","dir":"","previous_headings":"Quick Start","what":"Daemons","title":"Minimalist Async Evaluation Framework for R","text":"Daemons persistent background processes receiving mirai requests, created easily : Daemons may also deployed remotely distributed computing launchers can start daemons across network via (tunnelled) SSH cluster resource manager. Secure TLS connections can used remote daemon connections, zero configuration required.","code":"daemons(4) #> [1] 4"},{"path":"https://shikokuchuo.net/mirai/index.html","id":"async-parallel-map","dir":"","previous_headings":"Quick Start","what":"Async Parallel Map","title":"Minimalist Async Evaluation Framework for R","text":"mirai_map() maps function list vector, element processed separate parallel process. also performs multiple map rows dataframe matrix. ‘mirai_map’ object returned immediately. code can continue run point. value may retrieved time using [] method return list, just like purrr::map() base::lapply(). [] method also provides options flatmap, early stopping /progress indicators. errors returned ‘errorValues’, facilitating recovery partial failure. advantages alternative map implementations.","code":"df <- data.frame( fruit = c(\"melon\", \"grapes\", \"coconut\"), price = c(3L, 5L, 2L) ) m <- mirai_map(df, sprintf, .args = list(fmt = \"%s: $%d\")) m #> < mirai map [3/3] > m[.flat] #> [1] \"melon: $3\" \"grapes: $5\" \"coconut: $2\""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"design-concepts","dir":"","previous_headings":"","what":"Design Concepts","title":"Minimalist Async Evaluation Framework for R","text":"mirai designed ground provide production-grade experience. 900x responsive common alternatives [1] Built low-latency applications real time inference Shiny apps Consistent behaviour reliance global options variables mirai call evaluated explicitly transparent predictable results Launch millions tasks simultaneously thousands connections Proven track record handling heavy-duty workloads life sciences industry   mirai パッケージを試してみたところ、かなり速くて驚きました","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"integrations","dir":"","previous_headings":"","what":"Integrations","title":"Minimalist Async Evaluation Framework for R","text":"following core integrations documented, usage examples linked vignettes:   Provides alternative communications backend R, implementing new parallel cluster type, feature request R-Core R Project Sprint 2023. ‘miraiCluster’ may also used foreach via doParallel.   Implements next generation completely event-driven, non-polling promises. ‘mirai’ may used interchageably ‘promises’, including promise pipe %...>%.   Asynchronous parallel / distributed backend, supporting next level responsiveness scalability within Shiny, native support ExtendedTask.   Asynchronous parallel / distributed backend scaling Plumber applications production.   Allows queries using Apache Arrow format handled seamlessly ADBC database connections hosted background processes.   Allows Torch tensors complex objects models optimizers used seamlessly across parallel processes.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"powering-crew-and-targets-high-performance-computing","dir":"","previous_headings":"","what":"Powering Crew and Targets High Performance Computing","title":"Minimalist Async Evaluation Framework for R","text":"  Targets, Make-like pipeline tool statistics data science, integrated adopted crew default high-performance computing backend.   Crew distributed worker-launcher extending mirai different distributed computing platforms, traditional clusters cloud services.   crew.cluster enables mirai-based workflows traditional high-performance computing clusters using LFS, PBS/TORQUE, SGE Slurm.   crew.aws.batch extends mirai cloud computing using AWS Batch.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"thanks","dir":"","previous_headings":"","what":"Thanks","title":"Minimalist Async Evaluation Framework for R","text":"like thank particular: Landau instrumental shaping development package, initiating original request persistent daemons, orchestrating robustness testing high performance computing requirements crew targets. Joe Cheng integrating promises method work seamlessly within Shiny, prototyping event-driven promises. Luke Tierney R Core, discussion L’Ecuyer-CMRG streams ensure statistical independence parallel processing, making possible mirai first ‘alternative communications backend R’. Henrik Bengtsson valuable insights leading interface accepting broader usage patterns. Daniel Falbel discussion around efficient solution serialization transmission torch tensors. Kirill Müller discussion using ‘daemons’ host Arrow database connections.   funding work TLS implementation nanonext, used provide secure connections mirai.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"installation","dir":"","previous_headings":"","what":"Installation","title":"Minimalist Async Evaluation Framework for R","text":"Install latest release CRAN: current development version available R-universe:","code":"install.packages(\"mirai\") install.packages(\"mirai\", repos = \"https://shikokuchuo.r-universe.dev\")"},{"path":"https://shikokuchuo.net/mirai/index.html","id":"links--references","dir":"","previous_headings":"","what":"Links & References","title":"Minimalist Async Evaluation Framework for R","text":"◈ mirai R package: https://shikokuchuo.net/mirai/ ◈ nanonext R package: https://shikokuchuo.net/nanonext/ mirai listed CRAN High Performance Computing Task View: https://cran.r-project.org/view=HighPerformanceComputing – Please note project released Contributor Code Conduct. participating project agree abide terms.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"Make Mirai Promise — as.promise.mirai","title":"Make Mirai Promise — as.promise.mirai","text":"Creates ‘promise’ ‘mirai’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Make Mirai Promise — as.promise.mirai","text":"","code":"# S3 method for class 'mirai' as.promise(x)"},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Make Mirai Promise — as.promise.mirai","text":"x object class ‘mirai’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Make Mirai Promise — as.promise.mirai","text":"‘promise’ object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Make Mirai Promise — as.promise.mirai","text":"function S3 method generic .promise class ‘mirai’. Requires promises package. Allows ‘mirai’ used promise pipe %...>%, schedules function run upon resolution ‘mirai’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Make Mirai Promise — as.promise.mirai","text":"","code":"if (interactive() && requireNamespace(\"promises\", quietly = TRUE)) { library(promises) p <- as.promise(mirai(\"example\")) print(p) is.promise(p) p2 <- mirai(\"completed\") %...>% identity() p2$then(cat) is.promise(p2) }"},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Call Value) — call_mirai","title":"mirai (Call Value) — call_mirai","text":"call_mirai waits ‘mirai’ resolve still progress, storing value $data, returns ‘mirai’ object. call_mirai_ variant call_mirai allows user interrupts, suitable interactive use.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Call Value) — call_mirai","text":"","code":"call_mirai(x) call_mirai_(x)"},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Call Value) — call_mirai","text":"x ‘mirai’ object, list ‘mirai’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Call Value) — call_mirai","text":"passed object (invisibly). ‘mirai’, retrieved value stored $data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Call Value) — call_mirai","text":"functions accept list ‘mirai’ objects, returned mirai_map well individual ‘mirai’. wait asynchronous operation(s) complete still progress (blocking). x[] may also used wait return value mirai x, equivalent call_mirai_(x)$data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"alternatively","dir":"Reference","previous_headings":"","what":"Alternatively","title":"mirai (Call Value) — call_mirai","text":"value ‘mirai’ may accessed time $data, yet resolve, ‘unresolved’ logical NA returned instead. Using unresolved ‘mirai’ returns TRUE yet resolve FALSE otherwise. suitable use control flow statements .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"mirai (Call Value) — call_mirai","text":"error occurs evaluation, error message returned character string class ‘miraiError’ ‘errorValue’ (stack trace available $stack.trace error object). is_mirai_error may used test . daemon crashes terminates unexpectedly evaluation, ‘errorValue’ 19 (Connection reset) returned (using dispatcher using dispatcher retry = FALSE). Otherwise, using dispatcher retry = TRUE, mirai remain unresolved automatically re-tried next daemon connect particular instance. cancel task instead, use saisei(force = TRUE) (see saisei). is_error_value tests error conditions including ‘mirai’ errors, interrupts, timeouts.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Call Value) — call_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # using call_mirai() df1 <- data.frame(a = 1, b = 2) df2 <- data.frame(a = 3, b = 1) m <- mirai(as.matrix(rbind(df1, df2)), df1 = df1, df2 = df2, .timeout = 1000) call_mirai(m)$data # using unresolved() m <- mirai( { res <- rnorm(n) res / rev(res) }, n = 1e6 ) while (unresolved(m)) { cat(\"unresolved\\n\") Sys.sleep(0.1) } str(m$data) }"},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Collect Value) — collect_mirai","title":"mirai (Collect Value) — collect_mirai","text":"collect_mirai waits ‘mirai’ resolve still progress, returns value directly. efifcient version equivalent call_mirai(x)$data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Collect Value) — collect_mirai","text":"","code":"collect_mirai(x)"},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Collect Value) — collect_mirai","text":"x ‘mirai’ object, list ‘mirai’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Collect Value) — collect_mirai","text":"object (return value ‘mirai’), list objects (length ‘x’, preserving names).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Collect Value) — collect_mirai","text":"function wait asynchronous operation(s) complete still progress (blocking), interruptible. x[] may used wait return value mirai x, user-interruptible counterpart collect_mirai(x).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"alternatively","dir":"Reference","previous_headings":"","what":"Alternatively","title":"mirai (Collect Value) — collect_mirai","text":"value ‘mirai’ may accessed time $data, yet resolve, ‘unresolved’ logical NA returned instead. Using unresolved ‘mirai’ returns TRUE yet resolve FALSE otherwise. suitable use control flow statements .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"mirai (Collect Value) — collect_mirai","text":"error occurs evaluation, error message returned character string class ‘miraiError’ ‘errorValue’ (stack trace available $stack.trace error object). is_mirai_error may used test . daemon crashes terminates unexpectedly evaluation, ‘errorValue’ 19 (Connection reset) returned (using dispatcher using dispatcher retry = FALSE). Otherwise, using dispatcher retry = TRUE, mirai remain unresolved automatically re-tried next daemon connect particular instance. cancel task instead, use saisei(force = TRUE) (see saisei). is_error_value tests error conditions including ‘mirai’ errors, interrupts, timeouts.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Collect Value) — collect_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # using collect_mirai() df1 <- data.frame(a = 1, b = 2) df2 <- data.frame(a = 3, b = 1) m <- mirai(as.matrix(rbind(df1, df2)), df1 = df1, df2 = df2, .timeout = 1000) collect_mirai(m) # using x[] m[] }"},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":null,"dir":"Reference","previous_headings":"","what":"Daemon Instance — daemon","title":"Daemon Instance — daemon","text":"Starts execution daemon receive mirai requests. Awaits data, evaluates expression environment containing supplied data, returns value host caller. Daemon settings may controlled daemons function need invoked directly, unless deploying manually remote resources.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Daemon Instance — daemon","text":"","code":"daemon( url, asyncdial = FALSE, autoexit = TRUE, cleanup = TRUE, output = FALSE, maxtasks = Inf, idletime = Inf, walltime = Inf, timerstart = 0L, ..., tls = NULL, rs = NULL )"},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Daemon Instance — daemon","text":"url character host dispatcher URL dial , including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. asyncdial [default FALSE] whether perform dials asynchronously. default FALSE error connection immediately possible (instance daemons yet called host, specified port open etc.). Specifying TRUE continues retrying (indefinitely) immediately successful, resilient can mask potential connection issues. autoexit [default TRUE] logical value, whether daemon exit automatically socket connection ends. signal tools package, tools::SIGINT, equivalent integer value supplied, signal additionally raised exit (see 'Persistence' section ). cleanup [default TRUE] logical value, whether perform cleanup global environment restore loaded packages options initial state evaluation. granular control, also accepts integer value (see ‘Cleanup Options’ section ). output [default FALSE] logical value, output generated stdout / stderr TRUE, else discard FALSE. Specify TRUE ‘...’ argument daemons launch_local provide redirection output host process (applicable local daemons). maxtasks [default Inf] maximum number tasks execute (task limit) exiting. idletime [default Inf] maximum idle time, since completion last task (milliseconds) exiting. walltime [default Inf] soft walltime, minimum amount real time taken (milliseconds) exiting. timerstart [default 0L] number completed tasks start timer ‘idletime’ ‘walltime’. 0L implies timers started upon launch. ... reserved currently used. tls [default NULL] required secure TLS connections 'tls+tcp://' 'wss://'. Either character path file containing X.509 certificate(s) PEM format, comprising certificate authority certificate chain starting TLS certificate ending CA certificate, length 2 character vector comprising [] certificate authority certificate chain [ii] empty string ''. rs [default NULL] initial value .Random.seed. set automatically using L'Ecuyer-CMRG RNG streams generated host process independently supplied.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Daemon Instance — daemon","text":"Invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Daemon Instance — daemon","text":"network topology daemons dial host dispatcher, listens ‘url’ address. way, network resources may added removed dynamically host dispatcher automatically distributes tasks available daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"persistence","dir":"Reference","previous_headings":"","what":"Persistence","title":"Daemon Instance — daemon","text":"‘autoexit’ argument governs persistence settings daemon. default TRUE ensures exit cleanly socket connection ended. Instead TRUE, supplying signal tools package, tools::SIGINT, equivalent integer value, sets signal raised socket connection ends. instance, supplying SIGINT allows potentially immediate exit interrupting ongoing evaluation rather letting complete. Setting FALSE allows daemon persist indefinitely even longer socket connection. allows host session end new session connect URL daemon dialled . Daemons must terminated daemons(NULL) case, sends explicit exit instructions connected daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"cleanup-options","dir":"Reference","previous_headings":"","what":"Cleanup Options","title":"Daemon Instance — daemon","text":"‘cleanup’ argument also accepts integer value, operates additive bitmask: perform cleanup global environment (1L), reset loaded packages initial state (2L), restore options initial state (4L), perform garbage collection (8L). example, perform cleanup global environment garbage collection, specify 9L (1L + 8L). default argument value TRUE performs actions apart garbage collection equivalent value 7L. Caution: reset options loaded packages packages set options load.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":null,"dir":"Reference","previous_headings":"","what":"Daemons (Set Persistent Processes) — daemons","title":"Daemons (Set Persistent Processes) — daemons","text":"Set ‘daemons’ persistent background processes receive mirai requests. Specify ‘n’ create daemons local machine. Specify ‘url’ receiving connections remote daemons (distributed computing across network). Specify ‘remote’ optionally launch remote daemons via remote configuration. default, dispatcher ensures optimal scheduling.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Daemons (Set Persistent Processes) — daemons","text":"","code":"daemons( n, url = NULL, remote = NULL, dispatcher = c(\"process\", \"thread\", \"none\"), ..., force = TRUE, seed = NULL, tls = NULL, pass = NULL, .compute = \"default\" )"},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Daemons (Set Persistent Processes) — daemons","text":"n integer number daemons set. url [default NULL] specified, character URL vector URLs host remote daemons dial , including port accepting incoming connections (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. Specify URL starting 'tls+tcp://' 'wss://' use secure TLS connections. Auxiliary function host_url may used construct valid host URL. remote [default NULL] required launching remote daemons, configuration generated remote_config ssh_config. dispatcher [default 'process'] character value, one ‘process’, ‘thread’ ‘none’. Whether deploy dispatcher another process, thread . Dispatcher extension ensures optimal scheduling, although always required (details see Dispatcher section ). Note option ‘thread’ new currently considered experimental. ... (optional) additional arguments passed dispatcher using dispatcher /daemon launching daemons. include ‘retry’ ‘token’ dispatcher ‘asyncdial’, ‘autoexit’, ‘cleanup’, ‘output’, ‘maxtasks’, ‘idletime’, ‘walltime’ ‘timerstart’ daemon. force [default TRUE] logical value whether always reset daemons apply new settings compute profile, even already set. FALSE, applying new settings requires daemons explicitly reset first using daemons(0). seed [default NULL] (optional) supply random seed (single value, interpreted integer). used inititalise L'Ecuyer-CMRG RNG streams sent daemon. Note reproducible results can expected dispatcher = 'none', unpredictable timing task completions otherwise influence tasks sent daemon. Even dispatcher = 'none', reproducibility guaranteed order tasks sent deterministic. tls [default NULL] (optional secure TLS connections) supplied, zero-configuration single-use keys certificates automatically generated. supplied, either character path file containing PEM-encoded TLS certificate associated private key (may contain additional certificates leading validation chain, TLS certificate first), length 2 character vector comprising [] TLS certificate (optionally certificate chain) [ii] associated private key. pass [default NULL] (required private key supplied ‘tls’ encrypted password) security, provided function returns value, rather directly. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Daemons (Set Persistent Processes) — daemons","text":"using dispatcher, integer number daemons set, else integer number daemons launched locally (zero using remote launcher).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Daemons (Set Persistent Processes) — daemons","text":"Use daemons(0) reset daemon connections: connected daemons /dispatchers exit automatically. mirai reverts default behaviour creating new background process request. unresolved ‘mirai’ return ‘errorValue’ 19 (Connection reset) reset. Calling daemons revised (even ) settings compute profile resets daemons applying new settings force = TRUE. host session ends, connected dispatcher daemon processes automatically exit soon connections dropped (unless daemons started autoexit = FALSE). daemon processing task, exit soon task complete. reset persistent daemons started autoexit = FALSE, use daemons(NULL) instead, also sends exit instructions connected daemons prior resetting. historical reasons, daemons() arguments returns value status.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"local-daemons","dir":"Reference","previous_headings":"","what":"Local Daemons","title":"Daemons (Set Persistent Processes) — daemons","text":"Daemons provide potentially efficient solution asynchronous operations new processes longer need created ad hoc basis. Supply argument ‘n’ set number daemons. New background daemon processes automatically created local machine connecting back host process, either directly via dispatcher.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"dispatcher","dir":"Reference","previous_headings":"","what":"Dispatcher","title":"Daemons (Set Persistent Processes) — daemons","text":"default dispatcher = \"process\" launches background process running dispatcher. Dispatcher connects daemons behalf host ensures optimal FIFO scheduling tasks. Specifying dispatcher = \"thread\" runs dispatcher logic new thread, faster efficient alternative using separate process. new feature considered experimental. Specifying dispatcher = \"none\", uses default behaviour without additional dispatcher logic. case daemons connect directly host tasks distributed round-robin fashion. Optimal scheduling guaranteed duration tasks known priori, hence tasks can queued one daemon daemons remain idle. However, provides resource-light approach, suited working similar-length tasks, concurrent tasks typically exceed available daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"distributed-computing","dir":"Reference","previous_headings":"","what":"Distributed Computing","title":"Daemons (Set Persistent Processes) — daemons","text":"Specifying ‘url’ allows tasks distributed across network. character string ‘tcp://10.75.32.70:5555’ daemon processes connect . Switching URL scheme ‘tls+tcp://’ ‘wss://’ automatically upgrades connection use TLS. auxiliary function host_url may used automatically construct valid host URL based computer's hostname. Specify ‘remote’ call remote_config ssh_config launch daemons remote machines. Otherwise, launch_remote may used generate shell commands deploy daemons manually remote resources. IPv6 addresses also supported must enclosed square brackets [ ] avoid confusion final colon separating port. example, port 5555 IPv6 loopback address ::1 specified ‘tcp://[::1]:5555’. Specifying wildcard value zero port number e.g. ‘tcp://[::1]:0’ ‘ws://[::1]:0’ automatically assign free ephemeral port. Use status inspect actual assigned port time. Dispatcher using dispatcher, recommended use websocket URL rather TCP, requires one port connect daemons: websocket URL supports path port number, can made unique daemon. Specifying single host URL ‘ws://10.75.32.70:5555’ n = 6 automatically append sequence path, listening URLs ‘ws://10.75.32.70:5555/1’ ‘ws://110.75.32.70:5555/6’. Alternatively, specify vector URLs listen arbitrary port numbers / paths. case optional supply ‘n’ can inferred length vector supplied. Individual daemons dial host URLs. one daemon can dialled URL given time. Dispatcher automatically adjusts number daemons actually connected. Hence possible dynamically scale number daemons required, subject maximum number initially specified. Alternatively, supplying single TCP URL listen block URLs ports starting supplied port number incrementing one ‘n’ specified e.g. host URL ‘tcp://10.75.32.70:5555’ n = 6 listens contiguous block ports 5555 5560. Without Dispatcher TCP URL may used case host listens one address, utilising single port. network topology daemons (started daemon) indeed dispatchers (started dispatcher) dial host URL. ‘n’ required case, disregarded supplied, network resources may added removed time. host automatically distributes tasks connected daemons dispatchers round-robin fashion.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"compute-profiles","dir":"Reference","previous_headings":"","what":"Compute Profiles","title":"Daemons (Set Persistent Processes) — daemons","text":"default, ‘default’ compute profile used. Providing character value ‘.compute’ creates new compute profile name specified. compute profile retains daemons settings, may operated independently . usage examples follow: local / remote daemons may set host URL specifying ‘.compute’ ‘remote’, creates new compute profile. Subsequent mirai calls may sent local computation specifying ‘.compute’ argument, remote computation connected daemons specifying ‘.compute’ argument ‘remote’. cpu / gpu tasks may require access different types daemon, GPUs. case, daemons() may called set host URLs CPU-daemons GPUs, specifying ‘.compute’ argument ‘cpu’ ‘gpu’ respectively. supplying ‘.compute’ argument subsequent mirai calls, tasks may sent either ‘cpu’ ‘gpu’ daemons appropriate. Note: actions resetting daemons via daemons(0) carried desired ‘.compute’ argument specified.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Daemons (Set Persistent Processes) — daemons","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # Create 2 local daemons (using dispatcher) daemons(2) status() # Reset to zero daemons(0) # Create 2 local daemons (not using dispatcher) daemons(2, dispatcher = \"none\") status() # Reset to zero daemons(0) # 2 remote daemons via dispatcher using WebSockets daemons(2, url = host_url(ws = TRUE)) status() # Reset to zero daemons(0) # Set host URL for remote daemons to dial into daemons(url = host_url(), dispatcher = \"none\") status() # Reset to zero daemons(0) # Use with() to evaluate with daemons for the duration of the expression with( daemons(2), { m1 <- mirai(Sys.getpid()) m2 <- mirai(Sys.getpid()) cat(call_mirai(m1)$data, call_mirai(m2)$data, \"\\n\") } ) } if (FALSE) { # \\dontrun{ # Launch 2 daemons on remotes 'nodeone' and 'nodetwo' using SSH # connecting back directly to the host URL over a TLS connection: daemons(url = host_url(tls = TRUE), remote = ssh_config(c('ssh://nodeone', 'ssh://nodetwo')), dispatcher = \"none\") # Launch 4 daemons on the remote machine 10.75.32.90 using SSH tunnelling # over port 5555 ('url' hostname must be 'localhost' or '127.0.0.1'): daemons(n = 4, url = 'ws://localhost:5555', remote = ssh_config('ssh://10.75.32.90', tunnel = TRUE)) } # }"},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":null,"dir":"Reference","previous_headings":"","what":"Dispatcher — dispatcher","title":"Dispatcher — dispatcher","text":"Dispatches tasks host daemons processing, using FIFO scheduling, queuing tasks required. Daemon / dispatcher settings may controlled daemons function need invoked directly.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Dispatcher — dispatcher","text":"","code":"dispatcher( host, url = NULL, n = NULL, ..., retry = FALSE, token = FALSE, tls = NULL, pass = NULL, rs = NULL, monitor = NULL )"},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Dispatcher — dispatcher","text":"host character host URL dial (tasks sent ), including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. url (optional) character URL vector URLs dispatcher listen , including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. Specify 'tls+tcp://' 'wss://' use secure TLS connections. Tasks sent daemons dialled URLs. supplied, ‘n’ local inter-process URLs assigned automatically. n (optional) specified, integer number daemons listen . Otherwise ‘n’ inferred number URLs supplied ‘url’. single URL supplied ‘n’ > 1, ‘n’ unique URLs automatically assigned daemons dial . ... (optional) additional arguments passed daemon. include ‘asyncdial’, ‘autoexit’, ‘cleanup’, ‘maxtasks’, ‘idletime’, ‘walltime’ ‘timerstart’. retry [default FALSE] logical value, whether automatically retry tasks daemon crashes terminates unexpectedly next daemon instance connect. TRUE, mirai remain unresolved status show ‘online’ 0 ‘assigned’ > ‘complete’. cancel task case, use saisei(force = TRUE). FALSE, tasks returned ‘errorValue’ 19 (Connection reset). token [default FALSE] TRUE, appends unique 24-character token URL path dispatcher listens (applicable TCP URLs accept path). tls [default NULL] (required secure TLS connections) either character path file containing PEM-encoded TLS certificate associated private key (may contain additional certificates leading validation chain, TLS certificate first), length 2 character vector comprising [] TLS certificate (optionally certificate chain) [ii] associated private key. pass [default NULL] (required private key supplied ‘tls’ encrypted password) security, provided function returns value, rather directly. rs [default NULL] initial value .Random.seed. set automatically using L'Ecuyer-CMRG RNG streams generated host process independently supplied. monitor (package internal use ) set parameter.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Dispatcher — dispatcher","text":"Invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Dispatcher — dispatcher","text":"network topology dispatcher acts gateway host daemons, ensuring tasks received host dispatched FIFO basis processing. Tasks queued dispatcher ensure tasks sent daemons can begin immediate execution task.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai Map Options — .flat","title":"mirai Map Options — .flat","text":"Expressions insert [] method ‘mirai_map’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai Map Options — .flat","text":"","code":".flat .progress .progress_cli .stop"},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":"format","dir":"Reference","previous_headings":"","what":"Format","title":"mirai Map Options — .flat","text":"object class bytecode length 1. object class bytecode length 1. object class bytecode length 1. object class bytecode length 1.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":"results","dir":"Reference","previous_headings":"","what":"Results","title":"mirai Map Options — .flat","text":"x[] collects results ‘mirai_map’ x returns list. wait asynchronous operations complete still progress, blocking user-interruptible. x[.flat] collects flattens map results vector, checking type avoid coercion. Note: errors ‘errorValue’ returned results differing type. x[.progress] collects map results whilst showing simple text progress indicator parts completed total. x[.progress_cli] collects map results whilst showing progress bar cli package, available, completion percentage ETA. x[.stop] collects map results applying early stopping, stops first failure cancels remaining operations. Note: operations already progress continue completion, although results collected. options may combined manner : x[.stop, .progress] applies early stopping together progress indicator.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":null,"dir":"Reference","previous_headings":"","what":"Evaluate Everywhere — everywhere","title":"Evaluate Everywhere — everywhere","text":"Evaluate expression ‘everywhere’ connected daemons specified compute profile. Designed performing setup operations across daemons loading packages, exporting common data, registering custom serialization functions. Resultant changes global environment, loaded packages options persisted regardless daemon's ‘cleanup’ setting.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Evaluate Everywhere — everywhere","text":"","code":"everywhere(.expr, ..., .args = list(), .serial = NULL, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Evaluate Everywhere — everywhere","text":".expr expression evaluate asynchronously (arbitrary length, wrapped { } necessary), else pre-constructed language object. ... (optional) either named arguments (name = value pairs) specifying objects referenced, defined, ‘.expr’, environment containing objects. See ‘evaluation’ section . .args (optional) either named list specifying objects referenced, defined, ‘.expr’, environment containing objects. objects remain local evaluation environment opposed supplied ‘...’ - see ‘evaluation’ section . .serial [default NULL] (optional) configuration created serial_config register serialization unserialization functions normally non-exportable reference objects, Arrow Tables torch tensors. Updating new configuration replaces existing registered functions. remove configuration, provide empty list. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Evaluate Everywhere — everywhere","text":"Invisible NULL. error specified compute profile found, .e. yet set .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"evaluation","dir":"Reference","previous_headings":"","what":"Evaluation","title":"Evaluate Everywhere — everywhere","text":"expression ‘.expr’ evaluated separate R process clean environment (global environment), consisting objects list environment supplied ‘.args’, named objects passed ‘...’ (environment one supplied) assigned global environment process. evaluation occur global environment, supply objects ‘...’ rather ‘.args’. stricter scoping, use ‘.args’, limits, example, variables explicitly passed arguments functions found. evaluation occurs clean environment, undefined objects must supplied though ‘...’ /‘.args’, including self-defined functions. Functions package use namespaced calls mirai::mirai(), else package loaded beforehand part ‘.expr’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Evaluate Everywhere — everywhere","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1) # export common data by a super-assignment expression: everywhere(y <<- 3) # '...' variables are assigned to the global environment # '.expr' may be specified as an empty {} in such cases: everywhere({}, a = 1, b = 2) m <- mirai(a + b - y == 0L) call_mirai(m)$data daemons(0) # loading a package on all daemons and also # registering custom serialization functions: cfg <- serial_config(\"cls_name\", function(x) serialize(x, NULL), unserialize) daemons(1, dispatcher = \"none\") everywhere(library(parallel), .serial = cfg) m <- mirai(\"package:parallel\" %in% search()) call_mirai(m)$data daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":null,"dir":"Reference","previous_headings":"","what":"URL Constructors — host_url","title":"URL Constructors — host_url","text":"host_url constructs valid host URL (daemons may connect) based computer's hostname. may supplied directly ‘url’ argument daemons. local_url constructs random URL suitable local daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"URL Constructors — host_url","text":"","code":"host_url(ws = FALSE, tls = FALSE, port = 0) local_url()"},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"URL Constructors — host_url","text":"ws [default FALSE] logical value whether use WebSockets 'ws://' else TCP 'tcp://' scheme. tls [default FALSE] logical value whether use TLS case scheme used either 'wss://' 'tls+tcp://' accordingly. port [default 0] numeric port use. open connections network addresses daemons connecting . ‘0’ wildcard value automatically assigns free ephemeral port.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"URL Constructors — host_url","text":"character string comprising valid URL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"URL Constructors — host_url","text":"host_url relies using host name computer rather IP address typically works local networks, although always guaranteed. unsuccessful, substitute IPv4 IPv6 address place hostname. local_url generates random URL platform's default inter-process communications transport: abstract Unix domain sockets Linux, Unix domain sockets MacOS, Solaris POSIX platforms, named pipes Windows.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"URL Constructors — host_url","text":"","code":"host_url() #> [1] \"tcp://fv-az569-105:0\" host_url(ws = TRUE) #> [1] \"ws://fv-az569-105:0\" host_url(tls = TRUE) #> [1] \"tls+tcp://fv-az569-105:0\" host_url(ws = TRUE, tls = TRUE, port = 5555) #> [1] \"wss://fv-az569-105:5555\" local_url() #> [1] \"abstract://62806b3c386ff17f3d95d156\""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"Is mirai / mirai_map — is_mirai","title":"Is mirai / mirai_map — is_mirai","text":"object ‘mirai’ ‘mirai_map’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Is mirai / mirai_map — is_mirai","text":"","code":"is_mirai(x) is_mirai_map(x)"},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Is mirai / mirai_map — is_mirai","text":"x object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Is mirai / mirai_map — is_mirai","text":"Logical TRUE ‘x’ class ‘mirai’ ‘mirai_map’ respectively, FALSE otherwise.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Is mirai / mirai_map — is_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1, dispatcher = \"none\") df <- data.frame() m <- mirai(as.matrix(df), df = df) is_mirai(m) is_mirai(df) mp <- mirai_map(1:3, runif) is_mirai_map(mp) is_mirai_map(mp[]) daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":null,"dir":"Reference","previous_headings":"","what":"Error Validators — is_mirai_error","title":"Error Validators — is_mirai_error","text":"Validator functions error value types created mirai.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Error Validators — is_mirai_error","text":"","code":"is_mirai_error(x) is_mirai_interrupt(x) is_error_value(x)"},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Error Validators — is_mirai_error","text":"x object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Error Validators — is_mirai_error","text":"Logical value TRUE FALSE.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Error Validators — is_mirai_error","text":"object ‘miraiError’. execution ‘mirai’ process fails, error message returned character string class ‘miraiError’ ‘errorValue’. stack trace available $stack.trace error object. object ‘miraiInterrupt’. ongoing ‘mirai’ sent user interrupt, resolve empty character string classed ‘miraiInterrupt’ ‘errorValue’. object ‘errorValue’, ‘mirai’ timeout, ‘miraiError’ ‘miraiInterrupt’. catch-condition includes returned error values.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Error Validators — is_mirai_error","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions m <- mirai(stop()) call_mirai(m) is_mirai_error(m$data) is_mirai_interrupt(m$data) is_error_value(m$data) m$data$stack.trace m2 <- mirai(Sys.sleep(1L), .timeout = 100) call_mirai(m2) is_mirai_error(m2$data) is_mirai_interrupt(m2$data) is_error_value(m2$data) }"},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":null,"dir":"Reference","previous_headings":"","what":"Launch Daemon — launch_local","title":"Launch Daemon — launch_local","text":"launch_local spawns new background Rscript process calling daemon specified arguments. launch_remote returns shell command deploying daemons character vector. configuration generated remote_config ssh_config supplied used launch daemon remote machine.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Launch Daemon — launch_local","text":"","code":"launch_local(url, ..., tls = NULL, .compute = \"default\") launch_remote( url, remote = remote_config(), ..., tls = NULL, .compute = \"default\" )"},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Launch Daemon — launch_local","text":"url character host URL vector host URLs, including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path' integer index value, vector index values, dispatcher URLs, 1L host URL (using dispatcher). launch_remote , ‘miraiCluster’ ‘miraiNode’. ... (optional) additional arguments passed daemon. include ‘autoexit’, ‘cleanup’, ‘output’, ‘maxtasks’, ‘idletime’, ‘walltime’ ‘timerstart’. tls [default NULL] required secure TLS connections tls+tcp wss. Zero-configuration TLS certificates generated daemons automatically passed daemon, without requiring specified . Otherwise, supply either character path file containing X.509 certificate(s) PEM format, comprising certificate authority certificate chain, length 2 character vector comprising [] certificate authority certificate chain [ii] empty character ”. .compute [default 'default'] character value compute profile use (compute profile independent set daemons). remote required launching remote daemons, configuration generated remote_config ssh_config. empty remote_config effect daemon launches returns shell commands deploying manually remote machines.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Launch Daemon — launch_local","text":"launch_local: Invisible NULL. launch_remote: character vector daemon launch commands, classed ‘miraiLaunchCmd’. printed output may copy / pasted directly remote machine.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Launch Daemon — launch_local","text":"functions may used re-launch daemons exited reaching time task limits. Daemons must already set launchers work. generated command contains argument ‘rs’ specifying length 7 L'Ecuyer-CMRG random seed supplied daemon. values different time function called.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Launch Daemon — launch_local","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(url = host_url(ws = TRUE), dispatcher = \"none\") status() launch_local(status()$daemons, maxtasks = 10L) launch_remote(1L, maxtasks = 10L) Sys.sleep(1) status() daemons(0) daemons(n = 2L, url = host_url(tls = TRUE)) status() launch_local(1:2, idletime = 60000L, timerstart = 1L) launch_remote(1:2, idletime = 60000L, timerstart = 1L) Sys.sleep(1) status() daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":null,"dir":"Reference","previous_headings":"","what":"Make Mirai Cluster — make_cluster","title":"Make Mirai Cluster — make_cluster","text":"make_cluster creates cluster type ‘miraiCluster’, may used cluster object function parallel base package clusterApply parLapply. stop_cluster stops cluster created make_cluster.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Make Mirai Cluster — make_cluster","text":"","code":"make_cluster(n, url = NULL, remote = NULL, ...) stop_cluster(cl)"},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Make Mirai Cluster — make_cluster","text":"n integer number nodes (automatically launched local machine unless ‘url’ supplied). url [default NULL] (specify remote nodes) character URL host remote nodes dial , including port accepting incoming connections, e.g. 'tcp://10.75.37.40:5555'. Specify URL scheme ‘tls+tcp://’ use secure TLS connections. remote [default NULL] (specify launch remote nodes) remote launch configuration generated remote_config ssh_config. supplied, nodes may deployed manually remote resources. ... additional arguments passed onto daemons. cl ‘miraiCluster’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Make Mirai Cluster — make_cluster","text":"make_cluster: object class ‘miraiCluster’ ‘cluster’. ‘miraiCluster’ automatically assigned ID ‘n’ nodes class ‘miraiNode’. ‘url’ supplied ‘remote’, shell commands deployment nodes remote resources printed console. stop_cluster: invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"note","dir":"Reference","previous_headings":"","what":"Note","title":"Make Mirai Cluster — make_cluster","text":"default behaviour clusters created function designed map closely possible clusters created parallel package. However, ‘...’ arguments passed onto daemons additional customisation desired, although resultant behaviour may always supported.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"remote-nodes","dir":"Reference","previous_headings":"","what":"Remote Nodes","title":"Make Mirai Cluster — make_cluster","text":"Specify ‘url’ ‘n’ set host connection remote nodes dial . ‘n’ defaults one specified. Also specify ‘remote’ launch nodes using configuration generated remote_config ssh_config. case, number nodes inferred configuration provided ‘n’ disregarded. ‘remote’ supplied, shell commands deploying nodes manually remote resources automatically printed console. launch_remote may called time ‘miraiCluster’ return shell commands deployment nodes, ‘miraiNode’ return command single node.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"status","dir":"Reference","previous_headings":"","what":"Status","title":"Make Mirai Cluster — make_cluster","text":"Call status ‘miraiCluster’ check number currently active connections well host URL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"Make Mirai Cluster — make_cluster","text":"Errors thrown ‘parallel’ mechanism one nodes failed (quit unexpectedly). resulting ‘errorValue’ returned 19 (Connection reset). types error, e.g. evaluation, result usual ‘miraiError’ returned.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Make Mirai Cluster — make_cluster","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions cl <- make_cluster(2) cl cl[[1L]] Sys.sleep(0.5) status(cl) stop_cluster(cl) }"},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"Designed simplicity, 'mirai' evaluates R expression asynchronously parallel process, locally distributed network, result automatically available upon completion. Modern networking concurrency built 'nanonext' 'NNG' (Nanomsg Next Gen) ensure reliable efficient scheduling, fast inter-process communications TCP/IP secured TLS. Advantages include inherently queued thus handling many tasks available processes, storage file system, support otherwise non-exportable reference objects, event-driven promises implementation, built-asynchronous parallel map.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":"notes","dir":"Reference","previous_headings":"","what":"Notes","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"local mirai requests, default transport inter-process communications platform-dependent: abstract Unix domain sockets Linux, Unix domain sockets MacOS, Solaris POSIX platforms, named pipes Windows. may overriden, desired, specifying 'url' daemons interface launching daemons using launch_local.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":"reference-manual","dir":"Reference","previous_headings":"","what":"Reference Manual","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"vignette(\"mirai\", package = \"mirai\")","code":""},{"path":[]},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":"author","dir":"Reference","previous_headings":"","what":"Author","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"Charlie Gao charlie.gao@shikokuchuo.net (ORCID)","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Evaluate Async) — mirai","title":"mirai (Evaluate Async) — mirai","text":"Evaluate expression asynchronously new background R process persistent daemon (local remote). function return immediately ‘mirai’, resolve evaluated result complete.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Evaluate Async) — mirai","text":"","code":"mirai(.expr, ..., .args = list(), .timeout = NULL, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Evaluate Async) — mirai","text":".expr expression evaluate asynchronously (arbitrary length, wrapped { } necessary), else pre-constructed language object. ... (optional) either named arguments (name = value pairs) specifying objects referenced, defined, ‘.expr’, environment containing objects. See ‘evaluation’ section . .args (optional) either named list specifying objects referenced, defined, ‘.expr’, environment containing objects. objects remain local evaluation environment opposed supplied ‘...’ - see ‘evaluation’ section . .timeout [default NULL] timeout, integer value milliseconds. mirai resolve ‘errorValue’ 5 (timed ) evaluation exceeds limit. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Evaluate Async) — mirai","text":"‘mirai’ object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Evaluate Async) — mirai","text":"function return ‘mirai’ object immediately. value mirai may accessed time $data, yet resolve, ‘unresolved’ logical NA returned instead. unresolved may used mirai, returning TRUE ‘mirai’ yet resolve FALSE otherwise. suitable use control flow statements . Alternatively, call (wait ) result, use call_mirai returned ‘mirai’. block result returned. Specify ‘.compute’ send mirai using specific compute profile (previously created daemons), otherwise leave ‘default’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"evaluation","dir":"Reference","previous_headings":"","what":"Evaluation","title":"mirai (Evaluate Async) — mirai","text":"expression ‘.expr’ evaluated separate R process clean environment (global environment), consisting objects list environment supplied ‘.args’, named objects passed ‘...’ (environment one supplied) assigned global environment process. evaluation occur global environment, supply objects ‘...’ rather ‘.args’. stricter scoping, use ‘.args’, limits, example, variables explicitly passed arguments functions found. evaluation occurs clean environment, undefined objects must supplied though ‘...’ /‘.args’, including self-defined functions. Functions package use namespaced calls mirai::mirai(), else package loaded beforehand part ‘.expr’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"mirai (Evaluate Async) — mirai","text":"error occurs evaluation, error message returned character string class ‘miraiError’ ‘errorValue’ (stack trace available $stack.trace error object). is_mirai_error may used test . daemon crashes terminates unexpectedly evaluation, ‘errorValue’ 19 (Connection reset) returned (using dispatcher using dispatcher retry = FALSE). Otherwise, using dispatcher retry = TRUE, mirai remain unresolved automatically re-tried next daemon connect particular instance. cancel task instead, use saisei(force = TRUE) (see saisei). is_error_value tests error conditions including ‘mirai’ errors, interrupts, timeouts.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Evaluate Async) — mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # specifying objects via '...' n <- 3 m <- mirai(x + y + 2, x = 2, y = n) m m$data Sys.sleep(0.2) m$data # passing the calling environment to '...' df1 <- data.frame(a = 1, b = 2) df2 <- data.frame(a = 3, b = 1) m <- mirai(as.matrix(rbind(df1, df2)), environment(), .timeout = 1000) call_mirai(m)$data # using unresolved() m <- mirai( { res <- rnorm(n) res / rev(res) }, n = 1e6 ) while (unresolved(m)) { cat(\"unresolved\\n\") Sys.sleep(0.1) } str(m$data) # evaluating scripts using source() in '.expr' n <- 10L file <- tempfile() cat(\"r <- rnorm(n)\", file = file) m <- mirai({source(file); r}, file = file, n = n) call_mirai(m)[[\"data\"]] unlink(file) # use source(local = TRUE) when passing in local variables via '.args' n <- 10L file <- tempfile() cat(\"r <- rnorm(n)\", file = file) m <- mirai({source(file, local = TRUE); r}, .args = list(file = file, n = n)) call_mirai(m)[[\"data\"]] unlink(file) # passing a language object to '.expr' and a named list to '.args' expr <- quote(a + b + 2) args <- list(a = 2, b = 3) m <- mirai(.expr = expr, .args = args) call_mirai(m)$data }"},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai Map — mirai_map","title":"mirai Map — mirai_map","text":"Asynchronous parallel map function list vector using mirai, optional promises integration. Performs multiple map rows dataframe matrix.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai Map — mirai_map","text":"","code":"mirai_map(.x, .f, ..., .args = list(), .promise = NULL, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai Map — mirai_map","text":".x list atomic vector. Also accepts matrix dataframe, case multiple map performed rows. .f function applied element .x, row .x case may . ... (optional) named arguments (name = value pairs) specifying objects referenced, defined, .f. .args (optional) constant arguments .f, provided list. .promise (optional) supplied, registers promise mirai. Either function, supplied ‘onFulfilled’ argument promises::() list 2 functions, supplied respectively ‘onFulfilled’ ‘onRejected’ promises::(). Using argument requires promises package. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai Map — mirai_map","text":"‘mirai_map’ (list ‘mirai’ objects).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai Map — mirai_map","text":"Sends application function .f element .x (row .x) computation separate mirai call. simple transparent behaviour designed make full use mirai scheduling minimise overall execution time. Facilitates recovery partial failure returning ‘miraiError’ / ‘errorValue’ case may , thus allowing failures re-run. Note: requires daemons previously set. , one local daemon set function proceeds.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"results","dir":"Reference","previous_headings":"","what":"Results","title":"mirai Map — mirai_map","text":"x[] collects results ‘mirai_map’ x returns list. wait asynchronous operations complete still progress, blocking user-interruptible. x[.flat] collects flattens map results vector, checking type avoid coercion. Note: errors ‘errorValue’ returned results differing type. x[.progress] collects map results whilst showing simple text progress indicator parts completed total. x[.progress_cli] collects map results whilst showing progress bar cli package, available, completion percentage ETA. x[.stop] collects map results applying early stopping, stops first failure cancels remaining operations. Note: operations already progress continue completion, although results collected. options may combined manner : x[.stop, .progress] applies early stopping together progress indicator.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"multiple-map","dir":"Reference","previous_headings":"","what":"Multiple Map","title":"mirai Map — mirai_map","text":"Multiple map performed automatically rows object ‘dim’ attributes matrix dataframe. often desired behaviour cases. map columns instead, first wrap dataframe .list, transpose matrix using t.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai Map — mirai_map","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(4, dispatcher = \"none\") # map with constant args specified via '.args' mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[] # flatmap with function definition passed via '...' mirai_map(1:3, function(x) func(1L, x, x + 1L), func = stats::runif)[.flat] # sum rows of a dataframe (df <- data.frame(a = 1:3, b = c(4, 3, 2))) mirai_map(df, sum)[.flat] # sum rows of a matrix (mat <- matrix(1:4, nrow = 2L)) mirai_map(mat, sum)[.flat] # map over rows of a dataframe df <- data.frame(a = c(\"Aa\", \"Bb\"), b = c(1L, 4L)) mirai_map(df, function(...) sprintf(\"%s: %d\", ...))[.flat] # indexed map over a vector v <- c(\"egg\", \"got\", \"ten\", \"nap\", \"pie\") mirai_map( data.frame(1:length(v), v), sprintf, .args = list(fmt = \"%d_%s\") )[.flat] # return a 'mirai_map' object, check for resolution, collect later mp <- mirai_map( c(a = 2, b = 3, c = 4), function(x, y) do(x, as.logical(x %% y)), do = nanonext::random, .args = list(y = 2) ) unresolved(mp) mp mp[] unresolved(mp) # progress indicator counts up from 0 to 4 seconds res <- mirai_map(1:4, Sys.sleep)[.progress] daemons(0) # generates warning as daemons not set # stops early when second element returns an error tryCatch( mirai_map(list(1, \"a\", 3), sum)[.stop], error = identity ) # promises example that outputs the results, including errors, to the console if (requireNamespace(\"promises\", quietly = TRUE)) { daemons(1, dispatcher = \"none\") ml <- mirai_map( 1:30, function(x) {Sys.sleep(0.1); if (x == 30) stop(x) else x}, .promise = list( function(x) cat(paste(x, \"\")), function(x) { cat(conditionMessage(x), \"\\n\"); daemons(0) } ) ) } }"},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":null,"dir":"Reference","previous_headings":"","what":"Next >> Developer Interface — nextstream","title":"Next >> Developer Interface — nextstream","text":"nextstream retrieves currently stored L'Ecuyer-CMRG RNG stream specified compute profile advances next stream. nextget retrieves specified item specified compute profile.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Next >> Developer Interface — nextstream","text":"","code":"nextstream(.compute = \"default\") nextget(x, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Next >> Developer Interface — nextstream","text":".compute [default 'default'] character value compute profile use (compute profile independent set daemons). x character value item retrieve. One ‘n’ (number dispatcher daemons), ‘pid’ (dispatcher process ID), ‘urls’ (URLs dispatcher listening ) ‘tls’ (stored client TLS configuration use daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Next >> Developer Interface — nextstream","text":"nextstream: length 7 integer vector, given .Random.seed L'Ecuyer-CMRG RNG use (may passed directly ‘rs’ argument daemon), else NULL stream yet created. nextget: requested item, else NULL present.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Next >> Developer Interface — nextstream","text":"functions exported use packages extending mirai alternative launchers daemon processes. nextstream: function called return value required. function also side effect automatically advancing stream stored within compute profile. ensures next recursive stream returned function called .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Next >> Developer Interface — nextstream","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1L) nextstream() nextstream() nextget(\"pid\") nextget(\"urls\") daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":null,"dir":"Reference","previous_headings":"","what":"Register Mirai Cluster — register_cluster","title":"Register Mirai Cluster — register_cluster","text":"Registers 'miraiCluster' parallel package cluster type 'MIRAI' (optionally) makes default cluster type.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Register Mirai Cluster — register_cluster","text":"","code":"register_cluster(default = TRUE)"},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Register Mirai Cluster — register_cluster","text":"default [default TRUE] logical value whether also register 'miraiCluster' default cluster type.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Register Mirai Cluster — register_cluster","text":"Invisible NULL","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"note","dir":"Reference","previous_headings":"","what":"Note","title":"Register Mirai Cluster — register_cluster","text":"underlying implementation parallel package function relies iis currently available R-devel (4.5).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Register Mirai Cluster — register_cluster","text":"","code":"tryCatch(mirai::register_cluster(), error = identity) #> "},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":null,"dir":"Reference","previous_headings":"","what":"Generic and SSH Remote Launch Configuration — remote_config","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"remote_config provides flexible generic framework generating shell commands deploy daemons remotely. ssh_config generates remote configuration launching daemons SSH, option SSH tunnelling.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"","code":"remote_config( command = NULL, args = c(\"\", \".\"), rscript = \"Rscript\", quote = FALSE ) ssh_config( remotes, tunnel = FALSE, timeout = 10, command = \"ssh\", rscript = \"Rscript\", host )"},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"command command used effect daemon launch remote machine character string (e.g. 'ssh'). Defaults ‘ssh’ ssh_config, although may substituted full path specific SSH application. default NULL remote_config effect launches, causes launch_remote return shell commands manual deployment remote machines. args (optional) arguments passed ‘command’, character vector must include \".\" element, substituted daemon launch command. Alternatively, list character vectors effect multiple launches (one list element). rscript (optional) name / path Rscript executable remote machine. default assumes ‘Rscript’ executable search path. Prepend full path necessary. launching Windows, ‘Rscript’ replaced ‘Rscript.exe’. quote [default FALSE] logical value whether quote daemon launch command (required Slurm ‘srun’ example, required ‘ssh’). remotes character URL vector URLs SSH , using 'ssh://' scheme including port open SSH connections (defaults 22 specified), e.g. 'ssh://10.75.32.90:22' 'ssh://nodename'. tunnel [default FALSE] logical value whether use SSH reverse tunnelling. TRUE, tunnel created ports local remote machines. See ‘SSH Tunnelling’ section correctly specify required settings. timeout [default 10] maximum time allowed connection setup seconds. host (optional) applicable reverse tunnelling. specified creating standalone configuration object. calling function directly argument daemons, required can inferred ‘url’ supplied (see ‘SSH Tunnelling’ section ).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"list required format supplied ‘remote’ argument launch_remote, daemons, make_cluster.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ssh-direct-connections","dir":"Reference","previous_headings":"","what":"SSH Direct Connections","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"simplest use SSH execute daemon launch command remote machine, dial back host / dispatcher URL. assumed SSH key-based authentication already place. relevant port host must also open inbound connections remote machine.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ssh-tunnelling","dir":"Reference","previous_headings":"","what":"SSH Tunnelling","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"Use SSH tunnelling provides convenient way launch remote daemons without requiring remote machine able access host. Often firewall configurations security policies may prevent opening port accept outside connections. cases SSH tunnelling offers solution creating tunnel initial SSH connection made. simplicity, SSH tunnelling implementation uses port side host daemon. SSH key-based authentication must also already place. Tunnelling requires hostname ‘host’ argument (‘url’ argument daemons called directly context) either ‘127.0.0.1’ ‘localhost’. tunnel created 127.0.0.1:port equivalently localhost:port machine. host listens port machine remotes dial port respective machines.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"","code":"# Slurm srun example remote_config( command = \"srun\", args = c(\"--mem 512\", \"-n 1\", \".\"), rscript = file.path(R.home(\"bin\"), \"Rscript\") ) #> $command #> [1] \"srun\" #> #> $args #> [1] \"--mem 512\" \"-n 1\" \".\" #> #> $rscript #> [1] \"/opt/R/4.4.1/lib/R/bin/Rscript\" #> #> $quote #> [1] FALSE #> # Slurm sbatch requires 'quote = TRUE' remote_config( command = \"sbatch\", args = c(\"--mem 512\", \"-n 1\", \"--wrap\", \".\"), rscript = file.path(R.home(\"bin\"), \"Rscript\"), quote = TRUE ) #> $command #> [1] \"sbatch\" #> #> $args #> [1] \"--mem 512\" \"-n 1\" \"--wrap\" \".\" #> #> $rscript #> [1] \"/opt/R/4.4.1/lib/R/bin/Rscript\" #> #> $quote #> [1] TRUE #> # SSH also requires 'quote = TRUE' remote_config( command = \"/usr/bin/ssh\", args = c(\"-fTp 22 10.75.32.90\", \".\"), quote = TRUE ) #> $command #> [1] \"/usr/bin/ssh\" #> #> $args #> [1] \"-fTp 22 10.75.32.90\" \".\" #> #> $rscript #> [1] \"Rscript\" #> #> $quote #> [1] TRUE #> # can be used to start local dameons with special configurations remote_config( command = \"Rscript\", rscript = \"--default-packages=NULL --vanilla\" ) #> $command #> [1] \"Rscript\" #> #> $args #> [1] \"\" \".\" #> #> $rscript #> [1] \"--default-packages=NULL --vanilla\" #> #> $quote #> [1] FALSE #> # simple SSH example ssh_config( remotes = c(\"ssh://10.75.32.90:222\", \"ssh://nodename\"), timeout = 5 ) #> $command #> [1] \"ssh\" #> #> $args #> $args[[1]] #> [1] \"-o ConnectTimeout=5 -fTp 222\" \"10.75.32.90\" #> [3] \".\" #> #> $args[[2]] #> [1] \"-o ConnectTimeout=5 -fTp 22\" \"nodename\" #> [3] \".\" #> #> #> $rscript #> [1] \"Rscript\" #> #> $quote #> [1] TRUE #> # SSH tunnelling example ssh_config( remotes = c(\"ssh://10.75.32.90:222\", \"ssh://nodename\"), tunnel = TRUE, host = \"tls+tcp://127.0.0.1:5555\" ) #> $command #> [1] \"ssh\" #> #> $args #> $args[[1]] #> [1] \"-R 5555:127.0.0.1:5555\" \"-o ConnectTimeout=10 -fTp 222\" #> [3] \"10.75.32.90\" \".\" #> #> $args[[2]] #> [1] \"-R 5555:127.0.0.1:5555\" \"-o ConnectTimeout=10 -fTp 22\" #> [3] \"nodename\" \".\" #> #> #> $rscript #> [1] \"Rscript\" #> #> $quote #> [1] TRUE #> if (FALSE) { # \\dontrun{ # launch 2 daemons on the remote machines 10.75.32.90 and 10.75.32.91 using # SSH, connecting back directly to the host URL over a TLS connection: daemons( url = host_url(tls = TRUE), remote = ssh_config( remotes = c(\"ssh://10.75.32.90:222\", \"ssh://10.75.32.91:222\"), timeout = 1 ) ) # launch 2 nodes on the remote machine 10.75.32.90 using SSH tunnelling over # port 5555 ('url' hostname must be 'localhost' or '127.0.0.1'): cl <- make_cluster( url = \"tcp://localhost:5555\", remote = ssh_config( remotes = c(\"ssh://10.75.32.90\", \"ssh://10.75.32.90\"), tunnel = TRUE, timeout = 1 ) ) } # }"},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":null,"dir":"Reference","previous_headings":"","what":"Saisei (Regenerate Token) — saisei","title":"Saisei (Regenerate Token) — saisei","text":"using daemons dispatcher, regenerates token URL dispatcher socket listens .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Saisei (Regenerate Token) — saisei","text":"","code":"saisei(i, force = FALSE, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Saisei (Regenerate Token) — saisei","text":"integer index number URL regenerate dispatcher. force [default FALSE] logical value whether regenerate URL even existing active connection. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Saisei (Regenerate Token) — saisei","text":"regenerated character URL upon success, else NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Saisei (Regenerate Token) — saisei","text":"URL regenerated, listener specified socket closed replaced immediately, hence function successful existing connections socket (.e. ‘online’ status shows 0), unless argument ‘force’ specified TRUE. ‘force’ specified TRUE, socket immediately closed regenerated. happens mirai task still ongoing, returned ‘errorValue’ 7 (Object closed). may used cancel task consistently hangs crashes prevent failing repeatedly new daemons connect.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"timeouts","dir":"Reference","previous_headings":"","what":"Timeouts","title":"Saisei (Regenerate Token) — saisei","text":"Specifying ‘.timeout’ argument mirai ensures mirai always resolves. However, task may completed still ongoing daemon process. situations, dispatcher ensures queued tasks assigned busy process, however overall performance may still degraded remain use. process hangs restarted otherwise, saisei specifying force = TRUE may used cancel task regenerate particular URL new daemon connect .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Saisei (Regenerate Token) — saisei","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1L) Sys.sleep(1L) status() saisei(i = 1L, force = TRUE) status() daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":null,"dir":"Reference","previous_headings":"","what":"Create Serialization Configuration — serial_config","title":"Create Serialization Configuration — serial_config","text":"Returns serialization configuration, may set perform custom serialization unserialization normally non-exportable reference objects, allowing used seamlessly different R sessions. feature utilises 'refhook' system R native serialization. set, functions apply mirai requests specific compute profile.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create Serialization Configuration — serial_config","text":"","code":"serial_config(class, sfunc, ufunc, vec = FALSE)"},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create Serialization Configuration — serial_config","text":"class character string class object custom serialization functions applied , e.g. ‘ArrowTabular’ ‘torch_tensor’. sfunc function accepts reference object inheriting ‘class’ (list objects) returns raw vector. ufunc function accepts raw vector returns reference object (list objects). vec [default FALSE] whether serialization functions vectorized. FALSE, accept return reference objects individually e.g. arrow::write_to_raw arrow::read_ipc_stream. TRUE, accept return list reference objects, e.g. torch::torch_serialize torch::torch_load.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create Serialization Configuration — serial_config","text":"list comprising configuration. passed ‘.serial’ argument everywhere.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Create Serialization Configuration — serial_config","text":"","code":"cfg <- serial_config(\"test_cls\", function(x) serialize(x, NULL), unserialize) cfg #> [[1]] #> [1] \"test_cls\" #> #> [[2]] #> function (x) #> serialize(x, NULL) #> #> #> [[3]] #> function (connection, refhook = NULL) #> { #> if (typeof(connection) != \"raw\" && !is.character(connection) && #> !inherits(connection, \"connection\")) #> stop(\"'connection' must be a connection\") #> .Internal(unserialize(connection, refhook)) #> } #> #> #> #> [[4]] #> [1] FALSE #>"},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":null,"dir":"Reference","previous_headings":"","what":"Status Information — status","title":"Status Information — status","text":"Retrieve status information specified compute profile, comprising current connections daemons status.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Status Information — status","text":"","code":"status(.compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Status Information — status","text":".compute [default 'default'] character compute profile (compute profile set daemons connecting different resources). ‘miraiCluster’ obtain status.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Status Information — status","text":"named list comprising: connections - integer number active connections. Using dispatcher: Always 1L single connection dispatcher, connects daemons turn. daemons - variable type. Using dispatcher: status matrix (see Status Matrix section ), else integer ‘errorValue’ communication dispatcher failed. using dispatcher: character host URL. set: 0L.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"status-matrix","dir":"Reference","previous_headings":"","what":"Status Matrix","title":"Status Information — status","text":"using dispatcher, $daemons comprises integer matrix following columns: - integer index number. online - shows 1 active connection, else 0 daemon yet connect disconnected. instance - increments 1 every time new connection URL. counter designed track new daemon instances connecting previous ones ended (due time-outs etc.). count becomes negative immediately URL regenerated saisei, increments new daemon connects. assigned - shows cumulative number tasks assigned daemon. complete - shows cumulative number tasks completed daemon. dispatcher URLs stored row names matrix.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Status Information — status","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions status() daemons(n = 2L, url = \"wss://[::1]:0\") status() daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Stop) — stop_mirai","title":"mirai (Stop) — stop_mirai","text":"Stops ‘mirai’ still progress, causing resolve immediately ‘errorValue’ 20 (Operation canceled).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Stop) — stop_mirai","text":"","code":"stop_mirai(x)"},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Stop) — stop_mirai","text":"x ‘mirai’ object, list ‘mirai’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Stop) — stop_mirai","text":"Invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Stop) — stop_mirai","text":"Forces ‘mirai’ resolve immediately. effect ‘mirai’ already resolved. cancellation successful, value $data ‘errorValue’ 20 (Operation canceled). Note case, ‘mirai’ aborted value retrieved - ongoing evaluation daemon process continue completion interrupted.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Stop) — stop_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions m <- mirai(Sys.sleep(n), n = 5) stop_mirai(m) m$data }"},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":null,"dir":"Reference","previous_headings":"","what":"Query if a mirai is Unresolved — unresolved","title":"Query if a mirai is Unresolved — unresolved","text":"Query whether ‘mirai’, ‘mirai’ value list ‘mirai’ remains unresolved. Unlike call_mirai, function wait completion.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Query if a mirai is Unresolved — unresolved","text":"","code":"unresolved(x)"},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Query if a mirai is Unresolved — unresolved","text":"x ‘mirai’ object list ‘mirai’ objects, ‘mirai’ value stored $data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Query if a mirai is Unresolved — unresolved","text":"Logical TRUE ‘aio’ unresolved ‘mirai’ ‘mirai’ value list contains least one unresolved ‘mirai’, FALSE otherwise.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Query if a mirai is Unresolved — unresolved","text":"Suitable use control flow statements . Note: querying resolution may cause previously unresolved ‘mirai’ resolve.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Query if a mirai is Unresolved — unresolved","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions m <- mirai(Sys.sleep(0.1)) unresolved(m) Sys.sleep(0.3) unresolved(m) }"},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":null,"dir":"Reference","previous_headings":"","what":"With Mirai Daemons — with.miraiDaemons","title":"With Mirai Daemons — with.miraiDaemons","text":"Evaluate expression daemons last duration expression.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"With Mirai Daemons — with.miraiDaemons","text":"","code":"# S3 method for class 'miraiDaemons' with(data, expr, ...)"},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"With Mirai Daemons — with.miraiDaemons","text":"data call daemons. expr expression evaluate. ... used.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"With Mirai Daemons — with.miraiDaemons","text":"return value ‘expr’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"With Mirai Daemons — with.miraiDaemons","text":"function S3 method generic class 'miraiDaemons'.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"With Mirai Daemons — with.miraiDaemons","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions with( daemons(2), { m1 <- mirai(Sys.getpid()) m2 <- mirai(Sys.getpid()) cat(call_mirai(m1)$data, call_mirai(m2)$data, \"\\n\") } ) status() }"},{"path":[]},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"updates-1-3-0-9000","dir":"Changelog","previous_headings":"","what":"Updates","title":"mirai 1.3.0.9000 (development)","text":"serialization() deprecated mirai 1.2.0 now removed.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-130","dir":"Changelog","previous_headings":"","what":"mirai 1.3.0","title":"mirai 1.3.0","text":"CRAN release: 2024-10-09","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"new-features-1-3-0","dir":"Changelog","previous_headings":"","what":"New Features","title":"mirai 1.3.0","text":"daemons(dispatcher = \"thread\") implements threaded dispatcher (experimental), faster efficient alternative running dispatcher separate process. mirai_map() adds [.progress_cli] alternative progress indicator, using cli package show % complete ETA. daemons() gains argument ‘force’ control whether calls reset previous settings compute profile. daemon() gains argument ‘asyncdial’ allow control connection behaviour independently happens daemon exits.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"behavioural-changes-1-3-0","dir":"Changelog","previous_headings":"","what":"Behavioural Changes","title":"mirai 1.3.0","text":"Argument ‘dispatcher’ now takes character options ‘process’, ‘thread’ ‘none’. Previous values TRUE/FALSE continue accepted (thanks @hadley #157). Return value now always integer - either number daemons set using dispatcher, number daemons launched locally (zero using remote launcher). Invalid type ... arguments now dropped instead throwing error. allows ... containing unused arguments easily passed functions. Now performs multiple map rows matrices dataframes (thanks @andrewGhazi, #147). Combining collection options now easier, fashion : x[.stop, .progress]. Collection options now work even mirai search path e.g. mirai::mirai_map(1:4, Sys.sleep)[.progress]. dispatcher() drops argument ‘asyncdial’ rarely useful set . everywhere() now errors specified compute profile yet set , rather fail silently. launch_local() launch_remote() now strictly require daemons set, error otherwise. serial_config() now validates arguments provided returns list. means saved configurations previous package versions must re-generated.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"updates-1-3-0","dir":"Changelog","previous_headings":"","what":"Updates","title":"mirai 1.3.0","text":"Fixes daemons() correctly handle vector URLs passed ‘url’ . Fixes flatmap mirai_map()[.flat] assigning variable ‘typ’ calling environment. Performance enhancements mirai(), mirai_map() promises method. Requires nanonext >= 1.3.0. package shiny new hex logo.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-120","dir":"Changelog","previous_headings":"","what":"mirai 1.2.0","title":"mirai 1.2.0","text":"CRAN release: 2024-08-18 everywhere() adds argument ‘.serial’ accept serialization configurations created serial_config(). allow normally non-exportable reference objects Arrow Tables torch tensors used seamlessly across parallel processes without additional marshalling steps. Configurations apply per compute profile basis. serialization() now deprecated favour usage everywhere(), removed future version. mirai_map() enhanced perform multiple map 2D lists/vectors, allowing advanced patterns mapping rows dataframe matrix. ‘mirai_map’ [] method gains option [.flat] collect flatten results, avoiding coercion. Collecting ‘mirai_map’ longer spuriously introduces empty names none present originally. Faster local daemons(dispatcher = FALSE) make_cluster() using asynchronous launches (thanks @mtmorgan #123). Local dispatcher daemons now synchronize host, non-dispatcher daemons (prevents use connected). Fixes rare cases everywhere() reaching daemons using dispatcher. efficient dispatcher startup loading base package, addition reading startup configurations (thanks @krlmlr). Removes hard dependency stats utils base packages. Requires nanonext >= 1.2.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-111","dir":"Changelog","previous_headings":"","what":"mirai 1.1.1","title":"mirai 1.1.1","text":"CRAN release: 2024-07-01 serialization() function signature return value slightly modified clarity. Successful registration / cancellation messages longer printed console. dispatcher() argument ‘retry’ now defaults FALSE consistency non-dispatcher behaviour. remote_config() gains argument ‘quote’ control whether quote daemon launch command, now works Slurm (thanks @michaelmayer2 #119). Ephemeral daemons now exit soon permissible, eliminating 2s linger period. Requires nanonext >= 1.1.1.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-110","dir":"Changelog","previous_headings":"","what":"mirai 1.1.0","title":"mirai 1.1.0","text":"CRAN release: 2024-06-06 x[] collects results mirai_map x, waiting asynchronous operations complete. x[.progress] collects results whilst showing text progress bar. x[.stop] collects results applying early-stopping, stops first error, aborts remaining -progress operations. Adds ‘mirai’ method x[] efficient equivalent interruptible call_mirai_(x)$data. Adds collect_mirai() efficient equivalent non-interruptible call_mirai(x)$data. unresolved(), call_mirai(), collect_mirai() stop_mirai() now accept list ‘mirai’ returned mirai_map(). Improved mirai print method indicates whether mirai resolved. Calling daemons() new settings compute profile already set now implicitly resets daemons applying new settings instead silently nothing. Argument ‘resilience’ retired daemons() automatic re-tries longer performed non-dispatcher daemons. New argument ‘retry’ dispatcher() governs whether auto-retry dispatcher case. Fixes promises method potential crashes launching improbably short-lived mirai. Fixes bug cause hang crash launching additional non-dispatcher daemons. Requires nanonext >= 1.1.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-100","dir":"Changelog","previous_headings":"","what":"mirai 1.0.0","title":"mirai 1.0.0","text":"CRAN release: 2024-05-03 innovation allows higher responsiveness massive scalability ‘mirai’ promises. (breaking change) longer permits unnamed list supplied ‘.args’. allows environment e.g. environment() supplied ‘.args’ element ‘…’. allows evaluation symbol ‘mirai’ environment, e.g. mirai(x, x = 1). new argument ‘host’ allows specifying localhost URL port create standalone configuration object. order arguments ‘tunnel’ ‘timeout’ reversed. stop_mirai() now resolves ‘errorValue’ 20 (operation canceled) case asynchronous task still ongoing (thanks @jcheng5 #110). Rejected promises now show complete error code message case ‘errorValue’. ‘miraiError’ reverts including trailing line break (prior mirai 0.13.2). Non-dispatcher local daemons now synchronize host cases (prevents use connected). [ method ‘miraiCluster’ longer produces ‘miraiCluster’ object (thanks @HenrikBengtsson #83). Faster startup time parallel package now loaded first used. Requires nanonext >= 1.0.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0132","dir":"Changelog","previous_headings":"","what":"mirai 0.13.2","title":"mirai 0.13.2","text":"CRAN release: 2024-04-11 mirai() everywhere() behaviour changed ‘…’ args now assigned global environment daemon process. Adds () method mirai daemons, allowing example: (daemons(4), {expr}), daemons last duration ‘expr’. Adds register_cluster() registering ‘miraiCluster’ parallel Cluster type (requires R >= 4.4). Adds .promising() method ‘mirai’ promises package. ‘miraiError’ now includes full call stack, may accessed $stack.trace, includes trailing line break consistency ‘.character.error()’. mirai promises now preserve deep stacks ‘miraiError’ occurs within Shiny app (thanks @jcheng5 #104). Simplified registration ‘parallel’ ‘promises’ methods (thanks @jcheng5 #103). Fixes promises error handling Shiny vignette (thanks @jcheng5 #98 #99). Requires R >= 3.6.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0131","dir":"Changelog","previous_headings":"","what":"mirai 0.13.1","title":"mirai 0.13.1","text":"Fixes regression mirai 0.12.1, introduced potential unintentional low level errors emerge querying dispatcher (thanks @dsweber2 reporting downstream {targets}).","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0130","dir":"Changelog","previous_headings":"","what":"mirai 0.13.0","title":"mirai 0.13.0","text":"serialization adds arguments ‘class’ ‘vec’ custom serialisation reference object types. Requires nanonext >= 0.13.3.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0121","dir":"Changelog","previous_headings":"","what":"mirai 0.12.1","title":"mirai 0.12.1","text":"CRAN release: 2024-02-02 Dispatcher initial sync timeout widened 10s allow launching large numbers daemons. Default ssh_config() argument ‘timeout’ widened 10 (seconds). Fixes daemons() specifying ‘output = FALSE’ registering TRUE instead. Fixes use everywhere() specifying ‘.args’ unnamed list ‘.expr’ language object. Ensures compatibility nanonext >= 0.13.0. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0120","dir":"Changelog","previous_headings":"","what":"mirai 0.12.0","title":"mirai 0.12.0","text":"CRAN release: 2024-01-12 minimal print methods ‘mirai’ ‘miraiCluster’. Adds local_url() helper construct random inter-process communications URL local daemons (thanks @noamross #90). daemon() argument ‘autoexit’ now accepts signal value tools::SIGINT order raise upon exit. daemon() now records state initial global environment objects (e.g. created .Rprofile) cleanup purposes (thanks @noamross #91). Slightly optimal .promise() method ‘mirai’. Eliminates potential memory leaks along certain error paths. Requires nanonext >= 0.12.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0113","dir":"Changelog","previous_headings":"","what":"mirai 0.11.3","title":"mirai 0.11.3","text":"CRAN release: 2023-12-07 Implements serialization() registering custom serialization unserialization functions using daemons. Introduces call_mirai_(), user-interruptible version call_mirai() suitable interactive use. ‘.args’ now coerce list object list supplied, rather error. ‘.signal’ argument removed - now ‘mirai’ signal daemons set . everywhere() now returns invisible NULL case specified compute profile set , rather error. mirai() specifying timeout daemons() set - timeout begins immediately rather ephemeral daemon connected - please factor small amount time daemon launch. make_cluster() now prints daemon launch commands ‘url’ specified without ‘remote’ whether interactive. Cluster node failures load balanced operations now rely ‘parallel’ mechanism error longer fail early automatically stop cluster. Fixes regression since 0.11.0 prevented dispatcher exiting timely manner tasks backlogged (thanks @wlandau #86). Improved memory efficiency stability dispatcher. longer loads ‘promises’ package already loaded (makes ‘mirai’ method available via hook function). Requires nanonext >= 0.11.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0112","dir":"Changelog","previous_headings":"","what":"mirai 0.11.2","title":"mirai 0.11.2","text":"CRAN release: 2023-11-15 make_cluster() specifying ‘url’ now succeeds implied ‘n’ one. Fixes mirai() specifying language object name ‘.expr’ R versions 4.0 earlier. Fixes regression 0.11.1 prevented correct random seed set using dispatcher. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0111","dir":"Changelog","previous_headings":"","what":"mirai 0.11.1","title":"mirai 0.11.1","text":"CRAN release: 2023-11-04 Adds ‘mirai’ method ‘.promise()’ {promises} package (available). functionality merged package {mirai.promises}, allowing use promise pipe %...>% ‘mirai’. Parallel clusters (alternative communications backend R) now work existing R versions, longer requiring R >= 4.4. everywhere() evaluates expression ‘everywhere’ connected daemons compute profile. Resulting changes global environment, loaded pacakges options persisted regardless ‘cleanup’ setting (request @krlmlr #80). host_url() implemented helper function automatically construct host URL using computer’s hostname. daemon() adds argument ‘autoexit’, replaces ‘asyncdial’, govern persistence settings daemon. daemon can now survive host session re-connect another one (request @krlmlr #81). daemons(NULL) implemented variant daemons(0) also sends exit signals connected persistent daemons. dispatcher() argument ‘lock’ removed now applied cases prevent one daemon dialling dispatcher URL one time. daemon() argument ‘cleanup’ simplified logical argument, granular control offered existing integer bitmask (thanks @krlmlr #79). Daemons connecting TLS now perform synchronous dials default (documented). Fixes supplying ssh_config() specifying tunnelling ‘remote’ argument daemons(). Fixes print method subset ‘miraiCluster’ (thanks @HenrikBengtsson #83). Removes deprecated deferred evaluation pipe %>>%. Requires nanonext >= 0.10.4.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0110","dir":"Changelog","previous_headings":"","what":"mirai 0.11.0","title":"mirai 0.11.0","text":"CRAN release: 2023-10-06 Fulfils request R Core R Project Sprint 2023, requires R >= 4.4 (currently R-devel). make_cluster() creates ‘miraiCluster’, compatible existing functions taking ‘cluster’ object, example ‘parallel’ ‘doParallel’ / ‘foreach’ packages. status() can now take ‘miraiCluster’ argument query connection status. Simplified interface single ‘remote’ argument taking remote configuration launch daemons. Returned shell commands now custom print method means may directly copy/pasted remote machine. Can now take ‘miraiCluster’ ‘miraiNode’ return shell commands deployment remote nodes. Adds argument ‘remote’ launching remote daemons directly without recourse separate call launch_remote(). Adds argument ‘resilience’ control behaviour, using dispatcher, whether retry failed tasks daemons. remote_config() added generate configurations directly launching remote daemons, can supplied directly ‘remote’ argument. ssh_config() added convenience method generate launch configurations using SSH, including SSH tunnelling. mirai() adds logical argument ‘.signal’ whether signal condition variable within compute profile upon resolution ‘mirai’. daemon() argument ‘exitlinger’ retired daemons now synchronise host/dispatcher exit soon possible (although default ‘exitlinger’ period still applies ephemeral daemons). Optimises scheduling dispatcher: tasks longer assigned daemon exiting due specified time/task-outs. ‘errorValue’ 19 ‘Connection reset’ now returned ‘mirai’ connection either dispatcher ephemeral daemon drops, example crashed, rather remaining unresolved. Invalid type ‘…’ arguments specified daemons() dispatcher() now raise error early rather attempting launch daemons fail. Eliminates potential crash host process querying status() longer connection dispatcher. Reverts trailing line break added end ‘miraiError’ character string. Moves ‘…’ argument daemons(), dispatcher() daemon() clearly delineate core vs peripheral options. Deprecates Deferred Evaluation Pipe %>>% favour recommendation use package mirai.promises performing side effects upon ‘mirai’ resolution. Deprecated use alias server() daemon() retired. Adds ‘reference’ vignette, incorporating information readme. Requires nanonext >= 0.10.2.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0100","dir":"Changelog","previous_headings":"","what":"mirai 0.10.0","title":"mirai 0.10.0","text":"CRAN release: 2023-09-16 daemons() gains new argument ‘seed’ set random seed generating streams. daemon() dispatcher() gain argument ‘rs’ takes L’Ecuyer-CMRG random seed. New developer functions nextstream() nextget(), opening interfaces packages extend mirai. Runs R session --vanilla flags efficiency, avoiding lengthy startup configurations (thanks @alexpiper). Straight pass without serialization/unserialization allows higher performance lower memory utilisation. Fixes edge cases status() occasionally failing communicate dispatcher. Fixes edge cases ending session unresolved mirai resulting crash rather clean exit. specifying argument ‘force’ TRUE now immediately regenerates socket returns ongoing mirai ‘errorValue’. allows tasks consistently hang crash cancelled rather repeated new daemon connects. argument ‘’ now required longer defaults 1L. daemons status matrix adds column ‘’ ease use functions saisei() launch_local(). ‘instance’ column now always cumulative - regenerating URL saisei() longer resets counter instead turns negative new daemon connects. Improves shell quoting daemon launch commands, making easier deploy manually via launch_remote(). daemons() dispatcher() gain argument ‘pass’ support password-protected private keys supplying TLS credentials (thanks @wlandau #76). Cryptographic errors using dispatcher TLS now reported user (thanks @wlandau #76). Passing filename ‘tls’ argument daemons(), launch_local() launch_remote() now works correctly documented. Extends clarifies documentation surrounding use certificate authority signed TLS certificates. Certain error messages accurate informative. Increases performance lower resource utilisation due updates nanonext 0.10.0. Requires nanonext >= 0.10.0 R >= 3.5.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-091","dir":"Changelog","previous_headings":"","what":"mirai 0.9.1","title":"mirai 0.9.1","text":"CRAN release: 2023-07-19 Zero-configuration experience - simply specify tls+tcp:// wss:// URL daemons(). Single-use keys certificates automatically generated. Alternatively, custom certificates may passed ‘tls’ argument daemons() daemon(), generated via Ceritficate Signing Request (CSR) Certificate Authority (CA). Example using SSH: launch_remote(\"ws://192.168.0.1:5555\", command = \"ssh\", args = c(\"-p 22 192.168.0.2\", .). Documentation updated refer consistently host daemons (rather client server) clarity. daemon() replaces server(), deprecated (although currently retained alias). launch_local() replaces launch_server() now accepts vector argument ‘url’ well numeric values select relevant dispatcher host URL, returning invisible NULL instead integer value. status() now retrieves connections daemons status, replacing call daemons() arguments (deprecated). return value $daemons now always host URL using dispatcher. Redirection stdout stderr local daemons host process now possible (running without dispatcher) specifying output=TRUE daemons() launch_local(). daemon() accepts new ‘output’ argument. saisei() argument validation now happens prior sending request dispatcher rather dispatcher. ‘miraiError’ now includes trailing line break end character vector. Requires nanonext >= 0.9.1, R requirement relaxed back >= 2.12.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-090","dir":"Changelog","previous_headings":"","what":"mirai 0.9.0","title":"mirai 0.9.0","text":"CRAN release: 2023-06-24 mirai 0.9.0 major release focusing stability improvements. Ensures first URL retains format saisei(= 1L) called. Optimal scheduling tasks submitted prior servers coming online. Fixes rare occasions dispatcher running single server instance get stuck task. daemons() status requests rendered robust. Ensures saisei() always returns NULL ‘tcp://’ URLs used support tokens. Daemons status matrix ‘assigned’ ‘complete’ now cumulative statistics, reset upon new instances. Requires nanonext >= 0.9.0 R >= 3.5.0. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-087","dir":"Changelog","previous_headings":"","what":"mirai 0.8.7","title":"mirai 0.8.7","text":"CRAN release: 2023-05-11 server() dispatcher() argument ‘asyncdial’ now FALSE default, causing functions exit connection immediately available. means distributed computing purposes, daemons() called server() launched remote resources, else server(asyncdial = TRUE) allows servers wait connection. launch_server() now parses passed URL correctness attempting launch server, producing error valid.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-084","dir":"Changelog","previous_headings":"","what":"mirai 0.8.4","title":"mirai 0.8.4","text":"CRAN release: 2023-05-09 .() implemented wrap piped expression, ensuring return either ‘unresolvedExpr’ ‘resolvedExpr’. expressions may tested using unresolved() way ‘mirai’. allows general use contexts, including within functions. Improved error messages top level evaluation errors ‘mirai’. Requires nanonext >= 0.8.3. Internal stability performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-083","dir":"Changelog","previous_headings":"","what":"mirai 0.8.3","title":"mirai 0.8.3","text":"CRAN release: 2023-04-17 accepts language expression object passed ‘.expr’ evaluation. accepts list ‘name = value’ pairs passed ‘.args’ well existing ‘…’. objects specified via ‘…’ now take precedence ‘.args’ named object appears. token appending unique token URL dispatcher listens . lock locking sockets prevent one server connecting unique URL. saisei() implemented regenerate token used given dispatcher socket. launch_server() replaces launch() launching local instances, simpler interface directly mapping server(). Automatically-launched local daemons revised use unique tokens URLs. Daemons status matrix headers updated ‘online’, ‘instance’, ‘assigned’, ‘complete’. Fixes potential issue attempting use mirai() timeouts connection server. Requires nanonext >= 0.8.2. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-082","dir":"Changelog","previous_headings":"","what":"mirai 0.8.2","title":"mirai 0.8.2","text":"CRAN release: 2023-04-03 Arguments ‘pollfreqh’ ‘pollfreql’ removed longer applicable. Server dispatcher processes exit automatically connection client dropped. significantly reduces likelihood orphaned processes. launch() exported utility easily re-launching daemons timed , instance. Correct passthrough ... variables daemons() call. Requires nanonext >= 0.8.1. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-081","dir":"Changelog","previous_headings":"","what":"mirai 0.8.1","title":"mirai 0.8.1","text":"CRAN release: 2023-03-17 Fixes issue daemon processes may launch certain setups (affecting binary package builds).","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-080","dir":"Changelog","previous_headings":"","what":"mirai 0.8.0","title":"mirai 0.8.0","text":"CRAN release: 2023-03-15 mirai 0.8.0 major feature release. Special thanks @wlandau suggestions, discussion testing many new capabilities. daemons() can create new profiles connect different resources e.g. servers GPU, accelerators etc. mirai() tasks can sent using specific compute profile. Supplying client URL zero port number :0 automatically assign free ephemeral port, actual port number subsequently reported daemons(). Calling arguments now provides improved view current number connections / daemons (URL, online busy status, tasks assigned completed, instance), replacing previous daemons(\"view\") functionality. dispatcher() implemented new function dispatcher. asyncdial specify server dials client. maxtasks specifying maximum number tasks exiting. idletime specifying idle time, since completion last task exiting. walltime specifying soft walltime exiting. timerstart specifying minimum number task completions starting timers. Invalid URLs provided daemons() server() now error return immediately instead potentially causing hang. eval_mirai() removed alias mirai(). ‘mirai’ processes longer launched Rscript sessions --vanilla argument enable site / user profile environment files read. Requires nanonext >= 0.8.0. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-072","dir":"Changelog","previous_headings":"","what":"mirai 0.7.2","title":"mirai 0.7.2","text":"CRAN release: 2023-01-17 Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-071","dir":"Changelog","previous_headings":"","what":"mirai 0.7.1","title":"mirai 0.7.1","text":"CRAN release: 2022-11-15 Allow user interrupts call_mirai() (regression 0.7.0), now returning ‘miraiInterrupt’. Adds auxiliary function is_mirai_interrupt() test object ‘miraiInterrupt’. Requires nanonext >= 0.7.0: returned ‘errorValues’ e.g. mirai timeouts longer accompanied warnings. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-070","dir":"Changelog","previous_headings":"","what":"mirai 0.7.0","title":"mirai 0.7.0","text":"CRAN release: 2022-10-19 daemons() now takes ‘n’ ‘.url’ arguments. ‘.url’ optional client URL allowing mirai tasks distributed across network. Compatibility existing interface retained. server function server() exported creating daemon / ephemeral processes network resources. Mirai errors formatted better now print stdout rather stderr. Improvements performance stability requiring nanonext >= 0.6.0. Internal enhancements error handling mirai / daemon process.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-060","dir":"Changelog","previous_headings":"","what":"mirai 0.6.0","title":"mirai 0.6.0","text":"CRAN release: 2022-09-16 Notice: older package versions longer supported ‘nanonext’ >= 0.6.0. Please ensure using latest version ‘mirai’ else refrain upgrading ‘nanonext’. Internal enhancements daemons() %>>% deferred evaluation pipe.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-053","dir":"Changelog","previous_headings":"","what":"mirai 0.5.3","title":"mirai 0.5.3","text":"CRAN release: 2022-08-16 mirai() gains ‘.args’ argument passing list objects already calling environment, allowing example mirai(func(x, y, z), .args = list(x, y, z)) rather specify mirai(func(x, y, z), x = x, y = y, z = z). Errors inside mirai now return error message character string class ‘miraiError’ ‘errorValue’, rather just nul byte. Utility function is_mirai_error() used place is_nul_byte(), longer re-exported. is_error_value() can used test errors, including timeouts ‘.timeout’ argument used. re-exports ‘nanonext’ brought -package better documentation.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-052","dir":"Changelog","previous_headings":"","what":"mirai 0.5.2","title":"mirai 0.5.2","text":"CRAN release: 2022-07-21 Internal optimisations requiring nanonext >= 0.5.2.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-050","dir":"Changelog","previous_headings":"","what":"mirai 0.5.0","title":"mirai 0.5.0","text":"CRAN release: 2022-06-21 Implements %>>% deferred evaluation pipe. Adds ‘.timeout’ argument mirai() ensure mirai always resolves even child process crashes etc.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-041","dir":"Changelog","previous_headings":"","what":"mirai 0.4.1","title":"mirai 0.4.1","text":"CRAN release: 2022-04-21 Exits cleanly daemons explicitly zeroed prior ending R session. Fixes possible hang Windows shutting daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-040","dir":"Changelog","previous_headings":"","what":"mirai 0.4.0","title":"mirai 0.4.0","text":"CRAN release: 2022-04-14 Back pure R implementation thanks enhanced internal design nanonext. Adds auxiliary function is_mirai() test object mirai. Versioning system synchronise nanonext e.g. v0.4.x requires nanonext >= 0.4.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-020","dir":"Changelog","previous_headings":"","what":"mirai 0.2.0","title":"mirai 0.2.0","text":"CRAN release: 2022-03-28 value mirai now stored $data optimally align underlying implementation. Package now contains C code (requires compilation), using weak references simpler management resources. Switch abstract sockets Linux.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-011","dir":"Changelog","previous_headings":"","what":"mirai 0.1.1","title":"mirai 0.1.1","text":"CRAN release: 2022-03-15 mirai() added alias eval_mirai(); supports evaluating arbitrary length expressions wrapped {}. mirai now resolves automatically without requiring call_mirai(). Access $value directly ‘unresolved’ logical NA returned async operation yet complete. stop_mirai() added function stop evaluation ongoing async operation. Auxiliary functions is_nul_byte() unresolved() re-exported {nanonext} test evaluation errors resolution ‘mirai’ respectively. New daemons() interface set manage persistent background processes receiving ‘mirai’ requests.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-010","dir":"Changelog","previous_headings":"","what":"mirai 0.1.0","title":"mirai 0.1.0","text":"CRAN release: 2022-02-16 Initial release.","code":""}] +[{"path":[]},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"our-pledge","dir":"","previous_headings":"","what":"Our Pledge","title":"Contributor Covenant Code of Conduct","text":"members, contributors, leaders pledge make participation community harassment-free experience everyone, regardless age, body size, visible invisible disability, ethnicity, sex characteristics, gender identity expression, level experience, education, socio-economic status, nationality, personal appearance, race, religion, sexual identity orientation. pledge act interact ways contribute open, welcoming, diverse, inclusive, healthy community.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"our-standards","dir":"","previous_headings":"","what":"Our Standards","title":"Contributor Covenant Code of Conduct","text":"Examples behavior contributes positive environment community include: Demonstrating empathy kindness toward people respectful differing opinions, viewpoints, experiences Giving gracefully accepting constructive feedback Accepting responsibility apologizing affected mistakes, learning experience Focusing best just us individuals, overall community Examples unacceptable behavior include: use sexualized language imagery, sexual attention advances kind Trolling, insulting derogatory comments, personal political attacks Public private harassment Publishing others’ private information, physical email address, without explicit permission conduct reasonably considered inappropriate professional setting","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"enforcement-responsibilities","dir":"","previous_headings":"","what":"Enforcement Responsibilities","title":"Contributor Covenant Code of Conduct","text":"Community leaders responsible clarifying enforcing standards acceptable behavior take appropriate fair corrective action response behavior deem inappropriate, threatening, offensive, harmful. Community leaders right responsibility remove, edit, reject comments, commits, code, wiki edits, issues, contributions aligned Code Conduct, communicate reasons moderation decisions appropriate.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"scope","dir":"","previous_headings":"","what":"Scope","title":"Contributor Covenant Code of Conduct","text":"Code Conduct applies within community spaces, also applies individual officially representing community public spaces. Examples representing community include using official e-mail address, posting via official social media account, acting appointed representative online offline event.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"enforcement","dir":"","previous_headings":"","what":"Enforcement","title":"Contributor Covenant Code of Conduct","text":"Instances abusive, harassing, otherwise unacceptable behavior may reported community leaders responsible enforcement code@hibiki-ai.com. complaints reviewed investigated promptly fairly. community leaders obligated respect privacy security reporter incident.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"enforcement-guidelines","dir":"","previous_headings":"","what":"Enforcement Guidelines","title":"Contributor Covenant Code of Conduct","text":"Community leaders follow Community Impact Guidelines determining consequences action deem violation Code Conduct:","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_1-correction","dir":"","previous_headings":"Enforcement Guidelines","what":"1. Correction","title":"Contributor Covenant Code of Conduct","text":"Community Impact: Use inappropriate language behavior deemed unprofessional unwelcome community. Consequence: private, written warning community leaders, providing clarity around nature violation explanation behavior inappropriate. public apology may requested.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_2-warning","dir":"","previous_headings":"Enforcement Guidelines","what":"2. Warning","title":"Contributor Covenant Code of Conduct","text":"Community Impact: violation single incident series actions. Consequence: warning consequences continued behavior. interaction people involved, including unsolicited interaction enforcing Code Conduct, specified period time. includes avoiding interactions community spaces well external channels like social media. Violating terms may lead temporary permanent ban.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_3-temporary-ban","dir":"","previous_headings":"Enforcement Guidelines","what":"3. Temporary Ban","title":"Contributor Covenant Code of Conduct","text":"Community Impact: serious violation community standards, including sustained inappropriate behavior. Consequence: temporary ban sort interaction public communication community specified period time. public private interaction people involved, including unsolicited interaction enforcing Code Conduct, allowed period. Violating terms may lead permanent ban.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"id_4-permanent-ban","dir":"","previous_headings":"Enforcement Guidelines","what":"4. Permanent Ban","title":"Contributor Covenant Code of Conduct","text":"Community Impact: Demonstrating pattern violation community standards, including sustained inappropriate behavior, harassment individual, aggression toward disparagement classes individuals. Consequence: permanent ban sort public interaction within community.","code":""},{"path":"https://shikokuchuo.net/mirai/CODE_OF_CONDUCT.html","id":"attribution","dir":"","previous_headings":"","what":"Attribution","title":"Contributor Covenant Code of Conduct","text":"Code Conduct adapted Contributor Covenant, version 2.0, available https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. Community Impact Guidelines inspired Mozilla’s code conduct enforcement ladder. answers common questions code conduct, see FAQ https://www.contributor-covenant.org/faq. Translations available https://www.contributor-covenant.org/translations.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":null,"dir":"","previous_headings":"","what":"GNU General Public License","title":"GNU General Public License","text":"Version 3, 29 June 2007Copyright © 2007 Free Software Foundation, Inc.  Everyone permitted copy distribute verbatim copies license document, changing allowed.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"preamble","dir":"","previous_headings":"","what":"Preamble","title":"GNU General Public License","text":"GNU General Public License free, copyleft license software kinds works. licenses software practical works designed take away freedom share change works. contrast, GNU General Public License intended guarantee freedom share change versions program–make sure remains free software users. , Free Software Foundation, use GNU General Public License software; applies also work released way authors. can apply programs, . speak free software, referring freedom, price. General Public Licenses designed make sure freedom distribute copies free software (charge wish), receive source code can get want , can change software use pieces new free programs, know can things. protect rights, need prevent others denying rights asking surrender rights. Therefore, certain responsibilities distribute copies software, modify : responsibilities respect freedom others. example, distribute copies program, whether gratis fee, must pass recipients freedoms received. must make sure , , receive can get source code. must show terms know rights. Developers use GNU GPL protect rights two steps: (1) assert copyright software, (2) offer License giving legal permission copy, distribute /modify . developers’ authors’ protection, GPL clearly explains warranty free software. users’ authors’ sake, GPL requires modified versions marked changed, problems attributed erroneously authors previous versions. devices designed deny users access install run modified versions software inside , although manufacturer can . fundamentally incompatible aim protecting users’ freedom change software. systematic pattern abuse occurs area products individuals use, precisely unacceptable. Therefore, designed version GPL prohibit practice products. problems arise substantially domains, stand ready extend provision domains future versions GPL, needed protect freedom users. Finally, every program threatened constantly software patents. States allow patents restrict development use software general-purpose computers, , wish avoid special danger patents applied free program make effectively proprietary. prevent , GPL assures patents used render program non-free. precise terms conditions copying, distribution modification follow.","code":""},{"path":[]},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_0-definitions","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"0. Definitions","title":"GNU General Public License","text":"“License” refers version 3 GNU General Public License. “Copyright” also means copyright-like laws apply kinds works, semiconductor masks. “Program” refers copyrightable work licensed License. licensee addressed “”. “Licensees” “recipients” may individuals organizations. “modify” work means copy adapt part work fashion requiring copyright permission, making exact copy. resulting work called “modified version” earlier work work “based ” earlier work. “covered work” means either unmodified Program work based Program. “propagate” work means anything , without permission, make directly secondarily liable infringement applicable copyright law, except executing computer modifying private copy. Propagation includes copying, distribution (without modification), making available public, countries activities well. “convey” work means kind propagation enables parties make receive copies. Mere interaction user computer network, transfer copy, conveying. interactive user interface displays “Appropriate Legal Notices” extent includes convenient prominently visible feature (1) displays appropriate copyright notice, (2) tells user warranty work (except extent warranties provided), licensees may convey work License, view copy License. interface presents list user commands options, menu, prominent item list meets criterion.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_1-source-code","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"1. Source Code","title":"GNU General Public License","text":"“source code” work means preferred form work making modifications . “Object code” means non-source form work. “Standard Interface” means interface either official standard defined recognized standards body, , case interfaces specified particular programming language, one widely used among developers working language. “System Libraries” executable work include anything, work whole, () included normal form packaging Major Component, part Major Component, (b) serves enable use work Major Component, implement Standard Interface implementation available public source code form. “Major Component”, context, means major essential component (kernel, window system, ) specific operating system () executable work runs, compiler used produce work, object code interpreter used run . “Corresponding Source” work object code form means source code needed generate, install, (executable work) run object code modify work, including scripts control activities. However, include work’s System Libraries, general-purpose tools generally available free programs used unmodified performing activities part work. example, Corresponding Source includes interface definition files associated source files work, source code shared libraries dynamically linked subprograms work specifically designed require, intimate data communication control flow subprograms parts work. Corresponding Source need include anything users can regenerate automatically parts Corresponding Source. Corresponding Source work source code form work.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_2-basic-permissions","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"2. Basic Permissions","title":"GNU General Public License","text":"rights granted License granted term copyright Program, irrevocable provided stated conditions met. License explicitly affirms unlimited permission run unmodified Program. output running covered work covered License output, given content, constitutes covered work. License acknowledges rights fair use equivalent, provided copyright law. may make, run propagate covered works convey, without conditions long license otherwise remains force. may convey covered works others sole purpose make modifications exclusively , provide facilities running works, provided comply terms License conveying material control copyright. thus making running covered works must exclusively behalf, direction control, terms prohibit making copies copyrighted material outside relationship . Conveying circumstances permitted solely conditions stated . Sublicensing allowed; section 10 makes unnecessary.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_3-protecting-users-legal-rights-from-anti-circumvention-law","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"3. Protecting Users’ Legal Rights From Anti-Circumvention Law","title":"GNU General Public License","text":"covered work shall deemed part effective technological measure applicable law fulfilling obligations article 11 WIPO copyright treaty adopted 20 December 1996, similar laws prohibiting restricting circumvention measures. convey covered work, waive legal power forbid circumvention technological measures extent circumvention effected exercising rights License respect covered work, disclaim intention limit operation modification work means enforcing, work’s users, third parties’ legal rights forbid circumvention technological measures.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_4-conveying-verbatim-copies","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"4. Conveying Verbatim Copies","title":"GNU General Public License","text":"may convey verbatim copies Program’s source code receive , medium, provided conspicuously appropriately publish copy appropriate copyright notice; keep intact notices stating License non-permissive terms added accord section 7 apply code; keep intact notices absence warranty; give recipients copy License along Program. may charge price price copy convey, may offer support warranty protection fee.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_5-conveying-modified-source-versions","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"5. Conveying Modified Source Versions","title":"GNU General Public License","text":"may convey work based Program, modifications produce Program, form source code terms section 4, provided also meet conditions: ) work must carry prominent notices stating modified , giving relevant date. b) work must carry prominent notices stating released License conditions added section 7. requirement modifies requirement section 4 “keep intact notices”. c) must license entire work, whole, License anyone comes possession copy. License therefore apply, along applicable section 7 additional terms, whole work, parts, regardless packaged. License gives permission license work way, invalidate permission separately received . d) work interactive user interfaces, must display Appropriate Legal Notices; however, Program interactive interfaces display Appropriate Legal Notices, work need make . compilation covered work separate independent works, nature extensions covered work, combined form larger program, volume storage distribution medium, called “aggregate” compilation resulting copyright used limit access legal rights compilation’s users beyond individual works permit. Inclusion covered work aggregate cause License apply parts aggregate.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_6-conveying-non-source-forms","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"6. Conveying Non-Source Forms","title":"GNU General Public License","text":"may convey covered work object code form terms sections 4 5, provided also convey machine-readable Corresponding Source terms License, one ways: ) Convey object code , embodied , physical product (including physical distribution medium), accompanied Corresponding Source fixed durable physical medium customarily used software interchange. b) Convey object code , embodied , physical product (including physical distribution medium), accompanied written offer, valid least three years valid long offer spare parts customer support product model, give anyone possesses object code either (1) copy Corresponding Source software product covered License, durable physical medium customarily used software interchange, price reasonable cost physically performing conveying source, (2) access copy Corresponding Source network server charge. c) Convey individual copies object code copy written offer provide Corresponding Source. alternative allowed occasionally noncommercially, received object code offer, accord subsection 6b. d) Convey object code offering access designated place (gratis charge), offer equivalent access Corresponding Source way place charge. need require recipients copy Corresponding Source along object code. place copy object code network server, Corresponding Source may different server (operated third party) supports equivalent copying facilities, provided maintain clear directions next object code saying find Corresponding Source. Regardless server hosts Corresponding Source, remain obligated ensure available long needed satisfy requirements. e) Convey object code using peer--peer transmission, provided inform peers object code Corresponding Source work offered general public charge subsection 6d. separable portion object code, whose source code excluded Corresponding Source System Library, need included conveying object code work. “User Product” either (1) “consumer product”, means tangible personal property normally used personal, family, household purposes, (2) anything designed sold incorporation dwelling. determining whether product consumer product, doubtful cases shall resolved favor coverage. particular product received particular user, “normally used” refers typical common use class product, regardless status particular user way particular user actually uses, expects expected use, product. product consumer product regardless whether product substantial commercial, industrial non-consumer uses, unless uses represent significant mode use product. “Installation Information” User Product means methods, procedures, authorization keys, information required install execute modified versions covered work User Product modified version Corresponding Source. information must suffice ensure continued functioning modified object code case prevented interfered solely modification made. convey object code work section , , specifically use , User Product, conveying occurs part transaction right possession use User Product transferred recipient perpetuity fixed term (regardless transaction characterized), Corresponding Source conveyed section must accompanied Installation Information. requirement apply neither third party retains ability install modified object code User Product (example, work installed ROM). requirement provide Installation Information include requirement continue provide support service, warranty, updates work modified installed recipient, User Product modified installed. Access network may denied modification materially adversely affects operation network violates rules protocols communication across network. Corresponding Source conveyed, Installation Information provided, accord section must format publicly documented (implementation available public source code form), must require special password key unpacking, reading copying.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_7-additional-terms","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"7. Additional Terms","title":"GNU General Public License","text":"“Additional permissions” terms supplement terms License making exceptions one conditions. Additional permissions applicable entire Program shall treated though included License, extent valid applicable law. additional permissions apply part Program, part may used separately permissions, entire Program remains governed License without regard additional permissions. convey copy covered work, may option remove additional permissions copy, part . (Additional permissions may written require removal certain cases modify work.) may place additional permissions material, added covered work, can give appropriate copyright permission. Notwithstanding provision License, material add covered work, may (authorized copyright holders material) supplement terms License terms: ) Disclaiming warranty limiting liability differently terms sections 15 16 License; b) Requiring preservation specified reasonable legal notices author attributions material Appropriate Legal Notices displayed works containing ; c) Prohibiting misrepresentation origin material, requiring modified versions material marked reasonable ways different original version; d) Limiting use publicity purposes names licensors authors material; e) Declining grant rights trademark law use trade names, trademarks, service marks; f) Requiring indemnification licensors authors material anyone conveys material (modified versions ) contractual assumptions liability recipient, liability contractual assumptions directly impose licensors authors. non-permissive additional terms considered “restrictions” within meaning section 10. Program received , part , contains notice stating governed License along term restriction, may remove term. license document contains restriction permits relicensing conveying License, may add covered work material governed terms license document, provided restriction survive relicensing conveying. add terms covered work accord section, must place, relevant source files, statement additional terms apply files, notice indicating find applicable terms. Additional terms, permissive non-permissive, may stated form separately written license, stated exceptions; requirements apply either way.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_8-termination","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"8. Termination","title":"GNU General Public License","text":"may propagate modify covered work except expressly provided License. attempt otherwise propagate modify void, automatically terminate rights License (including patent licenses granted third paragraph section 11). However, cease violation License, license particular copyright holder reinstated () provisionally, unless copyright holder explicitly finally terminates license, (b) permanently, copyright holder fails notify violation reasonable means prior 60 days cessation. Moreover, license particular copyright holder reinstated permanently copyright holder notifies violation reasonable means, first time received notice violation License (work) copyright holder, cure violation prior 30 days receipt notice. Termination rights section terminate licenses parties received copies rights License. rights terminated permanently reinstated, qualify receive new licenses material section 10.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_9-acceptance-not-required-for-having-copies","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"9. Acceptance Not Required for Having Copies","title":"GNU General Public License","text":"required accept License order receive run copy Program. Ancillary propagation covered work occurring solely consequence using peer--peer transmission receive copy likewise require acceptance. However, nothing License grants permission propagate modify covered work. actions infringe copyright accept License. Therefore, modifying propagating covered work, indicate acceptance License .","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_10-automatic-licensing-of-downstream-recipients","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"10. Automatic Licensing of Downstream Recipients","title":"GNU General Public License","text":"time convey covered work, recipient automatically receives license original licensors, run, modify propagate work, subject License. responsible enforcing compliance third parties License. “entity transaction” transaction transferring control organization, substantially assets one, subdividing organization, merging organizations. propagation covered work results entity transaction, party transaction receives copy work also receives whatever licenses work party’s predecessor interest give previous paragraph, plus right possession Corresponding Source work predecessor interest, predecessor can get reasonable efforts. may impose restrictions exercise rights granted affirmed License. example, may impose license fee, royalty, charge exercise rights granted License, may initiate litigation (including cross-claim counterclaim lawsuit) alleging patent claim infringed making, using, selling, offering sale, importing Program portion .","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_11-patents","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"11. Patents","title":"GNU General Public License","text":"“contributor” copyright holder authorizes use License Program work Program based. work thus licensed called contributor’s “contributor version”. contributor’s “essential patent claims” patent claims owned controlled contributor, whether already acquired hereafter acquired, infringed manner, permitted License, making, using, selling contributor version, include claims infringed consequence modification contributor version. purposes definition, “control” includes right grant patent sublicenses manner consistent requirements License. contributor grants non-exclusive, worldwide, royalty-free patent license contributor’s essential patent claims, make, use, sell, offer sale, import otherwise run, modify propagate contents contributor version. following three paragraphs, “patent license” express agreement commitment, however denominated, enforce patent (express permission practice patent covenant sue patent infringement). “grant” patent license party means make agreement commitment enforce patent party. convey covered work, knowingly relying patent license, Corresponding Source work available anyone copy, free charge terms License, publicly available network server readily accessible means, must either (1) cause Corresponding Source available, (2) arrange deprive benefit patent license particular work, (3) arrange, manner consistent requirements License, extend patent license downstream recipients. “Knowingly relying” means actual knowledge , patent license, conveying covered work country, recipient’s use covered work country, infringe one identifiable patents country reason believe valid. , pursuant connection single transaction arrangement, convey, propagate procuring conveyance , covered work, grant patent license parties receiving covered work authorizing use, propagate, modify convey specific copy covered work, patent license grant automatically extended recipients covered work works based . patent license “discriminatory” include within scope coverage, prohibits exercise , conditioned non-exercise one rights specifically granted License. may convey covered work party arrangement third party business distributing software, make payment third party based extent activity conveying work, third party grants, parties receive covered work , discriminatory patent license () connection copies covered work conveyed (copies made copies), (b) primarily connection specific products compilations contain covered work, unless entered arrangement, patent license granted, prior 28 March 2007. Nothing License shall construed excluding limiting implied license defenses infringement may otherwise available applicable patent law.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_12-no-surrender-of-others-freedom","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"12. No Surrender of Others’ Freedom","title":"GNU General Public License","text":"conditions imposed (whether court order, agreement otherwise) contradict conditions License, excuse conditions License. convey covered work satisfy simultaneously obligations License pertinent obligations, consequence may convey . example, agree terms obligate collect royalty conveying convey Program, way satisfy terms License refrain entirely conveying Program.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_13-use-with-the-gnu-affero-general-public-license","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"13. Use with the GNU Affero General Public License","title":"GNU General Public License","text":"Notwithstanding provision License, permission link combine covered work work licensed version 3 GNU Affero General Public License single combined work, convey resulting work. terms License continue apply part covered work, special requirements GNU Affero General Public License, section 13, concerning interaction network apply combination .","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_14-revised-versions-of-this-license","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"14. Revised Versions of this License","title":"GNU General Public License","text":"Free Software Foundation may publish revised /new versions GNU General Public License time time. new versions similar spirit present version, may differ detail address new problems concerns. version given distinguishing version number. Program specifies certain numbered version GNU General Public License “later version” applies , option following terms conditions either numbered version later version published Free Software Foundation. Program specify version number GNU General Public License, may choose version ever published Free Software Foundation. Program specifies proxy can decide future versions GNU General Public License can used, proxy’s public statement acceptance version permanently authorizes choose version Program. Later license versions may give additional different permissions. However, additional obligations imposed author copyright holder result choosing follow later version.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_15-disclaimer-of-warranty","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"15. Disclaimer of Warranty","title":"GNU General Public License","text":"WARRANTY PROGRAM, EXTENT PERMITTED APPLICABLE LAW. EXCEPT OTHERWISE STATED WRITING COPYRIGHT HOLDERS /PARTIES PROVIDE PROGRAM “” WITHOUT WARRANTY KIND, EITHER EXPRESSED IMPLIED, INCLUDING, LIMITED , IMPLIED WARRANTIES MERCHANTABILITY FITNESS PARTICULAR PURPOSE. ENTIRE RISK QUALITY PERFORMANCE PROGRAM . PROGRAM PROVE DEFECTIVE, ASSUME COST NECESSARY SERVICING, REPAIR CORRECTION.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_16-limitation-of-liability","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"16. Limitation of Liability","title":"GNU General Public License","text":"EVENT UNLESS REQUIRED APPLICABLE LAW AGREED WRITING COPYRIGHT HOLDER, PARTY MODIFIES /CONVEYS PROGRAM PERMITTED , LIABLE DAMAGES, INCLUDING GENERAL, SPECIAL, INCIDENTAL CONSEQUENTIAL DAMAGES ARISING USE INABILITY USE PROGRAM (INCLUDING LIMITED LOSS DATA DATA RENDERED INACCURATE LOSSES SUSTAINED THIRD PARTIES FAILURE PROGRAM OPERATE PROGRAMS), EVEN HOLDER PARTY ADVISED POSSIBILITY DAMAGES.","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"id_17-interpretation-of-sections-15-and-16","dir":"","previous_headings":"TERMS AND CONDITIONS","what":"17. Interpretation of Sections 15 and 16","title":"GNU General Public License","text":"disclaimer warranty limitation liability provided given local legal effect according terms, reviewing courts shall apply local law closely approximates absolute waiver civil liability connection Program, unless warranty assumption liability accompanies copy Program return fee. END TERMS CONDITIONS","code":""},{"path":"https://shikokuchuo.net/mirai/LICENSE.html","id":"how-to-apply-these-terms-to-your-new-programs","dir":"","previous_headings":"","what":"How to Apply These Terms to Your New Programs","title":"GNU General Public License","text":"develop new program, want greatest possible use public, best way achieve make free software everyone can redistribute change terms. , attach following notices program. safest attach start source file effectively state exclusion warranty; file least “copyright” line pointer full notice found. Also add information contact electronic paper mail. program terminal interaction, make output short notice like starts interactive mode: hypothetical commands show w show c show appropriate parts General Public License. course, program’s commands might different; GUI interface, use “box”. also get employer (work programmer) school, , sign “copyright disclaimer” program, necessary. information , apply follow GNU GPL, see . GNU General Public License permit incorporating program proprietary programs. program subroutine library, may consider useful permit linking proprietary applications library. want , use GNU Lesser General Public License instead License. first, please read .","code":" Copyright (C) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . Copyright (C) This program comes with ABSOLUTELY NO WARRANTY; for details type 'show w'. This is free software, and you are welcome to redistribute it under certain conditions; type 'show c' for details."},{"path":"https://shikokuchuo.net/mirai/articles/databases.html","id":"database-hosting---the-basics","dir":"Articles","previous_headings":"","what":"Database Hosting - The Basics","title":"mirai - Databases and Arrow","text":"mirai supports hosting multiple database connections across processes local machine remote server. everywhere() easily sets identical database connections daemon process. following represents simple example, sets 2 local daemons, opens connection SQLite file database daemon. mirai() calls may used write query database, may executed either daemon. everywhere() can used cleanly tear databases, resetting daemons.","code":"file <- tempfile() library(mirai) daemons(2) #> [1] 2 everywhere({ library(DBI) con <<- dbConnect(RSQLite::SQLite(), file) }, file = file) m <- mirai(dbWriteTable(con, \"iris\", iris)) m[] #> [1] TRUE m <- mirai(dbListTables(con)) m[] #> [1] \"iris\" m <- mirai(dbGetQuery(con, 'SELECT * FROM iris WHERE \"Sepal.Length\" < 4.6')) m[] #> Sepal.Length Sepal.Width Petal.Length Petal.Width Species #> 1 4.4 2.9 1.4 0.2 setosa #> 2 4.3 3.0 1.1 0.1 setosa #> 3 4.4 3.0 1.3 0.2 setosa #> 4 4.5 2.3 1.3 0.3 setosa #> 5 4.4 3.2 1.3 0.2 setosa everywhere(dbDisconnect(con)) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/databases.html","id":"database-hosting---using-arrow-database-connectivity","dir":"Articles","previous_headings":"","what":"Database Hosting - Using Arrow Database Connectivity","title":"mirai - Databases and Arrow","text":"possible using DBI interface access manipulate data Apache Arrow data format efficiently ABDC (Arrow Database Connectivity). example creates -memory SQLite connection using adbcsqlite backend. Serialization set relevant serialization functions arrow package part everywhere() call. Note specified class ‘nanoarrow_array_stream’ nanoarrow backend queries made DBI db*Arrow() functions. mirai() calls may used write query database Arrow format. Due tight integration mirai serialization mechanism R’s ‘refhook’ system, can easily return complex / nested objects containing multiple queries Arrow format: , everywhere() can used cleanly tear databases, resetting daemons.","code":"library(mirai) daemons(1) #> [1] 1 cfg <- serial_config( class = \"nanoarrow_array_stream\", sfunc = arrow::write_to_raw, ufunc = function(x) arrow::read_ipc_stream(x, as_data_frame = FALSE) ) everywhere( { library(DBI) # `adbi` and `adbcsqlite` packages must also be installed con <<- dbConnect(adbi::adbi(\"adbcsqlite\"), uri = \":memory:\") }, .serial = cfg ) m <- mirai(dbWriteTableArrow(con, \"iris\", iris)) m[] #> [1] TRUE m <- mirai(dbReadTableArrow(con, \"iris\")) m[] #> Table #> 150 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species m <- mirai(dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Sepal.Length\" < 4.6')) m[] #> Table #> 5 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species m <- mirai({ a <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Sepal.Length\" < 4.6') b <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Sepal.Width\" < 2.6') x <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Petal.Length\" < 1.5') y <- dbGetQueryArrow(con, 'SELECT * FROM iris WHERE \"Petal.Width\" < 0.2') list(sepal = list(length = a, width = b), petal = list(length = x, width = y)) }) m[] #> $sepal #> $sepal$length #> Table #> 5 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species #> #> $sepal$width #> Table #> 19 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species #> #> #> $petal #> $petal$length #> Table #> 24 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species #> #> $petal$width #> Table #> 5 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species everywhere(dbDisconnect(con)) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/databases.html","id":"shiny-mirai-dbi-adbc-integrated-example","dir":"Articles","previous_headings":"","what":"Shiny / mirai / DBI / ADBC Integrated Example","title":"mirai - Databases and Arrow","text":"following example database connections hosted mirai daemons may used power Shiny app. one-time serialization() setup ensures seamless transport Apache Arrow data, occurs global environment outside server(). new database connection created new daemon process every new Shiny session. resources freed sesssion ends. logic defined within server(). unique ID used identify session, specified ‘compute profile’ daemons. Non-dispatcher daemons created scheduling required (queries expected take roughly time, case session uses one daemon anyway). Shiny ExtendedTask used perform query via mirai() call, using session-specific compute profile.","code":"library(mirai) library(secretbase) library(shiny) library(bslib) # create an Arrow serialization configuration cfg <- serial_config( class = \"nanoarrow_array_stream\", sfunc = arrow::write_to_raw, ufunc = nanoarrow::read_nanoarrow ) # write 'iris' dataset to temp database file (for this demonstration) file <- tempfile() con <- DBI::dbConnect(adbi::adbi(\"adbcsqlite\"), uri = file) DBI::dbWriteTableArrow(con, \"iris\", iris) DBI::dbDisconnect(con) # common input parameters slmin <- min(iris$Sepal.Length) slmax <- max(iris$Sepal.Length) ui <- page_fluid( p(\"The time is \", textOutput(\"current_time\", inline = TRUE)), hr(), h3(\"Shiny / mirai / DBI / ADBC demonstration\"), p(\"New daemon-hosted database connection is created for every Shiny session\"), sliderInput( \"sl\", \"Query iris dataset based on Sepal Length\", min = slmin, max = slmax, value = c(slmin, slmax), width = \"75%\" ), input_task_button(\"btn\", \"Return query\"), tableOutput(\"table\") ) # uses Shiny ExtendedTask with mirai server <- function(input, output, session) { # create unique session id by hashing current time with a random key id <- secretbase::siphash13(Sys.time(), key = nanonext::random(4L)) # create new (non-dispatcher) daemon for each session daemons(1L, dispatcher = \"none\", .compute = id) # tear down daemon when session ends session$onEnded(function() daemons(0L, .compute = id)) # everywhere() loads DBI and creates ADBC connection in each daemon # and sets up serialization everywhere( { library(DBI) # `adbi` and `adbcsqlite` packages must also be installed con <<- dbConnect(adbi::adbi(\"adbcsqlite\"), uri = file) }, file = file, .serial = cfg, .compute = id ) output$current_time <- renderText({ invalidateLater(1000) format(Sys.time(), \"%H:%M:%S %p\") }) task <- ExtendedTask$new( function(...) mirai( dbGetQueryArrow( con, sprintf( \"SELECT * FROM iris WHERE \\\"Sepal.Length\\\" BETWEEN %.2f AND %.2f\", sl[1L], sl[2L] ) ), ..., .compute = id ) ) |> bind_task_button(\"btn\") observeEvent(input$btn, task$invoke(sl = input$sl)) output$table <- renderTable(task$result()) } # run Shiny app shinyApp(ui = ui, server = server) # deletes temp database file (for this demonstration) unlink(file)"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"table-of-contents","dir":"Articles","previous_headings":"","what":"Table of Contents","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Example 1: Compute-intensive Operations Example 2: /O-bound Operations Example 3: Resilient Pipelines Daemons: Local Persistent Processes Distributed Computing: Remote Daemons Distributed Computing: Launching Daemons Distributed Computing: TLS Secure Connections Compute Profiles Errors, Interrupts Timeouts Serialization - Arrow, polars beyond Asynchronous Parallel Map Using mirai Package","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"example-1-compute-intensive-operations","dir":"Articles","previous_headings":"","what":"Example 1: Compute-intensive Operations","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use case: minimise execution times performing long-running tasks concurrently separate processes. Multiple long computes (model fits etc.) can performed parallel available computing cores. Use mirai() evaluate expression asynchronously separate, clean R process. following mimics expensive calculation eventually returns random value. mirai expression evaluated another process hence must self-contained, referring variables already exist . , variables time mean passed part mirai() call. ‘mirai’ object returned immediately - creating mirai never blocks session. Whilst async operation ongoing, attempting access mirai’s data yields ‘unresolved’ logical NA. check whether mirai remains unresolved (yet complete): wait collect return value, use mirai’s [] method: mirai represents async operation, never necessary wait . code can continue run. completes, return value automatically becomes available $data. easy programmatic use mirai(), ‘.expr’ accepts pre-constructed language object, also list named arguments passed via ‘.args’. , following equivalent : « Back ToC","code":"library(mirai) x <- list(time = 2L, mean = 4) m <- mirai({Sys.sleep(time); rnorm(5L, mean)}, time = x$time, mean = x$mean) m #> < mirai [] > m$data #> 'unresolved' logi NA unresolved(m) #> [1] TRUE m[] #> [1] 4.021118 4.931766 5.243967 3.375100 5.667178 m #> < mirai [$data] > m$data #> [1] 4.021118 4.931766 5.243967 3.375100 5.667178 expr <- quote({Sys.sleep(time); rnorm(5L, mean)}) args <- list(time = x$time, mean = x$mean) m <- mirai(.expr = expr, .args = args) m[] #> [1] 2.344214 5.151972 2.745938 5.707968 3.244619"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"example-2-io-bound-operations","dir":"Articles","previous_headings":"","what":"Example 2: I/O-bound Operations","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use case: ensure execution flow main process blocked. High-frequency real-time data written file/database synchronously without disrupting execution flow. Cache data memory use mirai() perform periodic write operations concurrently separate process. , ‘.args’ used pass environment(), calling environment. provides convenient method passing existing objects. ‘mirai’ object returned immediately. unresolved() may used control flow statements perform actions depend resolution ‘mirai’, . means need actually wait (block) ‘mirai’ resolve, example demonstrates. Now actions depend resolution may processed, example next write. « Back ToC","code":"library(mirai) x <- rnorm(1e6) file <- tempfile() m <- mirai(write.csv(x, file = file), .args = environment()) # unresolved() queries for resolution itself so no need to use it again within the while loop while (unresolved(m)) { cat(\"while unresolved\\n\") Sys.sleep(0.5) } #> while unresolved #> while unresolved cat(\"Write complete:\", is.null(m$data)) #> Write complete: TRUE"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"example-3-resilient-pipelines","dir":"Articles","previous_headings":"","what":"Example 3: Resilient Pipelines","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use case: isolating code can potentially fail separate process ensure continued uptime. part data science / machine learning pipeline, iterations model training may periodically fail stochastic uncontrollable reasons (e.g. buggy memory management graphics cards). Running iteration ‘mirai’ isolates potentially-problematic code even fail, bring entire pipeline. , testing return value ‘mirai’ errors, error-handling code able automate recovery re-attempts, example. details error handling can found section . end result resilient fault-tolerant pipeline minimises downtime eliminating interruptions long computes. « Back ToC","code":"library(mirai) run_iteration <- function(i) { if (runif(1) < 0.1) stop(\"random error\\n\", call. = FALSE) # simulates a stochastic error rate sprintf(\"iteration %d successful\\n\", i) } for (i in 1:10) { m <- mirai(run_iteration(i), environment()) while (is_error_value(call_mirai(m)$data)) { cat(m$data) m <- mirai(run_iteration(i), environment()) } cat(m$data) } #> iteration 1 successful #> iteration 2 successful #> Error: random error #> iteration 3 successful #> iteration 4 successful #> iteration 5 successful #> iteration 6 successful #> iteration 7 successful #> iteration 8 successful #> iteration 9 successful #> Error: random error #> iteration 10 successful"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"daemons-local-persistent-processes","dir":"Articles","previous_headings":"","what":"Daemons: Local Persistent Processes","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Daemons, persistent background processes, may set receive ‘mirai’ requests. potentially efficient new processes longer need created ad hoc basis. Daemons inherit default system configuration read relevant ‘.Renviron’ ‘.Rprofile’ etc. startup. also load default packages. instead load base package (cuts half R’s startup time), environment variable R_SCRIPT_DEFAULT_PACKAGES=NULL may set prior launching daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"with-dispatcher-default","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"With Dispatcher (default)","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Call daemons() specifying number daemons launch. view current status, status() provides number active connections along matrix statistics daemon. default dispatcher = \"process\" creates dispatcher() background process connects individual daemon processes local machine. ensures tasks dispatched efficiently first-first-(FIFO) basis daemons processing. Tasks queued dispatcher sent daemon soon can accept task immediate execution. Dispatcher uses synchronisation primitives nanonext, waiting upon rather polling tasks, efficient terms consuming resources waiting, also fully synchronised events (latency). Alternatively, specifying dispatcher = \"thread\" runs dispatcher logic thread, faster efficient alternative separate background process. new feature considered experimental. Set number daemons zero reset. reverts default creating new background process ‘mirai’ request.","code":"daemons(6) #> [1] 6 status() #> $connections #> [1] 1 #> #> $daemons #> i online instance assigned complete #> abstract://f52b35b9a832a2152cb68451 1 1 1 0 0 #> abstract://033141983db5a787d41c6b62 2 1 1 0 0 #> abstract://f5b1e54c4524b0ad70f97cb1 3 1 1 0 0 #> abstract://0c2dde598cb4db7faea10fa5 4 1 1 0 0 #> abstract://4c233fb0f8673194b1f4e043 5 1 1 0 0 #> abstract://32c59496251de9055106a772 6 1 1 0 0 daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"without-dispatcher","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"Without Dispatcher","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Alternatively, specifying dispatcher = 'none, background daemons connect directly host process. Requesting status now shows 6 connections, along host URL $daemons. implementation sends tasks immediately, ensures tasks evenly-distributed amongst daemons. means optimal scheduling guaranteed duration tasks known priori. example, tasks queued daemon behind long-running task, whilst daemons idle already completed tasks. advantage approach low-level require additional dispatcher process. well-suited working similar-length tasks, number concurrent tasks typically exceed available daemons.","code":"daemons(6, dispatcher = \"none\") #> [1] 6 status() #> $connections #> [1] 6 #> #> $daemons #> [1] \"abstract://d47e6c20df78276654d7893e\""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"everywhere","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"Everywhere","title":"mirai - Minimalist Async Evaluation Framework for R","text":"everywhere() may used evaluate expression connected daemons persist resultant state, regardless daemon’s ‘cleanup’ setting. keeps DBI package loaded evaluations. types setup task may also performed, including making common resource available, database connection: super-assignment, conenction ‘con’ available global environment daemon instances. Subsequent mirai calls may make use ‘con’. Disconnect database everywhere, set number daemons zero reset.","code":"everywhere(library(DBI)) file <- tempfile() everywhere(con <<- dbConnect(RSQLite::SQLite(), file), file = file) m <- mirai(capture.output(str(con))) m[] #> [1] \"Formal class 'SQLiteConnection' [package \\\"RSQLite\\\"] with 8 slots\" #> [2] \" ..@ ptr : \" #> [3] \" ..@ dbname : chr \\\"/tmp/Rtmp8RYD0H/file38fb134e0316\\\"\" #> [4] \" ..@ loadable.extensions: logi TRUE\" #> [5] \" ..@ flags : int 70\" #> [6] \" ..@ vfs : chr \\\"\\\"\" #> [7] \" ..@ ref : \" #> [8] \" ..@ bigint : chr \\\"integer64\\\"\" #> [9] \" ..@ extended_types : logi FALSE\" everywhere(dbDisconnect(con)) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"with-method","dir":"Articles","previous_headings":"Daemons: Local Persistent Processes","what":"With Method","title":"mirai - Minimalist Async Evaluation Framework for R","text":"daemons() () method, evaluates expression daemons created duration expression automatically torn upon completion. designed use case running Shiny app desired number daemons. Note: case, assumed app already created. Wrapping call shiny::shinyApp() work runApp() implicitly called app printed, however printing occurs () returned, hence app run outside scope () statement. « Back ToC","code":"with(daemons(4), shiny::runApp(app))"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"distributed-computing-remote-daemons","dir":"Articles","previous_headings":"","what":"Distributed Computing: Remote Daemons","title":"mirai - Minimalist Async Evaluation Framework for R","text":"daemons interface may also used send tasks computation remote daemon processes network. Call daemons() specifying ‘url’ character string : ‘tcp://10.75.32.70:5555’ daemon processes connect . Alternatively, use host_url() automatically construct valid URL. IPv6 addresses also supported must enclosed square brackets [] avoid confusion final colon separating port. example, port 5555 IPv6 address ::ffff:a6f:50d specified tcp://[::ffff:a6f:50d]:5555. options actually launching daemons, please see next section.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"connecting-to-remote-daemons-through-dispatcher","dir":"Articles","previous_headings":"Distributed Computing: Remote Daemons","what":"Connecting to Remote Daemons Through Dispatcher","title":"mirai - Minimalist Async Evaluation Framework for R","text":"default dispatcher = \"process\" alternatively dispatcher = \"thread\", dispatcher listens vector URLs remote daemon() processes dial , daemon unique URL. recommended use websocket URL starting ws:// instead TCP scenario (used interchangeably tcp://). websocket URL supports path port number, can made unique daemon. way dispatcher can connect arbitrary number daemons single port. Supplying vector URLs allows use arbitrary port numbers / paths. ‘n’ need specified can inferred length ‘url’ vector, example: Alternatively, single URL supplied, along n = 4 specify dispatcher listen 4 URLs. case, integer sequence automatically appended path /1 /4 produce URLs. Requesting status host machine: per local case, $connections shows single connection dispatcher, however $daemons now provides matrix statistics remote daemons. index number. online shows 1 active connection, else 0 daemon yet connect disconnected. instance increments 1 every time new connection URL. counter designed track new daemon instances connecting previous ones ended (due time-outs etc.). count becomes negative immediately URL regenerated saisei(), increments new daemon connects. assigned shows cumulative number tasks assigned daemon. complete shows cumulative number tasks completed daemon. Dispatcher automatically adjusts number daemons actually connected. Hence possible dynamically scale number daemons according requirements (limited ‘n’ URLs assigned). reset connections revert default behaviour: Closing connection causes dispatcher exit automatically, turn connected daemons respective connections dispatcher terminated.","code":"daemons(url = c(\"ws://10.75.32.70:5566/cpu\", \"ws://10.75.32.70:5566/gpu\", \"ws://10.75.32.70:7788/1\")) daemons(n = 4, url = host_url(port = 5555)) #> [1] 4 status() #> $connections #> [1] 1 #> #> $daemons #> i online instance assigned complete #> tcp://hostname:5555 1 0 0 0 0 #> tcp://hostname:5556 2 0 0 0 0 #> tcp://hostname:5557 3 0 0 0 0 #> tcp://hostname:5558 4 0 0 0 0 daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"connecting-to-remote-daemons-directly","dir":"Articles","previous_headings":"Distributed Computing: Remote Daemons","what":"Connecting to Remote Daemons Directly","title":"mirai - Minimalist Async Evaluation Framework for R","text":"specifying dispatcher = \"none\", remote daemons connect directly host process. host listens single URL, distributes tasks connected daemons. Note , calling host_url() without port value uses default ‘0’. wildcard value automatically cause free ephemeral port assigned. actual assigned port may queried time via status(): number daemons connecting host URL limited network resources may added removed time, tasks automatically distributed connected daemons. status query , $connections shows actual number connected daemons. reset connections revert default behaviour: causes connected daemons exit automatically. « Back ToC","code":"daemons(url = host_url(), dispatcher = \"none\") #> [1] 0 status() #> $connections #> [1] 0 #> #> $daemons #> [1] \"tcp://hostname:42771\" daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"distributed-computing-launching-daemons","dir":"Articles","previous_headings":"","what":"Distributed Computing: Launching Daemons","title":"mirai - Minimalist Async Evaluation Framework for R","text":"launch remote daemons, supply remote launch configuration ‘remote’ argument daemons() setting daemons, launch_remote() time afterwards. ssh_config() may used generate remote launch configuration SSH access remote machine, else remote_config() provides flexible method generating configuration involving custom resource manager / application.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"ssh-direct-connection","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"SSH Direct Connection","title":"mirai - Minimalist Async Evaluation Framework for R","text":"first example launches 4 daemons machine 10.75.32.90 (using default SSH port 22 specified), connecting back dispatcher URLs: second example launches one daemon 10.75.32.90 10.75.32.91 using custom SSH port 222: examples, remote daemons connect back directly, port 5555 local machine must open incoming connections remote addresses.","code":"daemons( n = 4, url = host_url(ws = TRUE, port = 5555), remote = ssh_config(remotes = \"ssh://10.75.32.90\") ) daemons( n = 2, url = host_url(ws = TRUE, port = 5555), remote = ssh_config(c(\"ssh://10.75.32.90:222\", \"ssh://10.75.32.91:222\")) )"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"ssh-tunnelling","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"SSH Tunnelling","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Use SSH tunnelling provides convenient way launch remote daemons without requiring remote machine able access host. Often firewall configurations security policies may prevent opening port accept outside connections. cases SSH tunnelling offers solution creating tunnel initial SSH connection made. simplicity, SSH tunnelling implementation uses port side host corresponding node. SSH key-based authentication must also already place. Tunnelling requires hostname ‘url’ specified setting daemons either ‘127.0.0.1’ ‘localhost’. tunnel created 127.0.0.1:port equivalently localhost:port machine. host listens localhost:port remotes dial localhost:port respective machines. example launches 2 nodes remote machine 10.75.32.90 using SSH tunnelling port 5555 (‘url’ hostname specified ‘localhost’):","code":"daemons( url = \"tcp://localhost:5555\", remote = ssh_config( remotes = c(\"ssh://10.75.32.90\", \"ssh://10.75.32.90\"), tunnel = TRUE ) )"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"cluster-resource-managers","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"Cluster Resource Managers","title":"mirai - Minimalist Async Evaluation Framework for R","text":"remote_config() may used run command deploy daemons using resource manager. Taking Slurm example, following uses sbatch launch 2 daemons cluster, additional arguments sbatch specifying resource allocation:","code":"daemons( n = 2, url = host_url(ws = TRUE), remote = remote_config( command = \"sbatch\", args = c(\"--mem 512\", \"-n 1\", \"--wrap\", \".\"), rscript = file.path(R.home(\"bin\"), \"Rscript\"), quote = TRUE ) )"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"manual-deployment","dir":"Articles","previous_headings":"Distributed Computing: Launching Daemons","what":"Manual Deployment","title":"mirai - Minimalist Async Evaluation Framework for R","text":"alternative automated launches, calling launch_remote() without specifying ‘remote’ may used return shell commands deploying daemons manually. printed return values may copy / pasted directly remote machine. Note daemons() set host machine launching daemon() remote resources, otherwise daemon instances exit connection immediately available. Alternatively, specifying argument autoexit = FALSE allow daemons wait (indefinitely) connection become available. « Back ToC","code":"daemons(n = 2, url = host_url()) #> [1] 2 launch_remote(1:2) #> [1] #> Rscript -e 'mirai::daemon(\"tcp://hostname:45023\",rs=c(10407,-1977357818,1537914063,-944747324,685206773,301969714,384986891))' #> #> [2] #> Rscript -e 'mirai::daemon(\"tcp://hostname:35177\",rs=c(10407,-2118558994,-1471211395,-214113251,1708896166,-1342731093,-35397167))' daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"distributed-computing-tls-secure-connections","dir":"Articles","previous_headings":"","what":"Distributed Computing: TLS Secure Connections","title":"mirai - Minimalist Async Evaluation Framework for R","text":"TLS available option secure communications local machine remote daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"zero-configuration","dir":"Articles","previous_headings":"Distributed Computing: TLS Secure Connections","what":"Zero-configuration","title":"mirai - Minimalist Async Evaluation Framework for R","text":"automatic zero-configuration default implemented. Simply specify secure URL form wss:// tls+tcp:// setting daemons, use host_url(tls = TRUE), example: Single-use keys certificates automatically generated configured, without requiring intervention. private key always retained host machine never transmitted. generated self-signed certificate available via launch_remote(). function conveniently constructs full shell command launch daemon, including correctly specified ‘tls’ argument daemon(). printed value may deployed directly remote machine. « Back ToC","code":"daemons(n = 4, url = host_url(ws = TRUE, tls = TRUE)) #> [1] 4 launch_remote(1) #> [1] #> Rscript -e 'mirai::daemon(\"wss://hostname:33667/1\",tls=c(\"-----BEGIN CERTIFICATE----- #> MIIFNzCCAx+gAwIBAgIBATANBgkqhkiG9w0BAQsFADAzMREwDwYDVQQDDAhrdW1h #> bW90bzERMA8GA1UECgwITmFub25leHQxCzAJBgNVBAYTAkpQMB4XDTAxMDEwMTAw #> MDAwMFoXDTMwMTIzMTIzNTk1OVowMzERMA8GA1UEAwwIa3VtYW1vdG8xETAPBgNV #> BAoMCE5hbm9uZXh0MQswCQYDVQQGEwJKUDCCAiIwDQYJKoZIhvcNAQEBBQADggIP #> ADCCAgoCggIBAMwra+feTABrzKCo8WrHmZg/rVUSz1oGNS8V1xNyrGMtdks7+PSu #> Wo8Zl4BYBUHKEWKMePsa6Xs5fjmxxoBj4GMbGP7y+YLWmZ5GEkjKCj4XqSX3vcgM #> Iw5iO+KsPnZAFfvYmXt1vXDwklcoHiFYO3A1YKRNrKP0RpIHCnsn2aLlinBw5bpf #> ZbnaJUcLvg3YM+K5KYvH5JTpE5dnrFMD00por0OGwhT/nUGng5ZAj9Cc/dzFjkDd #> s+UrHxwFbUfjaeiw4VV3vHr6mJEDWNYCdBrBr0q6kaiZUOJN+5DcUriXx059WfEw #> oDtJgY+9urVpCMsLlKWg4A14jnAVDGLexp/LxxsgEHIfG+Co95QBKCFLslQgS9PO #> HVE9bpEtXOOwAHnYzImfu3lR4uroiMWDSN/9XkLG01iIRYeAgSch7k2jDTvR/L6v #> AywWigfifHX3qGZrqr9uhpPukhXQ2lhN4PG+vLkmowovscIY0l7vDXOsVW+B2APH #> nZUjmd1tyKI7vvqzMb4gqjLIcETfV8a32ATB9zihRnwq4dzv4PTbv8X3ChYAaVvQ #> PL3CKXw4FIQGctwbTiSopsp5CfoJ9Dy6vregoe4+33KsMwYV3nGATpGZ+YMb4uf2 #> xEqN/0jrSJmEQqrIfhUxjwBQDuKbEsnmE/jQJTbJTdPneMhkf4wcOBerAgMBAAGj #> VjBUMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0OBBYEFHduHiXzvsvbh82mVQTA #> m3mgEwAsMB8GA1UdIwQYMBaAFHduHiXzvsvbh82mVQTAm3mgEwAsMA0GCSqGSIb3 #> DQEBCwUAA4ICAQA9ONCt0Ms6T9pz8jOiS++aEfYd9rOxAPHjqV0WtEIt5XqgbLmy #> UBqEho3gW6gPXT99AOzAAF93QHIoSHeaLLskbRsO5AtvhaFUx0D9tkcZCk46uT4i #> E4vk+H64r/EAaedkowA7c54vwHP+0XnDp8FKWDQaj9UWFC7WPnoYiAqu274YHpP2 #> HDneTf2qb0+ALI52/8EL7YKbsAOpi42x35die9qS2zYbA8CNThDvWsMFCviC8bgM #> YAnH/5bqrzh9VyFn6EasDk7NXt8Gyadn7avZRWDjbKvEwrBbvmoWUJh0YBShMZSy #> I+88xHid9WJWJYGNL7fBvVxzfnew8cVU0D20YWK2WN/JMrNKg570fhIGSXW2rNRw #> x/hZ7a3JdADmUSMV8YRZmLIx48P/X7CgEyhZZVWWxPez0vz6srOOxLvxTod84CDB #> ApyWcm5GylpkRtMZdX4fkvZtc22Hi21cjZTjL1OU8olRgjycRuxqJI+zBMlepUk+ #> jYGcMp4mczoK+KCqZI9BR5M4hs4nurE2QDaeEW8gC6tM4KlP5O+dI+CjXb8QV6IZ #> VgfxXVSch+9LpCynaQf+JdomygH35d+XYsps7KjkASJeig9poT7QdDqpWSAyID21 #> RAWLnGaioVrEqp5SDiVEGyh0bX4fmSFWXv1TorfLDNbB1GAm6pGXn8yLjQ== #> -----END CERTIFICATE----- #> \",\"\"),rs=c(10407,-190365515,-1460862478,1356191947,1879828880,1039554641,2031211998))'"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"ca-signed-certificates","dir":"Articles","previous_headings":"Distributed Computing: TLS Secure Connections","what":"CA Signed Certificates","title":"mirai - Minimalist Async Evaluation Framework for R","text":"alternative zero-configuration default, certificate may also generated via Certificate Signing Request (CSR) Certificate Authority (CA), may public CA CA internal organisation. Generate private key CSR. following resources describe : using Mbed TLS: https://mbed-tls.readthedocs.io/en/latest/kb/-/generate--certificate-request-csr/ using OpenSSL: https://www.feistyduck.com/library/openssl-cookbook/online/ (Chapter 1.2 Key Certificate Management) Provide generated CSR CA sign new TLS certificate. common name (CN) certificate must identical hostname IP address actually used connection. verified, fail . received certificate comprise block cipher text markers -----BEGIN CERTIFICATE----- -----END CERTIFICATE-----. Make sure request certificate PEM format. available formats, TLS library used usually provide conversion utilities. Check also private key block cipher text markers -----BEGIN PRIVATE KEY----- -----END PRIVATE KEY-----. setting daemons, TLS certificate private key provided ‘tls’ argument daemons(). certificate private key imported character strings cert key respectively, ‘tls’ argument may specified character vector c(cert, key). Alternatively, certificate may copied new text file, private key appended, case path/filename file may provided ‘tls’ argument. launching daemons, certificate chain CA supplied ‘tls’ argument daemon() launch_remote(). certificate chain comprise multiple certificates, -----BEGIN CERTIFICATE----- -----END CERTIFICATE----- markers. first one newly-generated TLS certificate, supplied daemons(), final one CA root certificate. certificates required certificate signed directly CA. , intermediate certificates included certificate chain starts TLS certificate ends certificate CA. concatenated together single character string certchain, character vector comprising empty character string c(certchain, \"\") may supplied relevant ‘tls’ argument. Alternatively, written file (file replicated remote machines), ‘tls’ argument may also specified path/filename (assuming machine). « Back ToC","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"compute-profiles","dir":"Articles","previous_headings":"","what":"Compute Profiles","title":"mirai - Minimalist Async Evaluation Framework for R","text":"daemons() interface also allows specification compute profiles managing tasks heterogeneous compute requirements: send tasks different daemons clusters daemons appropriate specifications (terms CPUs / memory / GPU / accelerators etc.) split tasks local remote computation Simply specify argument .compute calling daemons() profile name (‘default’ default profile). daemons settings saved named profile. create ‘mirai’ task using specific compute profile, specify ‘.compute’ argument mirai(), defaults ‘default’ compute profile. Similarly, functions status(), launch_local() launch_remote() specified desired ‘.compute’ argument. « Back ToC","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"errors-interrupts-and-timeouts","dir":"Articles","previous_headings":"","what":"Errors, Interrupts and Timeouts","title":"mirai - Minimalist Async Evaluation Framework for R","text":"execution mirai fails, error message returned character string class ‘miraiError’ ‘errorValue’ facilitate debugging. is_mirai_error() may used test mirai execution errors. full stack trace evaluation within mirai recorded accessible $stack.trace error object. daemon instance sent user interrupt, mirai resolve object class ‘miraiInterrupt’ ‘errorValue’. is_mirai_interrupt() may used test interrupts. execution mirai surpasses timeout set via ‘.timeout’ argument, mirai resolve ‘errorValue’ 5L (timed ). can, amongst things, guard mirai processes potential hang never return. is_error_value() tests mirai execution errors, user interrupts timeouts. « Back ToC","code":"m1 <- mirai(stop(\"occurred with a custom message\", call. = FALSE)) m1[] #> 'miraiError' chr Error: occurred with a custom message m2 <- mirai(mirai::mirai()) m2[] #> 'miraiError' chr Error in mirai::mirai(): missing expression, perhaps wrap in {}? is_mirai_error(m2$data) #> [1] TRUE is_error_value(m2$data) #> [1] TRUE f <- function(x) if (x > 0) stop(\"positive\") m3 <- mirai({f(-1); f(1)}, f = f) m3[] #> 'miraiError' chr Error in f(1): positive m3$data$stack.trace #> [[1]] #> [1] \"stop(\\\"positive\\\")\" #> #> [[2]] #> [1] \"f(1)\" m4 <- mirai(rlang::interrupt()) # simulates a user interrupt is_mirai_interrupt(m4[]) #> [1] TRUE m4 <- mirai(nanonext::msleep(1000), .timeout = 500) m4[] #> 'errorValue' int 5 | Timed out is_mirai_error(m4$data) #> [1] FALSE is_mirai_interrupt(m4$data) #> [1] FALSE is_error_value(m4$data) #> [1] TRUE"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"serialization-arrow-polars-and-beyond","dir":"Articles","previous_headings":"","what":"Serialization: Arrow, polars and beyond","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Native R serialization used sending data host daemons. R objects nature serialized, accessed via external pointer. cases, performing ‘mirai’ operations normally error. Using arrow package example: However, serial_config() can used create custom serialization configurations, specifying functions hook R’s native serialization mechanism reference objects (‘refhooks’). configuration can specified part everywhere() call via ‘.serial’ argument. can seen time, arrow table seamlessly handled ‘mirai’ process. case even object deeply nested inside lists structures. change registered serialization functions, just call everywhere() supplying new functions. example, can switch using polars, ‘lightning fast’ dataframe library written Rust (requires polars >= 0.16.4). cancel serialization functions entirely, supply empty list ‘.serial’ argument everywhere(): ‘vec’ argument serialization() may specified TRUE serialization functions vectorized take lists objects, case safetensors, used serialization torch. Please refer torch vignette examples. « Back ToC","code":"library(arrow, warn.conflicts = FALSE) daemons(2) #> [1] 2 everywhere(library(arrow)) x <- as_arrow_table(iris) m <- mirai(list(a = head(x), b = \"some text\"), x = x) m[] #> 'miraiError' chr Error: Invalid
, external pointer to null cfg <- serial_config( class = \"ArrowTabular\", sfunc = arrow::write_to_raw, ufunc = function(x) arrow::read_ipc_stream(x, as_data_frame = FALSE) ) daemons(2) #> [1] 2 everywhere(library(arrow), .serial = cfg) m <- mirai(list(a = head(x), b = \"some text\"), x = x) m[] #> $a #> Table #> 6 rows x 5 columns #> $Sepal.Length #> $Sepal.Width #> $Petal.Length #> $Petal.Width #> $Species > #> #> See $metadata for additional Schema metadata #> #> $b #> [1] \"some text\" everywhere( {}, .serial = serial_config( class = \"RPolarsDataFrame\", sfunc = function(x) polars::as_polars_df(x)$to_raw_ipc(), ufunc = polars::pl$read_ipc ) ) x <- polars::as_polars_df(iris) m <- mirai(list(a = head(x), b = \"some text\"), x = x) m[] #> $a #> shape: (6, 5) #> ┌──────────────┬─────────────┬──────────────┬─────────────┬─────────┐ #> │ Sepal.Length ┆ Sepal.Width ┆ Petal.Length ┆ Petal.Width ┆ Species │ #> │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ #> │ f64 ┆ f64 ┆ f64 ┆ f64 ┆ cat │ #> ╞══════════════╪═════════════╪══════════════╪═════════════╪═════════╡ #> │ 5.1 ┆ 3.5 ┆ 1.4 ┆ 0.2 ┆ setosa │ #> │ 4.9 ┆ 3.0 ┆ 1.4 ┆ 0.2 ┆ setosa │ #> │ 4.7 ┆ 3.2 ┆ 1.3 ┆ 0.2 ┆ setosa │ #> │ 4.6 ┆ 3.1 ┆ 1.5 ┆ 0.2 ┆ setosa │ #> │ 5.0 ┆ 3.6 ┆ 1.4 ┆ 0.2 ┆ setosa │ #> │ 5.4 ┆ 3.9 ┆ 1.7 ┆ 0.4 ┆ setosa │ #> └──────────────┴─────────────┴──────────────┴─────────────┴─────────┘ #> #> $b #> [1] \"some text\" everywhere({}, .serial = list()) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"asynchronous-parallel-map","dir":"Articles","previous_headings":"","what":"Asynchronous Parallel Map","title":"mirai - Minimalist Async Evaluation Framework for R","text":"mirai_map() performs asynchronous parallel/distributed map using mirai. function similar purrr::map(), returns ‘mirai_map’ object. also advanced allows multiple map rows dataframe matrix. results mirai_map x may collected using x[]. waits asynchronous operations complete still progress.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"key-advantages","dir":"Articles","previous_headings":"Asynchronous Parallel Map","what":"Key advantages:","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Returns immediately evaluations taking place asynchronously. Printing ‘mirai map’ object shows current completion progress. ‘.promise’ argument allows promise registered mirai, can used perform side-effects. Returns evaluation errors ‘miraiError’ ‘errorValue’ case may , rather causing entire operation fail. allows efficient recovery partial failure. rely ‘chunking’ algorithm attempts split work batches according number available daemons, implemented example parallel package. Chunking take account varying unpredictable compute times indices. can optimal rely mirai scheduling instead. demonstrated example . .args used specify constant arguments .f - ‘mean’ ‘sd’ example : Use ... specify objects referenced defined .f - ‘’ anonymous function : Use mirai_map() assumes daemons() previously set. one (non-dispatcher) daemon set allow function proceed. ensures safe behaviour, unlikely optimal, please ensure daemons set beforehand.","code":"library(mirai) library(parallel) cl <- make_cluster(4) daemons(4) #> [1] 4 vec <- c(1, 1, 4, 4, 1, 1, 1, 1) system.time(mirai_map(vec, Sys.sleep)[]) #> user system elapsed #> 0.003 0.003 4.007 system.time(parLapply(cl, vec, Sys.sleep)) #> user system elapsed #> 0.010 0.004 8.013 with( daemons(3, dispatcher = \"none\"), mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[] ) #> [[1]] #> [1] 20.27145 #> #> [[2]] #> [1] 20.55287 22.25358 #> #> [[3]] #> [1] 20.18246 20.24844 19.74873 ml <- mirai_map( c(a = 1, b = 2, c = 3), function(x) do(x, as.logical(x %% 2)), do = nanonext::random ) #> Warning: mirai is launching one local daemon for a map operation as none previously set ml #> < mirai map [3/3] > ml[] #> $a #> [1] \"c8\" #> #> $b #> [1] 96 66 #> #> $c #> [1] \"6b20bb\""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"collecting-results","dir":"Articles","previous_headings":"Asynchronous Parallel Map","what":"Collecting Results","title":"mirai - Minimalist Async Evaluation Framework for R","text":"collecting results, optionally specify arguments []: x[.flat] collects flattens results, checking type avoid coercion. x[.progress] collects results whilst showing simple text progress indicator parts completed total. x[.progress_cli] alternative uses cli progress bars, available, showing completion percentage ETA. x[.stop] collects results applying early stopping, stops first failure cancels remaining computations (note: computations already progress continue completion, results collected). Combinations may supplied fashion x[.stop, .progress].","code":"tryCatch( mirai_map(list(a = 1, b = \"a\", c = 3), sum)[.stop], error = identity ) #> with( daemons(4, dispatcher = \"none\"), mirai_map(c(0.1, 0.2, 0.3), Sys.sleep)[.progress, .flat] ) #> NULL"},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"multiple-map","dir":"Articles","previous_headings":"Asynchronous Parallel Map","what":"Multiple Map","title":"mirai - Minimalist Async Evaluation Framework for R","text":"Multiple map performed rows dataframe matrix, often desired behaviour. allows map 2 arguments specifying dataframe. One may index value indexed map. dataframe often contains columns differing type, unusual want map columns, however possible simply transforming beforehand list using .list(). Similarly, behaviour lapply() purrr::map() matrix vector. mirai_map() hand take account fact matrix dimensions, maps rows, consistent behaviour dataframes. instead, mapping columns desired, simply take transpose matrix beforehand using t(). « Back ToC","code":"fruit <- c(\"melon\", \"grapes\", \"coconut\") # create a dataframe for indexed map: df <- data.frame(i = seq_along(fruit), fruit = fruit) with( daemons(3, dispatcher = \"none\"), mirai_map(df, sprintf, .args = list(fmt = \"%d. %s\"))[.flat] ) #> [1] \"1. melon\" \"2. grapes\" \"3. coconut\""},{"path":"https://shikokuchuo.net/mirai/articles/mirai.html","id":"using-mirai-in-a-package","dir":"Articles","previous_headings":"","what":"Using mirai in a Package","title":"mirai - Minimalist Async Evaluation Framework for R","text":"mirai framework designed support completely transparent inter-operable use within packages. core design precept relying global options environment variables minimises likelihood conflict use different packages. hence requirements package authors. following points may, however, useful: daemons() settings usually left end-users. Users may guided mirai documentation desired. however, package wishes set default settings, example, daemons() always called specifying force = FALSE. ensures prior user settings respected, daemons set elsewhere prematurely terminated. Calling package functions mirai requires namespacing call, alternatively exporting function, .e. shape contents status() daemons matrix must relied upon, user interface subject change time. developer interface nextget(), querying values ‘urls’ described function documentation. Note: specifically-documented values supported interfaces. recommended practice case, especially relevant package developers: functions unresolved(), is_error_value(), is_mirai_error(), is_mirai_interrupt() used test relevant state mirai value. characteristics currently implemented, e.g. logical NA ‘unresolvedValue’, relied upon, subject change. Testing CRAN respect ’s 2-core usage limit. practically means limiting tests using one daemon (dispatcher = \"none\") ensure one additional process used. Always reset daemons done allow least one-second sleep ensure background processes properly exited. limits apply tests run CRAN machines run elsewhere. « Back ToC","code":"mirai(mypkg::my_func()) mirai(my_func(), .args = list(myfunc = myfunc))"},{"path":"https://shikokuchuo.net/mirai/articles/parallel.html","id":"parallel-integration","dir":"Articles","previous_headings":"","what":"Parallel Integration","title":"mirai - Parallel Integration","text":"mirai provides alternative communications backend R. functionality developed fulfil request R Core R Project Sprint 2023. make_cluster() creates cluster object class ‘miraiCluster’, fully-compatible parallel cluster types. Specify ‘n’ launch nodes local machine. Specify ‘url’ receiving connections remote nodes. Optionally, specify ‘remote’ launch remote daemons using remote configuration generated remote_config() ssh_config(). Created clusters may used function parallel base package parallel::clusterApply() parallel::parLapply(), load-balanced versions parallel::parLapplyLB(). status() may called ’miraiCluster` query number connected nodes time. Making cluster specifying ‘url’ without ‘remote’ causes shell commands manual deployment nodes printed console.","code":"library(mirai) cl <- make_cluster(4) cl #> < miraiCluster | ID: `0` nodes: 4 active: TRUE > parallel::parLapply(cl, iris, mean) #> $Sepal.Length #> [1] 5.843333 #> #> $Sepal.Width #> [1] 3.057333 #> #> $Petal.Length #> [1] 3.758 #> #> $Petal.Width #> [1] 1.199333 #> #> $Species #> [1] NA status(cl) #> $connections #> [1] 4 #> #> $daemons #> [1] \"abstract://e755083adb3ac002d81d87c7\" stop_cluster(cl) cl <- make_cluster(n = 2, url = host_url()) #> Shell commands for deployment on nodes: #> #> [1] #> Rscript -e 'mirai::daemon(\"tcp://hostname:41567\",rs=c(10407,-1229812554,-355079233,-704283916,613800293,-1049741598,-1259508485))' #> #> [2] #> Rscript -e 'mirai::daemon(\"tcp://hostname:41567\",rs=c(10407,-936602309,482053993,1829342693,1529156475,2009204106,1430761999))' stop_cluster(cl)"},{"path":"https://shikokuchuo.net/mirai/articles/parallel.html","id":"foreach-integration","dir":"Articles","previous_headings":"","what":"Foreach Integration","title":"mirai - Parallel Integration","text":"‘miraiCluster’ may also registered doParallel use foreach package. Running parallel examples foreach() function:","code":"library(foreach) library(iterators) cl <- make_cluster(4) doParallel::registerDoParallel(cl) # normalize the rows of a matrix m <- matrix(rnorm(9), 3, 3) foreach(i = 1:nrow(m), .combine = rbind) %dopar% (m[i, ] / mean(m[i, ])) #> [,1] [,2] [,3] #> result.1 0.6168828 1.1209518 1.2621654 #> result.2 1.9631338 0.3970546 0.6398116 #> result.3 1.8112756 8.5974163 -7.4086918 # simple parallel matrix multiply a <- matrix(1:16, 4, 4) b <- t(a) foreach(b = iter(b, by='col'), .combine = cbind) %dopar% (a %*% b) #> [,1] [,2] [,3] [,4] #> [1,] 276 304 332 360 #> [2,] 304 336 368 400 #> [3,] 332 368 404 440 #> [4,] 360 400 440 480"},{"path":"https://shikokuchuo.net/mirai/articles/plumber.html","id":"plumber-integration","dir":"Articles","previous_headings":"","what":"Plumber Integration","title":"mirai - Plumber Integration","text":"mirai may used asynchronous backend plumber pipelines. Example usage provided different types endpoint.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/plumber.html","id":"example-get-endpoint","dir":"Articles","previous_headings":"Plumber Integration","what":"Example GET Endpoint","title":"mirai - Plumber Integration","text":"plumber router code run daemon process block interactive process. /echo endpoint takes GET request, sleeps 1 second (simulating expensive computation) simply returns ‘msg’ request header together timestamp process ID process run . API can queried using async HTTP client nanonext::ncurl_aio(). , 8 requests submitted , note responses differing timestamps 4 can processed one time (limited number daemons set).","code":"library(mirai) # supply SIGINT so the plumber server is interrupted and exits cleanly when finished daemons(1L, dispatcher = \"none\", autoexit = tools::SIGINT) #> [1] 1 m <- mirai({ library(plumber) library(promises) # to provide the promise pipe library(mirai) # more efficient not to use dispatcher if all requests are similar length daemons(4L, dispatcher = \"none\") # handles 4 requests simultaneously pr() |> pr_get( \"/echo\", function(req, res) { mirai( { Sys.sleep(1L) list( status = 200L, body = list( time = format(Sys.time()), msg = msg, pid = Sys.getpid() ) ) }, msg = req[[\"HEADERS\"]][[\"msg\"]] ) %...>% (function(x) { res$status <- x$status res$body <- x$body }) } ) |> pr_run(host = \"127.0.0.1\", port = 8985) }) library(nanonext) res <- lapply( 1:8, function(i) ncurl_aio( \"http://127.0.0.1:8985/echo\", headers = c(msg = as.character(i)) ) ) collect_aio(res) #> [[1]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"1\\\"],\\\"pid\\\":[81109]}\" #> #> [[2]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"2\\\"],\\\"pid\\\":[81111]}\" #> #> [[3]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"3\\\"],\\\"pid\\\":[81107]}\" #> #> [[4]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:25\\\"],\\\"msg\\\":[\\\"4\\\"],\\\"pid\\\":[81115]}\" #> #> [[5]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"5\\\"],\\\"pid\\\":[81111]}\" #> #> [[6]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"6\\\"],\\\"pid\\\":[81115]}\" #> #> [[7]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"7\\\"],\\\"pid\\\":[81109]}\" #> #> [[8]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:26\\\"],\\\"msg\\\":[\\\"8\\\"],\\\"pid\\\":[81107]}\" daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/plumber.html","id":"example-post-endpoint","dir":"Articles","previous_headings":"Plumber Integration","what":"Example POST Endpoint","title":"mirai - Plumber Integration","text":"demonstration equivalent using POST endpoint, accepting JSON instruction sent request data. Note req$postBody always accessed router process passed argument ‘mirai’, retrieved using connection serializable. Querying endpoint produces set outputs previous example.","code":"library(mirai) # supply SIGINT so the plumber server is interrupted and exits cleanly when finished daemons(1L, dispatcher = \"none\", autoexit = tools::SIGINT) #> [1] 1 m <- mirai({ library(plumber) library(promises) # to provide the promise pipe library(mirai) # uses dispatcher - suitable when requests take differing times to complete daemons(4L, dispatcher = \"thread\") # handles 4 requests simultaneously pr() |> pr_post( \"/echo\", function(req, res) { mirai( { Sys.sleep(1L) # simulate expensive computation list( status = 200L, body = list( time = format(Sys.time()), msg = jsonlite::fromJSON(data)[[\"msg\"]], pid = Sys.getpid() ) ) }, data = req$postBody ) %...>% (function(x) { res$status <- x$status res$body <- x$body }) } ) |> pr_run(host = \"127.0.0.1\", port = 8986) }) library(nanonext) res <- lapply( 1:8, function(i) ncurl_aio( \"http://127.0.0.1:8986/echo\", method = \"POST\", data = sprintf('{\"msg\":\"%d\"}', i) ) ) collect_aio(res) #> [[1]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"1\\\"],\\\"pid\\\":[81336]}\" #> #> [[2]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"2\\\"],\\\"pid\\\":[81338]}\" #> #> [[3]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"3\\\"],\\\"pid\\\":[81344]}\" #> #> [[4]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"4\\\"],\\\"pid\\\":[81338]}\" #> #> [[5]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"5\\\"],\\\"pid\\\":[81336]}\" #> #> [[6]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:29\\\"],\\\"msg\\\":[\\\"6\\\"],\\\"pid\\\":[81340]}\" #> #> [[7]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"7\\\"],\\\"pid\\\":[81340]}\" #> #> [[8]] #> [1] \"{\\\"time\\\":[\\\"2024-10-08 12:21:30\\\"],\\\"msg\\\":[\\\"8\\\"],\\\"pid\\\":[81344]}\" daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/promises.html","id":"promises-integration","dir":"Articles","previous_headings":"","what":"Promises Integration","title":"mirai - Promises Integration","text":"mirai supplies .promise() method, allowing used promise promises package. next-generation, event-driven promises, developed collaboration Joe Cheng. require promise polled completion later loop like promises. Instead, promise actions automatically queued execution soon ‘mirai’ resolves. Allows much higher responsiveness (lower latency) massive scalability (situations thousand promises ). ‘mirai’ may piped directly using promise pipe &...>%, implicitly calls .promise() ‘mirai’. Similarly promise-aware functions promises::() shiny::ExtendedTask$new() take promise can also take ‘mirai’ (using promises >= 1.3.0). Alternatively, ‘mirai’ may explicitly converted promise .promise(), allows using methods $(), $finally() etc. following example outputs “hello” console one second ‘mirai’ resolves. possible access ‘mirai’ value $data use promise enacting side effect (assigning value environment example ). returning top level prompt:","code":"library(mirai) library(promises) p <- mirai({Sys.sleep(1); \"hello\"}) %...>% cat() p #> env <- new.env() m <- mirai({ Sys.sleep(1) \"hello\" }) promises::then(m, function(x) env$res <- x) m[] #> [1] \"hello\" env$res #> [1] \"hello\""},{"path":"https://shikokuchuo.net/mirai/articles/promises.html","id":"the-one-million-promises-challenge","dir":"Articles","previous_headings":"","what":"The One Million Promises Challenge","title":"mirai - Promises Integration","text":"code taken challenge launch collect one million promises. illustration, example scaled ten thousand. one million promises challenge took 6 mins 25 secs complete using Intel i7 11th gen mobile processor 16GB RAM.","code":"library(mirai) daemons(8, dispatcher = \"none\") #> [1] 8 r <- 0 start <- Sys.time() m <- mirai_map(1:10000, \\(x) x, .promise = \\(x) r <<- r + x) Sys.time() - start #> Time difference of 2.722618 secs later::run_now() r #> [1] 50005000 daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"shiny-integration","dir":"Articles","previous_headings":"","what":"Shiny Integration","title":"mirai - Shiny Integration","text":"mirai may used asynchronous backend scale Shiny applications. Depending options suppled daemons(), mirai tasks may distributed across local background processes multiple networked servers efficient performant manner. use Shiny, mirai implements truly event-driven promises, developed collaboration Joe Cheng (creator Shiny). promise polled loop completion (types promise). Instead, promise actions triggered execution soon ‘mirai’ completes (asynchronously). Allows much higher responsiveness (lower latency) massive scalability (situations thousand promises ). mirai natively supports Shiny ExtendedTask create scalable Shiny apps, remain responsive intra-session user, well inter-session multiple concurrent users. ‘mirai’ may used anywhere ‘promise’ ‘future_promise’ accepted (promises >= 1.3.0).","code":""},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"shiny-extendedtask-example-plot-with-clock","dir":"Articles","previous_headings":"Shiny Integration","what":"Shiny ExtendedTask Example: Plot with Clock","title":"mirai - Shiny Integration","text":"example , app remains responsive, clock continuing tick whilst simulated expensive computation running asynchronously parallel process. Also button disabled plot greyed computation complete. wrapping runApp() call (daemons(...), ...) daemons set duration app, exiting automatically app stopped. Thanks Joe Cheng providing examples based. key components using ExtendedTask : UI, use bslib::input_task_button(). button disabled computation prevent additional clicks. server, create ExtendedTask object calling ExtendedTask$new() anonymous function passing ... arguments mirai(), bind button created (1). server, create observer input button, invokes ExtendedTask, passing named arguments anonymous function (hence mirai) . server, create render function output, consumes result ExtendedTask.","code":"library(shiny) library(bslib) library(mirai) ui <- page_fluid( p(\"The time is \", textOutput(\"current_time\", inline = TRUE)), hr(), numericInput(\"n\", \"Sample size (n)\", 100), numericInput(\"delay\", \"Seconds to take for plot\", 5), input_task_button(\"btn\", \"Plot uniform distribution\"), plotOutput(\"plot\") ) server <- function(input, output, session) { output$current_time <- renderText({ invalidateLater(1000) format(Sys.time(), \"%H:%M:%S %p\") }) extended_task <- ExtendedTask$new( function(...) mirai({Sys.sleep(y); runif(x)}, ...) ) |> bind_task_button(\"btn\") observeEvent(input$btn, extended_task$invoke(x = input$n, y = input$delay)) output$plot <- renderPlot(hist(extended_task$result())) } app <- shinyApp(ui = ui, server = server) # run app using 2 local daemons with(daemons(2), runApp(app)) input_task_button(\"btn\", \"Plot uniform distribution\") extended_task <- ExtendedTask$new( function(...) mirai({Sys.sleep(y); runif(x)}, ...) ) |> bind_task_button(\"btn\") observeEvent(input$btn, extended_task$invoke(x = input$n, y = input$delay)) output$plot <- renderPlot(hist(extended_task$result()))"},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"shiny-extendedtask-example-generative-art","dir":"Articles","previous_headings":"Shiny Integration","what":"Shiny ExtendedTask Example: Generative Art","title":"mirai - Shiny Integration","text":"following app produces pretty spiral patterns. user can add multiple plots, making use Shiny modules, different calculation time. plots generated asynchronously, easy see practical limitations number daemons set. example, updating 4 plots, 3 daemons, 4th plot start generated one plots finished. example builds original code Joe Cheng, Daniel Woodie William Landau. uses environment() instead ... alternative equivalent way passing variables present calling environment mirai. key components using ExtendedTask example : UI, use bslib::input_task_button(). button disabled computation prevent additional clicks. server, create ExtendedTask object calling ExtendedTask$new() anonymous function passing named arguments mirai(), bind button created (1). passed mirai use environment(). server, create observer input button, invokes ExtendedTask, supplying arguments anonymous function . server, create render function output, consumes result ExtendedTask.","code":"library(shiny) library(mirai) library(bslib) library(ggplot2) library(aRtsy) # function definitions run_task <- function(calc_time) { Sys.sleep(calc_time) list( colors = aRtsy::colorPalette(name = \"random\", n = 3), angle = runif(n = 1, min = - 2 * pi, max = 2 * pi), size = 1, p = 1 ) } plot_result <- function(result) { do.call(what = canvas_phyllotaxis, args = result) } # modules for individual plots plotUI <- function(id, calc_time) { ns <- NS(id) card( strong(paste0(\"Plot (calc time = \", calc_time, \" secs)\")), input_task_button(ns(\"resample\"), \"Resample\"), plotOutput(ns(\"plot\"), height=\"400px\", width=\"400px\") ) } plotServer <- function(id, calc_time) { force(id) force(calc_time) moduleServer( id, function(input, output, session) { extended_task <- ExtendedTask$new( function(time, run) mirai(run(time), environment()) ) |> bind_task_button(\"resample\") observeEvent(input$resample, extended_task$invoke(calc_time, run_task)) output$plot <- renderPlot(plot_result(extended_task$result())) } ) } # ui and server ui <- page_sidebar(fillable = FALSE, sidebar = sidebar( numericInput(\"calc_time\", \"Calculation time (secs)\", 5), actionButton(\"add\", \"Add\", class=\"btn-primary\"), ), layout_column_wrap(id = \"results\", width = \"400px\", fillable = FALSE) ) server <- function(input, output, session) { observeEvent(input$add, { id <- nanonext::random(4) insertUI(\"#results\", where = \"beforeEnd\", ui = plotUI(id, input$calc_time)) plotServer(id, input$calc_time) }) } app <- shinyApp(ui, server) # run app using 3 local daemons with(daemons(3), runApp(app)) input_task_button(ns(\"resample\"), \"Resample\") extended_task <- ExtendedTask$new( function(time, run) mirai(run(time), environment()) ) |> bind_task_button(\"resample\") observeEvent(input$resample, extended_task$invoke(calc_time, run_task)) output$plot <- renderPlot(plot_result(extended_task$result()))"},{"path":"https://shikokuchuo.net/mirai/articles/shiny.html","id":"advanced-promises-example-coin-flips","dir":"Articles","previous_headings":"Shiny Integration","what":"Advanced Promises Example: Coin Flips","title":"mirai - Shiny Integration","text":"example demonstrates integrate mirai_map() operation Shiny app. specifying ‘.promise’ argument, registers promise action mapped operation. can used update reactive values otherwise interact Shiny app. adaptation original example provided Landau use crew Shiny. Please see https://wlandau.github.io/crew/articles/shiny.html.","code":"library(shiny) library(mirai) flip_coin <- function(...) { Sys.sleep(0.1) rbinom(n = 1, size = 1, prob = 0.501) } ui <- fluidPage( div(\"Is the coin fair?\"), actionButton(\"task\", \"Flip 1000 coins\"), textOutput(\"status\"), textOutput(\"outcomes\") ) server <- function(input, output, session) { # Keep running totals of heads, tails, and task errors flips <- reactiveValues(heads = 0, tails = 0, flips = 0) # Button to submit a batch of coin flips observeEvent(input$task, { flips$flips <- flips$flips + 1000 m <- mirai_map(1:1000, flip_coin, .promise = \\(x) if (x) flips$heads <- flips$heads + 1 else flips$tails <- flips$tails + 1) }) # Print time and task status output$status <- renderText({ input$task invalidateLater(millis = 1000) time <- format(Sys.time(), \"%H:%M:%S\") sprintf(\"%s %s flips submitted\", time, flips$flips) }) # Print number of heads and tails output$outcomes <- renderText( sprintf(\"%s heads %s tails\", flips$heads, flips$tails) ) } app <- shinyApp(ui = ui, server = server) # run app using 8 local non-dispatcher daemons (tasks are the same length) with(daemons(8, dispatcher = \"none\"), { # pre-load flip_coin function on all daemons for efficiency everywhere({}, flip_coin = flip_coin) runApp(app) })"},{"path":"https://shikokuchuo.net/mirai/articles/torch.html","id":"torch-integration","dir":"Articles","previous_headings":"","what":"Torch Integration","title":"mirai - Torch Integration","text":"Custom serialization functions may registered handle external pointer type reference objects. allows tensors torch package used seamlessly ‘mirai’ computations.","code":""},{"path":"https://shikokuchuo.net/mirai/articles/torch.html","id":"setup-steps","dir":"Articles","previous_headings":"Torch Integration","what":"Setup Steps","title":"mirai - Torch Integration","text":"Set dameons. Create serialization configuration, specifying ‘class’ ‘torch_tensor’ ‘vec’ TRUE. Use everywhere(), supplying configuration ‘.serial’ argument, (optionally) making torch package available daemons convenience.","code":"library(mirai) library(torch) daemons(1) #> [1] 1 cfg <- serial_config( class = \"torch_tensor\", sfunc = torch:::torch_serialize, ufunc = torch::torch_load, vec = TRUE ) everywhere(library(torch), .serial = cfg)"},{"path":"https://shikokuchuo.net/mirai/articles/torch.html","id":"example-usage","dir":"Articles","previous_headings":"Torch Integration","what":"Example Usage","title":"mirai - Torch Integration","text":"example creates convolutional neural network using torch::nn_module(). set model parameters also specified. model specification parameters passed initialized within ‘mirai’. returned model object containing many tensor elements. usual model parameters passed optimiser. can also initialized within ‘mirai’ process. , tensors complex objects containing tensors passed seamlessly host daemon processes, way R object. custom serialization mirai leverages R’s native ‘refhook’ mechanism allow completely transparent usage. Designed fast efficient, data copies minimised ‘official’ serialization methods torch package used directly.","code":"model <- nn_module( initialize = function(in_size, out_size) { self$conv1 <- nn_conv2d(in_size, out_size, 5) self$conv2 <- nn_conv2d(in_size, out_size, 5) }, forward = function(x) { x <- self$conv1(x) x <- nnf_relu(x) x <- self$conv2(x) x <- nnf_relu(x) x } ) params <- list(in_size = 1, out_size = 20) m <- mirai(do.call(model, params), model = model, params = params) m[] #> An `nn_module` containing 1,040 parameters. #> #> ── Modules ──────────────────────────────────────────────────────────────────────────────────────────────────────────── #> • conv1: #520 parameters #> • conv2: #520 parameters m$data$parameters$conv1.weight #> torch_tensor #> (1,1,.,.) = #> 0.1090 0.0691 -0.0591 -0.0461 -0.0532 #> -0.0153 0.1228 0.1182 -0.0665 0.0505 #> -0.0303 0.0163 -0.0647 -0.1798 -0.0441 #> -0.0588 0.0846 0.0857 0.0327 0.1972 #> 0.1442 0.0955 -0.1682 -0.0183 0.0960 #> #> (2,1,.,.) = #> -0.0650 0.0633 0.0921 -0.0372 0.1392 #> -0.0493 -0.0742 -0.1552 -0.0638 -0.0708 #> -0.0113 -0.1114 -0.0013 -0.0260 -0.0838 #> -0.0292 0.0165 -0.1340 -0.0556 0.0925 #> -0.0394 0.0905 0.1140 -0.1017 0.0363 #> #> (3,1,.,.) = #> -0.1762 0.0509 -0.1795 0.1617 -0.1282 #> 0.1735 -0.1951 -0.1044 0.1623 -0.1978 #> 0.1982 -0.1127 -0.1133 -0.0947 -0.0160 #> 0.1135 0.0198 -0.0254 0.0281 -0.0520 #> 0.0794 -0.0114 -0.1520 0.0267 -0.1980 #> #> (4,1,.,.) = #> 0.0412 0.0476 0.1843 -0.0444 0.0996 #> 0.0813 0.1186 0.1490 -0.1211 -0.0169 #> 0.0239 0.0793 -0.0484 0.0478 0.1343 #> 0.1461 0.1949 0.0382 0.0634 0.1292 #> 0.0958 0.0273 0.1933 0.0691 0.0905 #> #> (5,1,.,.) = #> -0.0724 0.1578 -0.0650 -0.0328 0.1338 #> ... [the output was truncated (use n=-1 to disable)] #> [ CPUFloatType{20,1,5,5} ][ requires_grad = TRUE ] optim <- mirai(optim_rmsprop(params = params), params = m$data$parameters) optim[] #> #> Inherits from: #> Public: #> add_param_group: function (param_group) #> clone: function (deep = FALSE) #> defaults: list #> initialize: function (params, lr = 0.01, alpha = 0.99, eps = 1e-08, weight_decay = 0, #> load_state_dict: function (state_dict, ..., .refer_to_state_dict = FALSE) #> param_groups: list #> state: State, R6 #> state_dict: function () #> step: function (closure = NULL) #> zero_grad: function () #> Private: #> step_helper: function (closure, loop_fun) daemons(0) #> [1] 0"},{"path":"https://shikokuchuo.net/mirai/authors.html","id":null,"dir":"","previous_headings":"","what":"Authors","title":"Authors and Citation","text":"Charlie Gao. Author, maintainer. Joe Cheng. Contributor. Hibiki AI Limited. Copyright holder.","code":""},{"path":"https://shikokuchuo.net/mirai/authors.html","id":"citation","dir":"","previous_headings":"","what":"Citation","title":"Authors and Citation","text":"Gao C (2024). mirai: Minimalist Async Evaluation Framework R. R package version 1.3.0.9000, https://doi.org/10.5281/zenodo.7912722.","code":"@Manual{, title = {mirai: Minimalist Async Evaluation Framework for R}, author = {Charlie Gao}, year = {2024}, note = {R package version 1.3.0.9000}, url = {https://doi.org/10.5281/zenodo.7912722}, }"},{"path":[]},{"path":"https://shikokuchuo.net/mirai/index.html","id":"ミライ","dir":"","previous_headings":"","what":"ミライ","title":"Minimalist Async Evaluation Framework for R","text":"みらい 未来 Minimalist Async Evaluation Framework R Designed simplicity, ‘mirai’ evaluates R expression asynchronously parallel process, locally distributed network, result automatically available upon completion. Modern networking concurrency built nanonext NNG (Nanomsg Next Gen) ensure reliable efficient scheduling, fast inter-process communications TCP/IP secured TLS. Advantages include inherently queued thus handling many tasks available processes, storage file system, support otherwise non-exportable reference objects, event-driven promises implementation, built-asynchronous parallel map.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"quick-start","dir":"","previous_headings":"","what":"Quick Start","title":"Minimalist Async Evaluation Framework for R","text":"Use mirai() evaluate expression asynchronously separate, clean R process. following mimics expensive calculation eventually returns vector random values. mirai expression evaluated another process hence must self-contained, referring variables already exist . , variables n mean passed part mirai() call. ‘mirai’ object returned immediately - creating mirai never blocks session. Whilst async operation ongoing, attempting access mirai’s data yields ‘unresolved’ logical NA. check whether mirai remains unresolved (yet complete): wait collect return value, use mirai’s [] method: mirai represents async operation, never necessary wait . code can continue run. completes, return value automatically becomes available $data.","code":"library(mirai) m <- mirai({Sys.sleep(n); rnorm(n, mean)}, n = 5L, mean = 7) m #> < mirai [] > m$data #> 'unresolved' logi NA unresolved(m) #> [1] TRUE m[] #> [1] 6.558945 6.774610 6.776656 7.419766 6.670370 while (unresolved(m)) { # do work here that does not depend on 'm' } m #> < mirai [$data] > m$data #> [1] 6.558945 6.774610 6.776656 7.419766 6.670370"},{"path":"https://shikokuchuo.net/mirai/index.html","id":"daemons","dir":"","previous_headings":"Quick Start","what":"Daemons","title":"Minimalist Async Evaluation Framework for R","text":"Daemons persistent background processes receiving mirai requests, created easily : Daemons may also deployed remotely distributed computing launchers can start daemons across network via (tunnelled) SSH cluster resource manager. Secure TLS connections can used remote daemon connections, zero configuration required.","code":"daemons(4) #> [1] 4"},{"path":"https://shikokuchuo.net/mirai/index.html","id":"async-parallel-map","dir":"","previous_headings":"Quick Start","what":"Async Parallel Map","title":"Minimalist Async Evaluation Framework for R","text":"mirai_map() maps function list vector, element processed separate parallel process. also performs multiple map rows dataframe matrix. ‘mirai_map’ object returned immediately. code can continue run point. value may retrieved time using [] method return list, just like purrr::map() base::lapply(). [] method also provides options flatmap, early stopping /progress indicators. errors returned ‘errorValues’, facilitating recovery partial failure. advantages alternative map implementations.","code":"df <- data.frame( fruit = c(\"melon\", \"grapes\", \"coconut\"), price = c(3L, 5L, 2L) ) m <- mirai_map(df, sprintf, .args = list(fmt = \"%s: $%d\")) m #> < mirai map [3/3] > m[.flat] #> [1] \"melon: $3\" \"grapes: $5\" \"coconut: $2\""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"design-concepts","dir":"","previous_headings":"","what":"Design Concepts","title":"Minimalist Async Evaluation Framework for R","text":"mirai designed ground provide production-grade experience. 900x responsive common alternatives [1] Built low-latency applications real time inference Shiny apps Consistent behaviour reliance global options variables mirai call evaluated explicitly transparent predictable results Launch millions tasks simultaneously thousands connections Proven track record handling heavy-duty workloads life sciences industry   mirai パッケージを試してみたところ、かなり速くて驚きました","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"integrations","dir":"","previous_headings":"","what":"Integrations","title":"Minimalist Async Evaluation Framework for R","text":"following core integrations documented, usage examples linked vignettes:   Provides alternative communications backend R, implementing new parallel cluster type, feature request R-Core R Project Sprint 2023. ‘miraiCluster’ may also used foreach via doParallel.   Implements next generation completely event-driven, non-polling promises. ‘mirai’ may used interchageably ‘promises’, including promise pipe %...>%.   Asynchronous parallel / distributed backend, supporting next level responsiveness scalability within Shiny, native support ExtendedTask.   Asynchronous parallel / distributed backend scaling Plumber applications production.   Allows queries using Apache Arrow format handled seamlessly ADBC database connections hosted background processes.   Allows Torch tensors complex objects models optimizers used seamlessly across parallel processes.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"powering-crew-and-targets-high-performance-computing","dir":"","previous_headings":"","what":"Powering Crew and Targets High Performance Computing","title":"Minimalist Async Evaluation Framework for R","text":"  Targets, Make-like pipeline tool statistics data science, integrated adopted crew default high-performance computing backend.   Crew distributed worker-launcher extending mirai different distributed computing platforms, traditional clusters cloud services.   crew.cluster enables mirai-based workflows traditional high-performance computing clusters using LFS, PBS/TORQUE, SGE Slurm.   crew.aws.batch extends mirai cloud computing using AWS Batch.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"thanks","dir":"","previous_headings":"","what":"Thanks","title":"Minimalist Async Evaluation Framework for R","text":"like thank particular: Landau instrumental shaping development package, initiating original request persistent daemons, orchestrating robustness testing high performance computing requirements crew targets. Joe Cheng integrating promises method work seamlessly within Shiny, prototyping event-driven promises. Luke Tierney R Core, discussion L’Ecuyer-CMRG streams ensure statistical independence parallel processing, making possible mirai first ‘alternative communications backend R’. Henrik Bengtsson valuable insights leading interface accepting broader usage patterns. Daniel Falbel discussion around efficient solution serialization transmission torch tensors. Kirill Müller discussion using ‘daemons’ host Arrow database connections.   funding work TLS implementation nanonext, used provide secure connections mirai.","code":""},{"path":"https://shikokuchuo.net/mirai/index.html","id":"installation","dir":"","previous_headings":"","what":"Installation","title":"Minimalist Async Evaluation Framework for R","text":"Install latest release CRAN: current development version available R-universe:","code":"install.packages(\"mirai\") install.packages(\"mirai\", repos = \"https://shikokuchuo.r-universe.dev\")"},{"path":"https://shikokuchuo.net/mirai/index.html","id":"links--references","dir":"","previous_headings":"","what":"Links & References","title":"Minimalist Async Evaluation Framework for R","text":"◈ mirai R package: https://shikokuchuo.net/mirai/ ◈ nanonext R package: https://shikokuchuo.net/nanonext/ mirai listed CRAN High Performance Computing Task View: https://cran.r-project.org/view=HighPerformanceComputing – Please note project released Contributor Code Conduct. participating project agree abide terms.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"Make Mirai Promise — as.promise.mirai","title":"Make Mirai Promise — as.promise.mirai","text":"Creates ‘promise’ ‘mirai’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Make Mirai Promise — as.promise.mirai","text":"","code":"# S3 method for class 'mirai' as.promise(x)"},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Make Mirai Promise — as.promise.mirai","text":"x object class ‘mirai’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Make Mirai Promise — as.promise.mirai","text":"‘promise’ object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Make Mirai Promise — as.promise.mirai","text":"function S3 method generic .promise class ‘mirai’. Requires promises package. Allows ‘mirai’ used promise pipe %...>%, schedules function run upon resolution ‘mirai’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/as.promise.mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Make Mirai Promise — as.promise.mirai","text":"","code":"if (interactive() && requireNamespace(\"promises\", quietly = TRUE)) { library(promises) p <- as.promise(mirai(\"example\")) print(p) is.promise(p) p2 <- mirai(\"completed\") %...>% identity() p2$then(cat) is.promise(p2) }"},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Call Value) — call_mirai","title":"mirai (Call Value) — call_mirai","text":"call_mirai waits ‘mirai’ resolve still progress, storing value $data, returns ‘mirai’ object. call_mirai_ variant call_mirai allows user interrupts, suitable interactive use.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Call Value) — call_mirai","text":"","code":"call_mirai(x) call_mirai_(x)"},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Call Value) — call_mirai","text":"x ‘mirai’ object, list ‘mirai’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Call Value) — call_mirai","text":"passed object (invisibly). ‘mirai’, retrieved value stored $data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Call Value) — call_mirai","text":"functions accept list ‘mirai’ objects, returned mirai_map well individual ‘mirai’. wait asynchronous operation(s) complete still progress (blocking). x[] may also used wait return value mirai x, equivalent call_mirai_(x)$data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"alternatively","dir":"Reference","previous_headings":"","what":"Alternatively","title":"mirai (Call Value) — call_mirai","text":"value ‘mirai’ may accessed time $data, yet resolve, ‘unresolved’ logical NA returned instead. Using unresolved ‘mirai’ returns TRUE yet resolve FALSE otherwise. suitable use control flow statements .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"mirai (Call Value) — call_mirai","text":"error occurs evaluation, error message returned character string class ‘miraiError’ ‘errorValue’ (stack trace available $stack.trace error object). is_mirai_error may used test . daemon crashes terminates unexpectedly evaluation, ‘errorValue’ 19 (Connection reset) returned (using dispatcher using dispatcher retry = FALSE). Otherwise, using dispatcher retry = TRUE, mirai remain unresolved automatically re-tried next daemon connect particular instance. cancel task instead, use saisei(force = TRUE) (see saisei). is_error_value tests error conditions including ‘mirai’ errors, interrupts, timeouts.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/call_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Call Value) — call_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # using call_mirai() df1 <- data.frame(a = 1, b = 2) df2 <- data.frame(a = 3, b = 1) m <- mirai(as.matrix(rbind(df1, df2)), df1 = df1, df2 = df2, .timeout = 1000) call_mirai(m)$data # using unresolved() m <- mirai( { res <- rnorm(n) res / rev(res) }, n = 1e6 ) while (unresolved(m)) { cat(\"unresolved\\n\") Sys.sleep(0.1) } str(m$data) }"},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Collect Value) — collect_mirai","title":"mirai (Collect Value) — collect_mirai","text":"collect_mirai waits ‘mirai’ resolve still progress, returns value directly. efifcient version equivalent call_mirai(x)$data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Collect Value) — collect_mirai","text":"","code":"collect_mirai(x)"},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Collect Value) — collect_mirai","text":"x ‘mirai’ object, list ‘mirai’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Collect Value) — collect_mirai","text":"object (return value ‘mirai’), list objects (length ‘x’, preserving names).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Collect Value) — collect_mirai","text":"function wait asynchronous operation(s) complete still progress (blocking), interruptible. x[] may used wait return value mirai x, user-interruptible counterpart collect_mirai(x).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"alternatively","dir":"Reference","previous_headings":"","what":"Alternatively","title":"mirai (Collect Value) — collect_mirai","text":"value ‘mirai’ may accessed time $data, yet resolve, ‘unresolved’ logical NA returned instead. Using unresolved ‘mirai’ returns TRUE yet resolve FALSE otherwise. suitable use control flow statements .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"mirai (Collect Value) — collect_mirai","text":"error occurs evaluation, error message returned character string class ‘miraiError’ ‘errorValue’ (stack trace available $stack.trace error object). is_mirai_error may used test . daemon crashes terminates unexpectedly evaluation, ‘errorValue’ 19 (Connection reset) returned (using dispatcher using dispatcher retry = FALSE). Otherwise, using dispatcher retry = TRUE, mirai remain unresolved automatically re-tried next daemon connect particular instance. cancel task instead, use saisei(force = TRUE) (see saisei). is_error_value tests error conditions including ‘mirai’ errors, interrupts, timeouts.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/collect_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Collect Value) — collect_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # using collect_mirai() df1 <- data.frame(a = 1, b = 2) df2 <- data.frame(a = 3, b = 1) m <- mirai(as.matrix(rbind(df1, df2)), df1 = df1, df2 = df2, .timeout = 1000) collect_mirai(m) # using x[] m[] }"},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":null,"dir":"Reference","previous_headings":"","what":"Daemon Instance — daemon","title":"Daemon Instance — daemon","text":"Starts execution daemon receive mirai requests. Awaits data, evaluates expression environment containing supplied data, returns value host caller. Daemon settings may controlled daemons function need invoked directly, unless deploying manually remote resources.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Daemon Instance — daemon","text":"","code":"daemon( url, asyncdial = FALSE, autoexit = TRUE, cleanup = TRUE, output = FALSE, maxtasks = Inf, idletime = Inf, walltime = Inf, timerstart = 0L, ..., tls = NULL, rs = NULL )"},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Daemon Instance — daemon","text":"url character host dispatcher URL dial , including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. asyncdial [default FALSE] whether perform dials asynchronously. default FALSE error connection immediately possible (instance daemons yet called host, specified port open etc.). Specifying TRUE continues retrying (indefinitely) immediately successful, resilient can mask potential connection issues. autoexit [default TRUE] logical value, whether daemon exit automatically socket connection ends. signal tools package, tools::SIGINT, equivalent integer value supplied, signal additionally raised exit (see 'Persistence' section ). cleanup [default TRUE] logical value, whether perform cleanup global environment restore loaded packages options initial state evaluation. granular control, also accepts integer value (see ‘Cleanup Options’ section ). output [default FALSE] logical value, output generated stdout / stderr TRUE, else discard FALSE. Specify TRUE ‘...’ argument daemons launch_local provide redirection output host process (applicable local daemons). maxtasks [default Inf] maximum number tasks execute (task limit) exiting. idletime [default Inf] maximum idle time, since completion last task (milliseconds) exiting. walltime [default Inf] soft walltime, minimum amount real time taken (milliseconds) exiting. timerstart [default 0L] number completed tasks start timer ‘idletime’ ‘walltime’. 0L implies timers started upon launch. ... reserved currently used. tls [default NULL] required secure TLS connections 'tls+tcp://' 'wss://'. Either character path file containing X.509 certificate(s) PEM format, comprising certificate authority certificate chain starting TLS certificate ending CA certificate, length 2 character vector comprising [] certificate authority certificate chain [ii] empty string ''. rs [default NULL] initial value .Random.seed. set automatically using L'Ecuyer-CMRG RNG streams generated host process independently supplied.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Daemon Instance — daemon","text":"Invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Daemon Instance — daemon","text":"network topology daemons dial host dispatcher, listens ‘url’ address. way, network resources may added removed dynamically host dispatcher automatically distributes tasks available daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"persistence","dir":"Reference","previous_headings":"","what":"Persistence","title":"Daemon Instance — daemon","text":"‘autoexit’ argument governs persistence settings daemon. default TRUE ensures exit cleanly socket connection ended. Instead TRUE, supplying signal tools package, tools::SIGINT, equivalent integer value, sets signal raised socket connection ends. instance, supplying SIGINT allows potentially immediate exit interrupting ongoing evaluation rather letting complete. Setting FALSE allows daemon persist indefinitely even longer socket connection. allows host session end new session connect URL daemon dialled . Daemons must terminated daemons(NULL) case, sends explicit exit instructions connected daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemon.html","id":"cleanup-options","dir":"Reference","previous_headings":"","what":"Cleanup Options","title":"Daemon Instance — daemon","text":"‘cleanup’ argument also accepts integer value, operates additive bitmask: perform cleanup global environment (1L), reset loaded packages initial state (2L), restore options initial state (4L), perform garbage collection (8L). example, perform cleanup global environment garbage collection, specify 9L (1L + 8L). default argument value TRUE performs actions apart garbage collection equivalent value 7L. Caution: reset options loaded packages packages set options load.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":null,"dir":"Reference","previous_headings":"","what":"Daemons (Set Persistent Processes) — daemons","title":"Daemons (Set Persistent Processes) — daemons","text":"Set ‘daemons’ persistent background processes receive mirai requests. Specify ‘n’ create daemons local machine. Specify ‘url’ receiving connections remote daemons (distributed computing across network). Specify ‘remote’ optionally launch remote daemons via remote configuration. default, dispatcher ensures optimal scheduling.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Daemons (Set Persistent Processes) — daemons","text":"","code":"daemons( n, url = NULL, remote = NULL, dispatcher = c(\"process\", \"thread\", \"none\"), ..., force = TRUE, seed = NULL, tls = NULL, pass = NULL, .compute = \"default\" )"},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Daemons (Set Persistent Processes) — daemons","text":"n integer number daemons set. url [default NULL] specified, character URL vector URLs host remote daemons dial , including port accepting incoming connections (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. Specify URL starting 'tls+tcp://' 'wss://' use secure TLS connections. Auxiliary function host_url may used construct valid host URL. remote [default NULL] required launching remote daemons, configuration generated remote_config ssh_config. dispatcher [default 'process'] character value, one ‘process’, ‘thread’ ‘none’. Whether deploy dispatcher another process, thread . Dispatcher extension ensures optimal scheduling, although always required (details see Dispatcher section ). Note option ‘thread’ new currently considered experimental. ... (optional) additional arguments passed dispatcher using dispatcher /daemon launching daemons. include ‘retry’ ‘token’ dispatcher ‘asyncdial’, ‘autoexit’, ‘cleanup’, ‘output’, ‘maxtasks’, ‘idletime’, ‘walltime’ ‘timerstart’ daemon. force [default TRUE] logical value whether always reset daemons apply new settings compute profile, even already set. FALSE, applying new settings requires daemons explicitly reset first using daemons(0). seed [default NULL] (optional) supply random seed (single value, interpreted integer). used inititalise L'Ecuyer-CMRG RNG streams sent daemon. Note reproducible results can expected dispatcher = 'none', unpredictable timing task completions otherwise influence tasks sent daemon. Even dispatcher = 'none', reproducibility guaranteed order tasks sent deterministic. tls [default NULL] (optional secure TLS connections) supplied, zero-configuration single-use keys certificates automatically generated. supplied, either character path file containing PEM-encoded TLS certificate associated private key (may contain additional certificates leading validation chain, TLS certificate first), length 2 character vector comprising [] TLS certificate (optionally certificate chain) [ii] associated private key. pass [default NULL] (required private key supplied ‘tls’ encrypted password) security, provided function returns value, rather directly. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Daemons (Set Persistent Processes) — daemons","text":"using dispatcher, integer number daemons set, else integer number daemons launched locally (zero using remote launcher).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Daemons (Set Persistent Processes) — daemons","text":"Use daemons(0) reset daemon connections: connected daemons /dispatchers exit automatically. mirai reverts default behaviour creating new background process request. unresolved ‘mirai’ return ‘errorValue’ 19 (Connection reset) reset. Calling daemons revised (even ) settings compute profile resets daemons applying new settings force = TRUE. host session ends, connected dispatcher daemon processes automatically exit soon connections dropped (unless daemons started autoexit = FALSE). daemon processing task, exit soon task complete. reset persistent daemons started autoexit = FALSE, use daemons(NULL) instead, also sends exit instructions connected daemons prior resetting. historical reasons, daemons() arguments returns value status.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"local-daemons","dir":"Reference","previous_headings":"","what":"Local Daemons","title":"Daemons (Set Persistent Processes) — daemons","text":"Daemons provide potentially efficient solution asynchronous operations new processes longer need created ad hoc basis. Supply argument ‘n’ set number daemons. New background daemon processes automatically created local machine connecting back host process, either directly via dispatcher.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"dispatcher","dir":"Reference","previous_headings":"","what":"Dispatcher","title":"Daemons (Set Persistent Processes) — daemons","text":"default dispatcher = \"process\" launches background process running dispatcher. Dispatcher connects daemons behalf host ensures optimal FIFO scheduling tasks. Specifying dispatcher = \"thread\" runs dispatcher logic new thread, faster efficient alternative using separate process. new feature considered experimental. Specifying dispatcher = \"none\", uses default behaviour without additional dispatcher logic. case daemons connect directly host tasks distributed round-robin fashion. Optimal scheduling guaranteed duration tasks known priori, hence tasks can queued one daemon daemons remain idle. However, provides resource-light approach, suited working similar-length tasks, concurrent tasks typically exceed available daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"distributed-computing","dir":"Reference","previous_headings":"","what":"Distributed Computing","title":"Daemons (Set Persistent Processes) — daemons","text":"Specifying ‘url’ allows tasks distributed across network. character string ‘tcp://10.75.32.70:5555’ daemon processes connect . Switching URL scheme ‘tls+tcp://’ ‘wss://’ automatically upgrades connection use TLS. auxiliary function host_url may used automatically construct valid host URL based computer's hostname. Specify ‘remote’ call remote_config ssh_config launch daemons remote machines. Otherwise, launch_remote may used generate shell commands deploy daemons manually remote resources. IPv6 addresses also supported must enclosed square brackets [ ] avoid confusion final colon separating port. example, port 5555 IPv6 loopback address ::1 specified ‘tcp://[::1]:5555’. Specifying wildcard value zero port number e.g. ‘tcp://[::1]:0’ ‘ws://[::1]:0’ automatically assign free ephemeral port. Use status inspect actual assigned port time. Dispatcher using dispatcher, recommended use websocket URL rather TCP, requires one port connect daemons: websocket URL supports path port number, can made unique daemon. Specifying single host URL ‘ws://10.75.32.70:5555’ n = 6 automatically append sequence path, listening URLs ‘ws://10.75.32.70:5555/1’ ‘ws://110.75.32.70:5555/6’. Alternatively, specify vector URLs listen arbitrary port numbers / paths. case optional supply ‘n’ can inferred length vector supplied. Individual daemons dial host URLs. one daemon can dialled URL given time. Dispatcher automatically adjusts number daemons actually connected. Hence possible dynamically scale number daemons required, subject maximum number initially specified. Alternatively, supplying single TCP URL listen block URLs ports starting supplied port number incrementing one ‘n’ specified e.g. host URL ‘tcp://10.75.32.70:5555’ n = 6 listens contiguous block ports 5555 5560. Without Dispatcher TCP URL may used case host listens one address, utilising single port. network topology daemons (started daemon) indeed dispatchers (started dispatcher) dial host URL. ‘n’ required case, disregarded supplied, network resources may added removed time. host automatically distributes tasks connected daemons dispatchers round-robin fashion.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"compute-profiles","dir":"Reference","previous_headings":"","what":"Compute Profiles","title":"Daemons (Set Persistent Processes) — daemons","text":"default, ‘default’ compute profile used. Providing character value ‘.compute’ creates new compute profile name specified. compute profile retains daemons settings, may operated independently . usage examples follow: local / remote daemons may set host URL specifying ‘.compute’ ‘remote’, creates new compute profile. Subsequent mirai calls may sent local computation specifying ‘.compute’ argument, remote computation connected daemons specifying ‘.compute’ argument ‘remote’. cpu / gpu tasks may require access different types daemon, GPUs. case, daemons() may called set host URLs CPU-daemons GPUs, specifying ‘.compute’ argument ‘cpu’ ‘gpu’ respectively. supplying ‘.compute’ argument subsequent mirai calls, tasks may sent either ‘cpu’ ‘gpu’ daemons appropriate. Note: actions resetting daemons via daemons(0) carried desired ‘.compute’ argument specified.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/daemons.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Daemons (Set Persistent Processes) — daemons","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # Create 2 local daemons (using dispatcher) daemons(2) status() # Reset to zero daemons(0) # Create 2 local daemons (not using dispatcher) daemons(2, dispatcher = \"none\") status() # Reset to zero daemons(0) # 2 remote daemons via dispatcher using WebSockets daemons(2, url = host_url(ws = TRUE)) status() # Reset to zero daemons(0) # Set host URL for remote daemons to dial into daemons(url = host_url(), dispatcher = \"none\") status() # Reset to zero daemons(0) # Use with() to evaluate with daemons for the duration of the expression with( daemons(2), { m1 <- mirai(Sys.getpid()) m2 <- mirai(Sys.getpid()) cat(call_mirai(m1)$data, call_mirai(m2)$data, \"\\n\") } ) } if (FALSE) { # \\dontrun{ # Launch 2 daemons on remotes 'nodeone' and 'nodetwo' using SSH # connecting back directly to the host URL over a TLS connection: daemons(url = host_url(tls = TRUE), remote = ssh_config(c('ssh://nodeone', 'ssh://nodetwo')), dispatcher = \"none\") # Launch 4 daemons on the remote machine 10.75.32.90 using SSH tunnelling # over port 5555 ('url' hostname must be 'localhost' or '127.0.0.1'): daemons(n = 4, url = 'ws://localhost:5555', remote = ssh_config('ssh://10.75.32.90', tunnel = TRUE)) } # }"},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":null,"dir":"Reference","previous_headings":"","what":"Dispatcher — dispatcher","title":"Dispatcher — dispatcher","text":"Dispatches tasks host daemons processing, using FIFO scheduling, queuing tasks required. Daemon / dispatcher settings may controlled daemons function need invoked directly.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Dispatcher — dispatcher","text":"","code":"dispatcher( host, url = NULL, n = NULL, ..., retry = FALSE, token = FALSE, tls = NULL, pass = NULL, rs = NULL, monitor = NULL )"},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Dispatcher — dispatcher","text":"host character host URL dial (tasks sent ), including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. url (optional) character URL vector URLs dispatcher listen , including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path'. Specify 'tls+tcp://' 'wss://' use secure TLS connections. Tasks sent daemons dialled URLs. supplied, ‘n’ local inter-process URLs assigned automatically. n (optional) specified, integer number daemons listen . Otherwise ‘n’ inferred number URLs supplied ‘url’. single URL supplied ‘n’ > 1, ‘n’ unique URLs automatically assigned daemons dial . ... (optional) additional arguments passed daemon. include ‘asyncdial’, ‘autoexit’, ‘cleanup’, ‘maxtasks’, ‘idletime’, ‘walltime’ ‘timerstart’. retry [default FALSE] logical value, whether automatically retry tasks daemon crashes terminates unexpectedly next daemon instance connect. TRUE, mirai remain unresolved status show ‘online’ 0 ‘assigned’ > ‘complete’. cancel task case, use saisei(force = TRUE). FALSE, tasks returned ‘errorValue’ 19 (Connection reset). token [default FALSE] TRUE, appends unique 24-character token URL path dispatcher listens (applicable TCP URLs accept path). tls [default NULL] (required secure TLS connections) either character path file containing PEM-encoded TLS certificate associated private key (may contain additional certificates leading validation chain, TLS certificate first), length 2 character vector comprising [] TLS certificate (optionally certificate chain) [ii] associated private key. pass [default NULL] (required private key supplied ‘tls’ encrypted password) security, provided function returns value, rather directly. rs [default NULL] initial value .Random.seed. set automatically using L'Ecuyer-CMRG RNG streams generated host process independently supplied. monitor (package internal use ) set parameter.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Dispatcher — dispatcher","text":"Invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dispatcher.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Dispatcher — dispatcher","text":"network topology dispatcher acts gateway host daemons, ensuring tasks received host dispatched FIFO basis processing. Tasks queued dispatcher ensure tasks sent daemons can begin immediate execution task.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai Map Options — .flat","title":"mirai Map Options — .flat","text":"Expressions insert [] method ‘mirai_map’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai Map Options — .flat","text":"","code":".flat .progress .progress_cli .stop"},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":"format","dir":"Reference","previous_headings":"","what":"Format","title":"mirai Map Options — .flat","text":"object class bytecode length 1. object class bytecode length 1. object class bytecode length 1. object class bytecode length 1.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/dot-flat.html","id":"results","dir":"Reference","previous_headings":"","what":"Results","title":"mirai Map Options — .flat","text":"x[] collects results ‘mirai_map’ x returns list. wait asynchronous operations complete still progress, blocking user-interruptible. x[.flat] collects flattens map results vector, checking type avoid coercion. Note: errors ‘errorValue’ returned results differing type. x[.progress] collects map results whilst showing simple text progress indicator parts completed total. x[.progress_cli] collects map results whilst showing progress bar cli package, available, completion percentage ETA. x[.stop] collects map results applying early stopping, stops first failure cancels remaining operations. Note: operations already progress continue completion, although results collected. options may combined manner : x[.stop, .progress] applies early stopping together progress indicator.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":null,"dir":"Reference","previous_headings":"","what":"Evaluate Everywhere — everywhere","title":"Evaluate Everywhere — everywhere","text":"Evaluate expression ‘everywhere’ connected daemons specified compute profile. Designed performing setup operations across daemons loading packages, exporting common data, registering custom serialization functions. Resultant changes global environment, loaded packages options persisted regardless daemon's ‘cleanup’ setting.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Evaluate Everywhere — everywhere","text":"","code":"everywhere(.expr, ..., .args = list(), .serial = NULL, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Evaluate Everywhere — everywhere","text":".expr expression evaluate asynchronously (arbitrary length, wrapped { } necessary), else pre-constructed language object. ... (optional) either named arguments (name = value pairs) specifying objects referenced, defined, ‘.expr’, environment containing objects. See ‘evaluation’ section . .args (optional) either named list specifying objects referenced, defined, ‘.expr’, environment containing objects. objects remain local evaluation environment opposed supplied ‘...’ - see ‘evaluation’ section . .serial [default NULL] (optional) configuration created serial_config register serialization unserialization functions normally non-exportable reference objects, Arrow Tables torch tensors. Updating new configuration replaces existing registered functions. remove configuration, provide empty list. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Evaluate Everywhere — everywhere","text":"Invisible NULL. error specified compute profile found, .e. yet set .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"evaluation","dir":"Reference","previous_headings":"","what":"Evaluation","title":"Evaluate Everywhere — everywhere","text":"expression ‘.expr’ evaluated separate R process clean environment (global environment), consisting objects list environment supplied ‘.args’, named objects passed ‘...’ (environment one supplied) assigned global environment process. evaluation occur global environment, supply objects ‘...’ rather ‘.args’. stricter scoping, use ‘.args’, limits, example, variables explicitly passed arguments functions found. evaluation occurs clean environment, undefined objects must supplied though ‘...’ /‘.args’, including self-defined functions. Functions package use namespaced calls mirai::mirai(), else package loaded beforehand part ‘.expr’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/everywhere.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Evaluate Everywhere — everywhere","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1) # export common data by a super-assignment expression: everywhere(y <<- 3) # '...' variables are assigned to the global environment # '.expr' may be specified as an empty {} in such cases: everywhere({}, a = 1, b = 2) m <- mirai(a + b - y == 0L) call_mirai(m)$data daemons(0) # loading a package on all daemons and also # registering custom serialization functions: cfg <- serial_config(\"cls_name\", function(x) serialize(x, NULL), unserialize) daemons(1, dispatcher = \"none\") everywhere(library(parallel), .serial = cfg) m <- mirai(\"package:parallel\" %in% search()) call_mirai(m)$data daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":null,"dir":"Reference","previous_headings":"","what":"URL Constructors — host_url","title":"URL Constructors — host_url","text":"host_url constructs valid host URL (daemons may connect) based computer's hostname. may supplied directly ‘url’ argument daemons. local_url constructs random URL suitable local daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"URL Constructors — host_url","text":"","code":"host_url(ws = FALSE, tls = FALSE, port = 0) local_url()"},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"URL Constructors — host_url","text":"ws [default FALSE] logical value whether use WebSockets 'ws://' else TCP 'tcp://' scheme. tls [default FALSE] logical value whether use TLS case scheme used either 'wss://' 'tls+tcp://' accordingly. port [default 0] numeric port use. open connections network addresses daemons connecting . ‘0’ wildcard value automatically assigns free ephemeral port.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"URL Constructors — host_url","text":"character string comprising valid URL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"URL Constructors — host_url","text":"host_url relies using host name computer rather IP address typically works local networks, although always guaranteed. unsuccessful, substitute IPv4 IPv6 address place hostname. local_url generates random URL platform's default inter-process communications transport: abstract Unix domain sockets Linux, Unix domain sockets MacOS, Solaris POSIX platforms, named pipes Windows.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/host_url.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"URL Constructors — host_url","text":"","code":"host_url() #> [1] \"tcp://fv-az661-991:0\" host_url(ws = TRUE) #> [1] \"ws://fv-az661-991:0\" host_url(tls = TRUE) #> [1] \"tls+tcp://fv-az661-991:0\" host_url(ws = TRUE, tls = TRUE, port = 5555) #> [1] \"wss://fv-az661-991:5555\" local_url() #> [1] \"abstract://b303001ab17932607922a093\""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"Is mirai / mirai_map — is_mirai","title":"Is mirai / mirai_map — is_mirai","text":"object ‘mirai’ ‘mirai_map’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Is mirai / mirai_map — is_mirai","text":"","code":"is_mirai(x) is_mirai_map(x)"},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Is mirai / mirai_map — is_mirai","text":"x object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Is mirai / mirai_map — is_mirai","text":"Logical TRUE ‘x’ class ‘mirai’ ‘mirai_map’ respectively, FALSE otherwise.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Is mirai / mirai_map — is_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1, dispatcher = \"none\") df <- data.frame() m <- mirai(as.matrix(df), df = df) is_mirai(m) is_mirai(df) mp <- mirai_map(1:3, runif) is_mirai_map(mp) is_mirai_map(mp[]) daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":null,"dir":"Reference","previous_headings":"","what":"Error Validators — is_mirai_error","title":"Error Validators — is_mirai_error","text":"Validator functions error value types created mirai.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Error Validators — is_mirai_error","text":"","code":"is_mirai_error(x) is_mirai_interrupt(x) is_error_value(x)"},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Error Validators — is_mirai_error","text":"x object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Error Validators — is_mirai_error","text":"Logical value TRUE FALSE.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Error Validators — is_mirai_error","text":"object ‘miraiError’. execution ‘mirai’ process fails, error message returned character string class ‘miraiError’ ‘errorValue’. stack trace available $stack.trace error object. object ‘miraiInterrupt’. ongoing ‘mirai’ sent user interrupt, resolve empty character string classed ‘miraiInterrupt’ ‘errorValue’. object ‘errorValue’, ‘mirai’ timeout, ‘miraiError’ ‘miraiInterrupt’. catch-condition includes returned error values.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/is_mirai_error.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Error Validators — is_mirai_error","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions m <- mirai(stop()) call_mirai(m) is_mirai_error(m$data) is_mirai_interrupt(m$data) is_error_value(m$data) m$data$stack.trace m2 <- mirai(Sys.sleep(1L), .timeout = 100) call_mirai(m2) is_mirai_error(m2$data) is_mirai_interrupt(m2$data) is_error_value(m2$data) }"},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":null,"dir":"Reference","previous_headings":"","what":"Launch Daemon — launch_local","title":"Launch Daemon — launch_local","text":"launch_local spawns new background Rscript process calling daemon specified arguments. launch_remote returns shell command deploying daemons character vector. configuration generated remote_config ssh_config supplied used launch daemon remote machine.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Launch Daemon — launch_local","text":"","code":"launch_local(url, ..., tls = NULL, .compute = \"default\") launch_remote( url, remote = remote_config(), ..., tls = NULL, .compute = \"default\" )"},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Launch Daemon — launch_local","text":"url character host URL vector host URLs, including port connect (optionally websockets, path), e.g. 'tcp://hostname:5555' 'ws://10.75.32.70:5555/path' integer index value, vector index values, dispatcher URLs, 1L host URL (using dispatcher). launch_remote , ‘miraiCluster’ ‘miraiNode’. ... (optional) additional arguments passed daemon. include ‘autoexit’, ‘cleanup’, ‘output’, ‘maxtasks’, ‘idletime’, ‘walltime’ ‘timerstart’. tls [default NULL] required secure TLS connections tls+tcp wss. Zero-configuration TLS certificates generated daemons automatically passed daemon, without requiring specified . Otherwise, supply either character path file containing X.509 certificate(s) PEM format, comprising certificate authority certificate chain, length 2 character vector comprising [] certificate authority certificate chain [ii] empty character ”. .compute [default 'default'] character value compute profile use (compute profile independent set daemons). remote required launching remote daemons, configuration generated remote_config ssh_config. empty remote_config effect daemon launches returns shell commands deploying manually remote machines.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Launch Daemon — launch_local","text":"launch_local: Invisible NULL. launch_remote: character vector daemon launch commands, classed ‘miraiLaunchCmd’. printed output may copy / pasted directly remote machine.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Launch Daemon — launch_local","text":"functions may used re-launch daemons exited reaching time task limits. Daemons must already set launchers work. generated command contains argument ‘rs’ specifying length 7 L'Ecuyer-CMRG random seed supplied daemon. values different time function called.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/launch_local.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Launch Daemon — launch_local","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(url = host_url(ws = TRUE), dispatcher = \"none\") status() launch_local(status()$daemons, maxtasks = 10L) launch_remote(1L, maxtasks = 10L) Sys.sleep(1) status() daemons(0) daemons(n = 2L, url = host_url(tls = TRUE)) status() launch_local(1:2, idletime = 60000L, timerstart = 1L) launch_remote(1:2, idletime = 60000L, timerstart = 1L) Sys.sleep(1) status() daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":null,"dir":"Reference","previous_headings":"","what":"Make Mirai Cluster — make_cluster","title":"Make Mirai Cluster — make_cluster","text":"make_cluster creates cluster type ‘miraiCluster’, may used cluster object function parallel base package clusterApply parLapply. stop_cluster stops cluster created make_cluster.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Make Mirai Cluster — make_cluster","text":"","code":"make_cluster(n, url = NULL, remote = NULL, ...) stop_cluster(cl)"},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Make Mirai Cluster — make_cluster","text":"n integer number nodes (automatically launched local machine unless ‘url’ supplied). url [default NULL] (specify remote nodes) character URL host remote nodes dial , including port accepting incoming connections, e.g. 'tcp://10.75.37.40:5555'. Specify URL scheme ‘tls+tcp://’ use secure TLS connections. remote [default NULL] (specify launch remote nodes) remote launch configuration generated remote_config ssh_config. supplied, nodes may deployed manually remote resources. ... additional arguments passed onto daemons. cl ‘miraiCluster’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Make Mirai Cluster — make_cluster","text":"make_cluster: object class ‘miraiCluster’ ‘cluster’. ‘miraiCluster’ automatically assigned ID ‘n’ nodes class ‘miraiNode’. ‘url’ supplied ‘remote’, shell commands deployment nodes remote resources printed console. stop_cluster: invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"note","dir":"Reference","previous_headings":"","what":"Note","title":"Make Mirai Cluster — make_cluster","text":"default behaviour clusters created function designed map closely possible clusters created parallel package. However, ‘...’ arguments passed onto daemons additional customisation desired, although resultant behaviour may always supported.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"remote-nodes","dir":"Reference","previous_headings":"","what":"Remote Nodes","title":"Make Mirai Cluster — make_cluster","text":"Specify ‘url’ ‘n’ set host connection remote nodes dial . ‘n’ defaults one specified. Also specify ‘remote’ launch nodes using configuration generated remote_config ssh_config. case, number nodes inferred configuration provided ‘n’ disregarded. ‘remote’ supplied, shell commands deploying nodes manually remote resources automatically printed console. launch_remote may called time ‘miraiCluster’ return shell commands deployment nodes, ‘miraiNode’ return command single node.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"status","dir":"Reference","previous_headings":"","what":"Status","title":"Make Mirai Cluster — make_cluster","text":"Call status ‘miraiCluster’ check number currently active connections well host URL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"Make Mirai Cluster — make_cluster","text":"Errors thrown ‘parallel’ mechanism one nodes failed (quit unexpectedly). resulting ‘errorValue’ returned 19 (Connection reset). types error, e.g. evaluation, result usual ‘miraiError’ returned.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/make_cluster.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Make Mirai Cluster — make_cluster","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions cl <- make_cluster(2) cl cl[[1L]] Sys.sleep(0.5) status(cl) stop_cluster(cl) }"},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"Designed simplicity, 'mirai' evaluates R expression asynchronously parallel process, locally distributed network, result automatically available upon completion. Modern networking concurrency built 'nanonext' 'NNG' (Nanomsg Next Gen) ensure reliable efficient scheduling, fast inter-process communications TCP/IP secured TLS. Advantages include inherently queued thus handling many tasks available processes, storage file system, support otherwise non-exportable reference objects, event-driven promises implementation, built-asynchronous parallel map.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":"notes","dir":"Reference","previous_headings":"","what":"Notes","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"local mirai requests, default transport inter-process communications platform-dependent: abstract Unix domain sockets Linux, Unix domain sockets MacOS, Solaris POSIX platforms, named pipes Windows. may overriden, desired, specifying 'url' daemons interface launching daemons using launch_local.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":"reference-manual","dir":"Reference","previous_headings":"","what":"Reference Manual","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"vignette(\"mirai\", package = \"mirai\")","code":""},{"path":[]},{"path":"https://shikokuchuo.net/mirai/reference/mirai-package.html","id":"author","dir":"Reference","previous_headings":"","what":"Author","title":"mirai: Minimalist Async Evaluation Framework for R — mirai-package","text":"Charlie Gao charlie.gao@shikokuchuo.net (ORCID)","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Evaluate Async) — mirai","title":"mirai (Evaluate Async) — mirai","text":"Evaluate expression asynchronously new background R process persistent daemon (local remote). function return immediately ‘mirai’, resolve evaluated result complete.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Evaluate Async) — mirai","text":"","code":"mirai(.expr, ..., .args = list(), .timeout = NULL, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Evaluate Async) — mirai","text":".expr expression evaluate asynchronously (arbitrary length, wrapped { } necessary), else pre-constructed language object. ... (optional) either named arguments (name = value pairs) specifying objects referenced, defined, ‘.expr’, environment containing objects. See ‘evaluation’ section . .args (optional) either named list specifying objects referenced, defined, ‘.expr’, environment containing objects. objects remain local evaluation environment opposed supplied ‘...’ - see ‘evaluation’ section . .timeout [default NULL] timeout, integer value milliseconds. mirai resolve ‘errorValue’ 5 (timed ) evaluation exceeds limit. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Evaluate Async) — mirai","text":"‘mirai’ object.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Evaluate Async) — mirai","text":"function return ‘mirai’ object immediately. value mirai may accessed time $data, yet resolve, ‘unresolved’ logical NA returned instead. unresolved may used mirai, returning TRUE ‘mirai’ yet resolve FALSE otherwise. suitable use control flow statements . Alternatively, call (wait ) result, use call_mirai returned ‘mirai’. block result returned. Specify ‘.compute’ send mirai using specific compute profile (previously created daemons), otherwise leave ‘default’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"evaluation","dir":"Reference","previous_headings":"","what":"Evaluation","title":"mirai (Evaluate Async) — mirai","text":"expression ‘.expr’ evaluated separate R process clean environment (global environment), consisting objects list environment supplied ‘.args’, named objects passed ‘...’ (environment one supplied) assigned global environment process. evaluation occur global environment, supply objects ‘...’ rather ‘.args’. stricter scoping, use ‘.args’, limits, example, variables explicitly passed arguments functions found. evaluation occurs clean environment, undefined objects must supplied though ‘...’ /‘.args’, including self-defined functions. Functions package use namespaced calls mirai::mirai(), else package loaded beforehand part ‘.expr’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"errors","dir":"Reference","previous_headings":"","what":"Errors","title":"mirai (Evaluate Async) — mirai","text":"error occurs evaluation, error message returned character string class ‘miraiError’ ‘errorValue’ (stack trace available $stack.trace error object). is_mirai_error may used test . daemon crashes terminates unexpectedly evaluation, ‘errorValue’ 19 (Connection reset) returned (using dispatcher using dispatcher retry = FALSE). Otherwise, using dispatcher retry = TRUE, mirai remain unresolved automatically re-tried next daemon connect particular instance. cancel task instead, use saisei(force = TRUE) (see saisei). is_error_value tests error conditions including ‘mirai’ errors, interrupts, timeouts.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Evaluate Async) — mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions # specifying objects via '...' n <- 3 m <- mirai(x + y + 2, x = 2, y = n) m m$data Sys.sleep(0.2) m$data # passing the calling environment to '...' df1 <- data.frame(a = 1, b = 2) df2 <- data.frame(a = 3, b = 1) m <- mirai(as.matrix(rbind(df1, df2)), environment(), .timeout = 1000) call_mirai(m)$data # using unresolved() m <- mirai( { res <- rnorm(n) res / rev(res) }, n = 1e6 ) while (unresolved(m)) { cat(\"unresolved\\n\") Sys.sleep(0.1) } str(m$data) # evaluating scripts using source() in '.expr' n <- 10L file <- tempfile() cat(\"r <- rnorm(n)\", file = file) m <- mirai({source(file); r}, file = file, n = n) call_mirai(m)[[\"data\"]] unlink(file) # use source(local = TRUE) when passing in local variables via '.args' n <- 10L file <- tempfile() cat(\"r <- rnorm(n)\", file = file) m <- mirai({source(file, local = TRUE); r}, .args = list(file = file, n = n)) call_mirai(m)[[\"data\"]] unlink(file) # passing a language object to '.expr' and a named list to '.args' expr <- quote(a + b + 2) args <- list(a = 2, b = 3) m <- mirai(.expr = expr, .args = args) call_mirai(m)$data }"},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai Map — mirai_map","title":"mirai Map — mirai_map","text":"Asynchronous parallel map function list vector using mirai, optional promises integration. Performs multiple map rows dataframe matrix.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai Map — mirai_map","text":"","code":"mirai_map(.x, .f, ..., .args = list(), .promise = NULL, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai Map — mirai_map","text":".x list atomic vector. Also accepts matrix dataframe, case multiple map performed rows. .f function applied element .x, row .x case may . ... (optional) named arguments (name = value pairs) specifying objects referenced, defined, .f. .args (optional) constant arguments .f, provided list. .promise (optional) supplied, registers promise mirai. Either function, supplied ‘onFulfilled’ argument promises::() list 2 functions, supplied respectively ‘onFulfilled’ ‘onRejected’ promises::(). Using argument requires promises package. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai Map — mirai_map","text":"‘mirai_map’ (list ‘mirai’ objects).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai Map — mirai_map","text":"Sends application function .f element .x (row .x) computation separate mirai call. simple transparent behaviour designed make full use mirai scheduling minimise overall execution time. Facilitates recovery partial failure returning ‘miraiError’ / ‘errorValue’ case may , thus allowing failures re-run. Note: requires daemons previously set. , one local daemon set function proceeds.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"results","dir":"Reference","previous_headings":"","what":"Results","title":"mirai Map — mirai_map","text":"x[] collects results ‘mirai_map’ x returns list. wait asynchronous operations complete still progress, blocking user-interruptible. x[.flat] collects flattens map results vector, checking type avoid coercion. Note: errors ‘errorValue’ returned results differing type. x[.progress] collects map results whilst showing simple text progress indicator parts completed total. x[.progress_cli] collects map results whilst showing progress bar cli package, available, completion percentage ETA. x[.stop] collects map results applying early stopping, stops first failure cancels remaining operations. Note: operations already progress continue completion, although results collected. options may combined manner : x[.stop, .progress] applies early stopping together progress indicator.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"multiple-map","dir":"Reference","previous_headings":"","what":"Multiple Map","title":"mirai Map — mirai_map","text":"Multiple map performed automatically rows object ‘dim’ attributes matrix dataframe. often desired behaviour cases. map columns instead, first wrap dataframe .list, transpose matrix using t.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/mirai_map.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai Map — mirai_map","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(4, dispatcher = \"none\") # map with constant args specified via '.args' mirai_map(1:3, rnorm, .args = list(mean = 20, sd = 2))[] # flatmap with function definition passed via '...' mirai_map(1:3, function(x) func(1L, x, x + 1L), func = stats::runif)[.flat] # sum rows of a dataframe (df <- data.frame(a = 1:3, b = c(4, 3, 2))) mirai_map(df, sum)[.flat] # sum rows of a matrix (mat <- matrix(1:4, nrow = 2L)) mirai_map(mat, sum)[.flat] # map over rows of a dataframe df <- data.frame(a = c(\"Aa\", \"Bb\"), b = c(1L, 4L)) mirai_map(df, function(...) sprintf(\"%s: %d\", ...))[.flat] # indexed map over a vector v <- c(\"egg\", \"got\", \"ten\", \"nap\", \"pie\") mirai_map( data.frame(1:length(v), v), sprintf, .args = list(fmt = \"%d_%s\") )[.flat] # return a 'mirai_map' object, check for resolution, collect later mp <- mirai_map( c(a = 2, b = 3, c = 4), function(x, y) do(x, as.logical(x %% y)), do = nanonext::random, .args = list(y = 2) ) unresolved(mp) mp mp[] unresolved(mp) # progress indicator counts up from 0 to 4 seconds res <- mirai_map(1:4, Sys.sleep)[.progress] daemons(0) # generates warning as daemons not set # stops early when second element returns an error tryCatch( mirai_map(list(1, \"a\", 3), sum)[.stop], error = identity ) # promises example that outputs the results, including errors, to the console if (requireNamespace(\"promises\", quietly = TRUE)) { daemons(1, dispatcher = \"none\") ml <- mirai_map( 1:30, function(x) {Sys.sleep(0.1); if (x == 30) stop(x) else x}, .promise = list( function(x) cat(paste(x, \"\")), function(x) { cat(conditionMessage(x), \"\\n\"); daemons(0) } ) ) } }"},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":null,"dir":"Reference","previous_headings":"","what":"Next >> Developer Interface — nextstream","title":"Next >> Developer Interface — nextstream","text":"nextstream retrieves currently stored L'Ecuyer-CMRG RNG stream specified compute profile advances next stream. nextget retrieves specified item specified compute profile.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Next >> Developer Interface — nextstream","text":"","code":"nextstream(.compute = \"default\") nextget(x, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Next >> Developer Interface — nextstream","text":".compute [default 'default'] character value compute profile use (compute profile independent set daemons). x character value item retrieve. One ‘n’ (number dispatcher daemons), ‘pid’ (dispatcher process ID), ‘urls’ (URLs dispatcher listening ) ‘tls’ (stored client TLS configuration use daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Next >> Developer Interface — nextstream","text":"nextstream: length 7 integer vector, given .Random.seed L'Ecuyer-CMRG RNG use (may passed directly ‘rs’ argument daemon), else NULL stream yet created. nextget: requested item, else NULL present.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Next >> Developer Interface — nextstream","text":"functions exported use packages extending mirai alternative launchers daemon processes. nextstream: function called return value required. function also side effect automatically advancing stream stored within compute profile. ensures next recursive stream returned function called .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/nextstream.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Next >> Developer Interface — nextstream","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1L) nextstream() nextstream() nextget(\"pid\") nextget(\"urls\") daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":null,"dir":"Reference","previous_headings":"","what":"Register Mirai Cluster — register_cluster","title":"Register Mirai Cluster — register_cluster","text":"Registers 'miraiCluster' parallel package cluster type 'MIRAI' (optionally) makes default cluster type.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Register Mirai Cluster — register_cluster","text":"","code":"register_cluster(default = TRUE)"},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Register Mirai Cluster — register_cluster","text":"default [default TRUE] logical value whether also register 'miraiCluster' default cluster type.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Register Mirai Cluster — register_cluster","text":"Invisible NULL","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"note","dir":"Reference","previous_headings":"","what":"Note","title":"Register Mirai Cluster — register_cluster","text":"underlying implementation parallel package function relies iis currently available R-devel (4.5).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/register_cluster.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Register Mirai Cluster — register_cluster","text":"","code":"tryCatch(mirai::register_cluster(), error = identity) #> "},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":null,"dir":"Reference","previous_headings":"","what":"Generic and SSH Remote Launch Configuration — remote_config","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"remote_config provides flexible generic framework generating shell commands deploy daemons remotely. ssh_config generates remote configuration launching daemons SSH, option SSH tunnelling.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"","code":"remote_config( command = NULL, args = c(\"\", \".\"), rscript = \"Rscript\", quote = FALSE ) ssh_config( remotes, tunnel = FALSE, timeout = 10, command = \"ssh\", rscript = \"Rscript\", host )"},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"command command used effect daemon launch remote machine character string (e.g. 'ssh'). Defaults ‘ssh’ ssh_config, although may substituted full path specific SSH application. default NULL remote_config effect launches, causes launch_remote return shell commands manual deployment remote machines. args (optional) arguments passed ‘command’, character vector must include \".\" element, substituted daemon launch command. Alternatively, list character vectors effect multiple launches (one list element). rscript (optional) name / path Rscript executable remote machine. default assumes ‘Rscript’ executable search path. Prepend full path necessary. launching Windows, ‘Rscript’ replaced ‘Rscript.exe’. quote [default FALSE] logical value whether quote daemon launch command (required Slurm ‘srun’ example, required ‘ssh’ Slurm ‘sbatch’). remotes character URL vector URLs SSH , using 'ssh://' scheme including port open SSH connections (defaults 22 specified), e.g. 'ssh://10.75.32.90:22' 'ssh://nodename'. tunnel [default FALSE] logical value whether use SSH reverse tunnelling. TRUE, tunnel created ports local remote machines. See ‘SSH Tunnelling’ section correctly specify required settings. timeout [default 10] maximum time allowed connection setup seconds. host (optional) applicable reverse tunnelling. specified creating standalone configuration object. calling function directly argument daemons, required can inferred ‘url’ supplied (see ‘SSH Tunnelling’ section ).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"list required format supplied ‘remote’ argument launch_remote, daemons, make_cluster.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ssh-direct-connections","dir":"Reference","previous_headings":"","what":"SSH Direct Connections","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"simplest use SSH execute daemon launch command remote machine, dial back host / dispatcher URL. assumed SSH key-based authentication already place. relevant port host must also open inbound connections remote machine.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ssh-tunnelling","dir":"Reference","previous_headings":"","what":"SSH Tunnelling","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"Use SSH tunnelling provides convenient way launch remote daemons without requiring remote machine able access host. Often firewall configurations security policies may prevent opening port accept outside connections. cases SSH tunnelling offers solution creating tunnel initial SSH connection made. simplicity, SSH tunnelling implementation uses port side host daemon. SSH key-based authentication must also already place. Tunnelling requires hostname ‘host’ argument (‘url’ argument daemons called directly context) either ‘127.0.0.1’ ‘localhost’. tunnel created 127.0.0.1:port equivalently localhost:port machine. host listens port machine remotes dial port respective machines.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/remote_config.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Generic and SSH Remote Launch Configuration — remote_config","text":"","code":"# Slurm srun example remote_config( command = \"srun\", args = c(\"--mem 512\", \"-n 1\", \".\"), rscript = file.path(R.home(\"bin\"), \"Rscript\") ) #> $command #> [1] \"srun\" #> #> $args #> [1] \"--mem 512\" \"-n 1\" \".\" #> #> $rscript #> [1] \"/opt/R/4.4.1/lib/R/bin/Rscript\" #> #> $quote #> [1] FALSE #> # Slurm sbatch requires 'quote = TRUE' remote_config( command = \"sbatch\", args = c(\"--mem 512\", \"-n 1\", \"--wrap\", \".\"), rscript = file.path(R.home(\"bin\"), \"Rscript\"), quote = TRUE ) #> $command #> [1] \"sbatch\" #> #> $args #> [1] \"--mem 512\" \"-n 1\" \"--wrap\" \".\" #> #> $rscript #> [1] \"/opt/R/4.4.1/lib/R/bin/Rscript\" #> #> $quote #> [1] TRUE #> # SSH also requires 'quote = TRUE' remote_config( command = \"/usr/bin/ssh\", args = c(\"-fTp 22 10.75.32.90\", \".\"), quote = TRUE ) #> $command #> [1] \"/usr/bin/ssh\" #> #> $args #> [1] \"-fTp 22 10.75.32.90\" \".\" #> #> $rscript #> [1] \"Rscript\" #> #> $quote #> [1] TRUE #> # can be used to start local dameons with special configurations remote_config( command = \"Rscript\", rscript = \"--default-packages=NULL --vanilla\" ) #> $command #> [1] \"Rscript\" #> #> $args #> [1] \"\" \".\" #> #> $rscript #> [1] \"--default-packages=NULL --vanilla\" #> #> $quote #> [1] FALSE #> # simple SSH example ssh_config( remotes = c(\"ssh://10.75.32.90:222\", \"ssh://nodename\"), timeout = 5 ) #> $command #> [1] \"ssh\" #> #> $args #> $args[[1]] #> [1] \"-o ConnectTimeout=5 -fTp 222\" \"10.75.32.90\" #> [3] \".\" #> #> $args[[2]] #> [1] \"-o ConnectTimeout=5 -fTp 22\" \"nodename\" #> [3] \".\" #> #> #> $rscript #> [1] \"Rscript\" #> #> $quote #> [1] TRUE #> # SSH tunnelling example ssh_config( remotes = c(\"ssh://10.75.32.90:222\", \"ssh://nodename\"), tunnel = TRUE, host = \"tls+tcp://127.0.0.1:5555\" ) #> $command #> [1] \"ssh\" #> #> $args #> $args[[1]] #> [1] \"-R 5555:127.0.0.1:5555\" \"-o ConnectTimeout=10 -fTp 222\" #> [3] \"10.75.32.90\" \".\" #> #> $args[[2]] #> [1] \"-R 5555:127.0.0.1:5555\" \"-o ConnectTimeout=10 -fTp 22\" #> [3] \"nodename\" \".\" #> #> #> $rscript #> [1] \"Rscript\" #> #> $quote #> [1] TRUE #> if (FALSE) { # \\dontrun{ # launch 2 daemons on the remote machines 10.75.32.90 and 10.75.32.91 using # SSH, connecting back directly to the host URL over a TLS connection: daemons( url = host_url(tls = TRUE), remote = ssh_config( remotes = c(\"ssh://10.75.32.90:222\", \"ssh://10.75.32.91:222\"), timeout = 1 ) ) # launch 2 nodes on the remote machine 10.75.32.90 using SSH tunnelling over # port 5555 ('url' hostname must be 'localhost' or '127.0.0.1'): cl <- make_cluster( url = \"tcp://localhost:5555\", remote = ssh_config( remotes = c(\"ssh://10.75.32.90\", \"ssh://10.75.32.90\"), tunnel = TRUE, timeout = 1 ) ) } # }"},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":null,"dir":"Reference","previous_headings":"","what":"Saisei (Regenerate Token) — saisei","title":"Saisei (Regenerate Token) — saisei","text":"using daemons dispatcher, regenerates token URL dispatcher socket listens .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Saisei (Regenerate Token) — saisei","text":"","code":"saisei(i, force = FALSE, .compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Saisei (Regenerate Token) — saisei","text":"integer index number URL regenerate dispatcher. force [default FALSE] logical value whether regenerate URL even existing active connection. .compute [default 'default'] character value compute profile use (compute profile independent set daemons).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Saisei (Regenerate Token) — saisei","text":"regenerated character URL upon success, else NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Saisei (Regenerate Token) — saisei","text":"URL regenerated, listener specified socket closed replaced immediately, hence function successful existing connections socket (.e. ‘online’ status shows 0), unless argument ‘force’ specified TRUE. ‘force’ specified TRUE, socket immediately closed regenerated. happens mirai task still ongoing, returned ‘errorValue’ 7 (Object closed). may used cancel task consistently hangs crashes prevent failing repeatedly new daemons connect.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"timeouts","dir":"Reference","previous_headings":"","what":"Timeouts","title":"Saisei (Regenerate Token) — saisei","text":"Specifying ‘.timeout’ argument mirai ensures mirai always resolves. However, task may completed still ongoing daemon process. situations, dispatcher ensures queued tasks assigned busy process, however overall performance may still degraded remain use. process hangs restarted otherwise, saisei specifying force = TRUE may used cancel task regenerate particular URL new daemon connect .","code":""},{"path":"https://shikokuchuo.net/mirai/reference/saisei.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Saisei (Regenerate Token) — saisei","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions daemons(1L) Sys.sleep(1L) status() saisei(i = 1L, force = TRUE) status() daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":null,"dir":"Reference","previous_headings":"","what":"Create Serialization Configuration — serial_config","title":"Create Serialization Configuration — serial_config","text":"Returns serialization configuration, may set perform custom serialization unserialization normally non-exportable reference objects, allowing used seamlessly different R sessions. feature utilises 'refhook' system R native serialization. set, functions apply mirai requests specific compute profile.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Create Serialization Configuration — serial_config","text":"","code":"serial_config(class, sfunc, ufunc, vec = FALSE)"},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Create Serialization Configuration — serial_config","text":"class character string class object custom serialization functions applied , e.g. ‘ArrowTabular’ ‘torch_tensor’. sfunc function accepts reference object inheriting ‘class’ (list objects) returns raw vector. ufunc function accepts raw vector returns reference object (list objects). vec [default FALSE] whether serialization functions vectorized. FALSE, accept return reference objects individually e.g. arrow::write_to_raw arrow::read_ipc_stream. TRUE, accept return list reference objects, e.g. torch::torch_serialize torch::torch_load.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Create Serialization Configuration — serial_config","text":"list comprising configuration. passed ‘.serial’ argument everywhere.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/serial_config.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Create Serialization Configuration — serial_config","text":"","code":"cfg <- serial_config(\"test_cls\", function(x) serialize(x, NULL), unserialize) cfg #> [[1]] #> [1] \"test_cls\" #> #> [[2]] #> function (x) #> serialize(x, NULL) #> #> #> [[3]] #> function (connection, refhook = NULL) #> { #> if (typeof(connection) != \"raw\" && !is.character(connection) && #> !inherits(connection, \"connection\")) #> stop(\"'connection' must be a connection\") #> .Internal(unserialize(connection, refhook)) #> } #> #> #> #> [[4]] #> [1] FALSE #>"},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":null,"dir":"Reference","previous_headings":"","what":"Status Information — status","title":"Status Information — status","text":"Retrieve status information specified compute profile, comprising current connections daemons status.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Status Information — status","text":"","code":"status(.compute = \"default\")"},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Status Information — status","text":".compute [default 'default'] character compute profile (compute profile set daemons connecting different resources). ‘miraiCluster’ obtain status.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Status Information — status","text":"named list comprising: connections - integer number active connections. Using dispatcher: Always 1L single connection dispatcher, connects daemons turn. daemons - variable type. Using dispatcher: status matrix (see Status Matrix section ), else integer ‘errorValue’ communication dispatcher failed. using dispatcher: character host URL. set: 0L.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"status-matrix","dir":"Reference","previous_headings":"","what":"Status Matrix","title":"Status Information — status","text":"using dispatcher, $daemons comprises integer matrix following columns: - integer index number. online - shows 1 active connection, else 0 daemon yet connect disconnected. instance - increments 1 every time new connection URL. counter designed track new daemon instances connecting previous ones ended (due time-outs etc.). count becomes negative immediately URL regenerated saisei, increments new daemon connects. assigned - shows cumulative number tasks assigned daemon. complete - shows cumulative number tasks completed daemon. dispatcher URLs stored row names matrix.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/status.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Status Information — status","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions status() daemons(n = 2L, url = \"wss://[::1]:0\") status() daemons(0) }"},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":null,"dir":"Reference","previous_headings":"","what":"mirai (Stop) — stop_mirai","title":"mirai (Stop) — stop_mirai","text":"Stops ‘mirai’ still progress, causing resolve immediately ‘errorValue’ 20 (Operation canceled).","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"mirai (Stop) — stop_mirai","text":"","code":"stop_mirai(x)"},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"mirai (Stop) — stop_mirai","text":"x ‘mirai’ object, list ‘mirai’ objects.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"mirai (Stop) — stop_mirai","text":"Invisible NULL.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"mirai (Stop) — stop_mirai","text":"Forces ‘mirai’ resolve immediately. effect ‘mirai’ already resolved. cancellation successful, value $data ‘errorValue’ 20 (Operation canceled). Note case, ‘mirai’ aborted value retrieved - ongoing evaluation daemon process continue completion interrupted.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/stop_mirai.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"mirai (Stop) — stop_mirai","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions m <- mirai(Sys.sleep(n), n = 5) stop_mirai(m) m$data }"},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":null,"dir":"Reference","previous_headings":"","what":"Query if a mirai is Unresolved — unresolved","title":"Query if a mirai is Unresolved — unresolved","text":"Query whether ‘mirai’, ‘mirai’ value list ‘mirai’ remains unresolved. Unlike call_mirai, function wait completion.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"Query if a mirai is Unresolved — unresolved","text":"","code":"unresolved(x)"},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"Query if a mirai is Unresolved — unresolved","text":"x ‘mirai’ object list ‘mirai’ objects, ‘mirai’ value stored $data.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"Query if a mirai is Unresolved — unresolved","text":"Logical TRUE ‘aio’ unresolved ‘mirai’ ‘mirai’ value list contains least one unresolved ‘mirai’, FALSE otherwise.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"Query if a mirai is Unresolved — unresolved","text":"Suitable use control flow statements . Note: querying resolution may cause previously unresolved ‘mirai’ resolve.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/unresolved.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"Query if a mirai is Unresolved — unresolved","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions m <- mirai(Sys.sleep(0.1)) unresolved(m) Sys.sleep(0.3) unresolved(m) }"},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":null,"dir":"Reference","previous_headings":"","what":"With Mirai Daemons — with.miraiDaemons","title":"With Mirai Daemons — with.miraiDaemons","text":"Evaluate expression daemons last duration expression.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"ref-usage","dir":"Reference","previous_headings":"","what":"Usage","title":"With Mirai Daemons — with.miraiDaemons","text":"","code":"# S3 method for class 'miraiDaemons' with(data, expr, ...)"},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"arguments","dir":"Reference","previous_headings":"","what":"Arguments","title":"With Mirai Daemons — with.miraiDaemons","text":"data call daemons. expr expression evaluate. ... used.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"value","dir":"Reference","previous_headings":"","what":"Value","title":"With Mirai Daemons — with.miraiDaemons","text":"return value ‘expr’.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"details","dir":"Reference","previous_headings":"","what":"Details","title":"With Mirai Daemons — with.miraiDaemons","text":"function S3 method generic class 'miraiDaemons'.","code":""},{"path":"https://shikokuchuo.net/mirai/reference/with.miraiDaemons.html","id":"ref-examples","dir":"Reference","previous_headings":"","what":"Examples","title":"With Mirai Daemons — with.miraiDaemons","text":"","code":"if (interactive()) { # Only run examples in interactive R sessions with( daemons(2), { m1 <- mirai(Sys.getpid()) m2 <- mirai(Sys.getpid()) cat(call_mirai(m1)$data, call_mirai(m2)$data, \"\\n\") } ) status() }"},{"path":[]},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"updates-1-3-0-9000","dir":"Changelog","previous_headings":"","what":"Updates","title":"mirai 1.3.0.9000 (development)","text":"serialization() deprecated mirai 1.2.0 now removed.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-130","dir":"Changelog","previous_headings":"","what":"mirai 1.3.0","title":"mirai 1.3.0","text":"CRAN release: 2024-10-09","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"new-features-1-3-0","dir":"Changelog","previous_headings":"","what":"New Features","title":"mirai 1.3.0","text":"daemons(dispatcher = \"thread\") implements threaded dispatcher (experimental), faster efficient alternative running dispatcher separate process. mirai_map() adds [.progress_cli] alternative progress indicator, using cli package show % complete ETA. daemons() gains argument ‘force’ control whether calls reset previous settings compute profile. daemon() gains argument ‘asyncdial’ allow control connection behaviour independently happens daemon exits.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"behavioural-changes-1-3-0","dir":"Changelog","previous_headings":"","what":"Behavioural Changes","title":"mirai 1.3.0","text":"Argument ‘dispatcher’ now takes character options ‘process’, ‘thread’ ‘none’. Previous values TRUE/FALSE continue accepted (thanks @hadley #157). Return value now always integer - either number daemons set using dispatcher, number daemons launched locally (zero using remote launcher). Invalid type ... arguments now dropped instead throwing error. allows ... containing unused arguments easily passed functions. Now performs multiple map rows matrices dataframes (thanks @andrewGhazi, #147). Combining collection options now easier, fashion : x[.stop, .progress]. Collection options now work even mirai search path e.g. mirai::mirai_map(1:4, Sys.sleep)[.progress]. dispatcher() drops argument ‘asyncdial’ rarely useful set . everywhere() now errors specified compute profile yet set , rather fail silently. launch_local() launch_remote() now strictly require daemons set, error otherwise. serial_config() now validates arguments provided returns list. means saved configurations previous package versions must re-generated.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"updates-1-3-0","dir":"Changelog","previous_headings":"","what":"Updates","title":"mirai 1.3.0","text":"Fixes daemons() correctly handle vector URLs passed ‘url’ . Fixes flatmap mirai_map()[.flat] assigning variable ‘typ’ calling environment. Performance enhancements mirai(), mirai_map() promises method. Requires nanonext >= 1.3.0. package shiny new hex logo.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-120","dir":"Changelog","previous_headings":"","what":"mirai 1.2.0","title":"mirai 1.2.0","text":"CRAN release: 2024-08-18 everywhere() adds argument ‘.serial’ accept serialization configurations created serial_config(). allow normally non-exportable reference objects Arrow Tables torch tensors used seamlessly across parallel processes without additional marshalling steps. Configurations apply per compute profile basis. serialization() now deprecated favour usage everywhere(), removed future version. mirai_map() enhanced perform multiple map 2D lists/vectors, allowing advanced patterns mapping rows dataframe matrix. ‘mirai_map’ [] method gains option [.flat] collect flatten results, avoiding coercion. Collecting ‘mirai_map’ longer spuriously introduces empty names none present originally. Faster local daemons(dispatcher = FALSE) make_cluster() using asynchronous launches (thanks @mtmorgan #123). Local dispatcher daemons now synchronize host, non-dispatcher daemons (prevents use connected). Fixes rare cases everywhere() reaching daemons using dispatcher. efficient dispatcher startup loading base package, addition reading startup configurations (thanks @krlmlr). Removes hard dependency stats utils base packages. Requires nanonext >= 1.2.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-111","dir":"Changelog","previous_headings":"","what":"mirai 1.1.1","title":"mirai 1.1.1","text":"CRAN release: 2024-07-01 serialization() function signature return value slightly modified clarity. Successful registration / cancellation messages longer printed console. dispatcher() argument ‘retry’ now defaults FALSE consistency non-dispatcher behaviour. remote_config() gains argument ‘quote’ control whether quote daemon launch command, now works Slurm (thanks @michaelmayer2 #119). Ephemeral daemons now exit soon permissible, eliminating 2s linger period. Requires nanonext >= 1.1.1.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-110","dir":"Changelog","previous_headings":"","what":"mirai 1.1.0","title":"mirai 1.1.0","text":"CRAN release: 2024-06-06 x[] collects results mirai_map x, waiting asynchronous operations complete. x[.progress] collects results whilst showing text progress bar. x[.stop] collects results applying early-stopping, stops first error, aborts remaining -progress operations. Adds ‘mirai’ method x[] efficient equivalent interruptible call_mirai_(x)$data. Adds collect_mirai() efficient equivalent non-interruptible call_mirai(x)$data. unresolved(), call_mirai(), collect_mirai() stop_mirai() now accept list ‘mirai’ returned mirai_map(). Improved mirai print method indicates whether mirai resolved. Calling daemons() new settings compute profile already set now implicitly resets daemons applying new settings instead silently nothing. Argument ‘resilience’ retired daemons() automatic re-tries longer performed non-dispatcher daemons. New argument ‘retry’ dispatcher() governs whether auto-retry dispatcher case. Fixes promises method potential crashes launching improbably short-lived mirai. Fixes bug cause hang crash launching additional non-dispatcher daemons. Requires nanonext >= 1.1.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-100","dir":"Changelog","previous_headings":"","what":"mirai 1.0.0","title":"mirai 1.0.0","text":"CRAN release: 2024-05-03 innovation allows higher responsiveness massive scalability ‘mirai’ promises. (breaking change) longer permits unnamed list supplied ‘.args’. allows environment e.g. environment() supplied ‘.args’ element ‘…’. allows evaluation symbol ‘mirai’ environment, e.g. mirai(x, x = 1). new argument ‘host’ allows specifying localhost URL port create standalone configuration object. order arguments ‘tunnel’ ‘timeout’ reversed. stop_mirai() now resolves ‘errorValue’ 20 (operation canceled) case asynchronous task still ongoing (thanks @jcheng5 #110). Rejected promises now show complete error code message case ‘errorValue’. ‘miraiError’ reverts including trailing line break (prior mirai 0.13.2). Non-dispatcher local daemons now synchronize host cases (prevents use connected). [ method ‘miraiCluster’ longer produces ‘miraiCluster’ object (thanks @HenrikBengtsson #83). Faster startup time parallel package now loaded first used. Requires nanonext >= 1.0.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0132","dir":"Changelog","previous_headings":"","what":"mirai 0.13.2","title":"mirai 0.13.2","text":"CRAN release: 2024-04-11 mirai() everywhere() behaviour changed ‘…’ args now assigned global environment daemon process. Adds () method mirai daemons, allowing example: (daemons(4), {expr}), daemons last duration ‘expr’. Adds register_cluster() registering ‘miraiCluster’ parallel Cluster type (requires R >= 4.4). Adds .promising() method ‘mirai’ promises package. ‘miraiError’ now includes full call stack, may accessed $stack.trace, includes trailing line break consistency ‘.character.error()’. mirai promises now preserve deep stacks ‘miraiError’ occurs within Shiny app (thanks @jcheng5 #104). Simplified registration ‘parallel’ ‘promises’ methods (thanks @jcheng5 #103). Fixes promises error handling Shiny vignette (thanks @jcheng5 #98 #99). Requires R >= 3.6.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0131","dir":"Changelog","previous_headings":"","what":"mirai 0.13.1","title":"mirai 0.13.1","text":"Fixes regression mirai 0.12.1, introduced potential unintentional low level errors emerge querying dispatcher (thanks @dsweber2 reporting downstream {targets}).","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0130","dir":"Changelog","previous_headings":"","what":"mirai 0.13.0","title":"mirai 0.13.0","text":"serialization adds arguments ‘class’ ‘vec’ custom serialisation reference object types. Requires nanonext >= 0.13.3.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0121","dir":"Changelog","previous_headings":"","what":"mirai 0.12.1","title":"mirai 0.12.1","text":"CRAN release: 2024-02-02 Dispatcher initial sync timeout widened 10s allow launching large numbers daemons. Default ssh_config() argument ‘timeout’ widened 10 (seconds). Fixes daemons() specifying ‘output = FALSE’ registering TRUE instead. Fixes use everywhere() specifying ‘.args’ unnamed list ‘.expr’ language object. Ensures compatibility nanonext >= 0.13.0. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0120","dir":"Changelog","previous_headings":"","what":"mirai 0.12.0","title":"mirai 0.12.0","text":"CRAN release: 2024-01-12 minimal print methods ‘mirai’ ‘miraiCluster’. Adds local_url() helper construct random inter-process communications URL local daemons (thanks @noamross #90). daemon() argument ‘autoexit’ now accepts signal value tools::SIGINT order raise upon exit. daemon() now records state initial global environment objects (e.g. created .Rprofile) cleanup purposes (thanks @noamross #91). Slightly optimal .promise() method ‘mirai’. Eliminates potential memory leaks along certain error paths. Requires nanonext >= 0.12.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0113","dir":"Changelog","previous_headings":"","what":"mirai 0.11.3","title":"mirai 0.11.3","text":"CRAN release: 2023-12-07 Implements serialization() registering custom serialization unserialization functions using daemons. Introduces call_mirai_(), user-interruptible version call_mirai() suitable interactive use. ‘.args’ now coerce list object list supplied, rather error. ‘.signal’ argument removed - now ‘mirai’ signal daemons set . everywhere() now returns invisible NULL case specified compute profile set , rather error. mirai() specifying timeout daemons() set - timeout begins immediately rather ephemeral daemon connected - please factor small amount time daemon launch. make_cluster() now prints daemon launch commands ‘url’ specified without ‘remote’ whether interactive. Cluster node failures load balanced operations now rely ‘parallel’ mechanism error longer fail early automatically stop cluster. Fixes regression since 0.11.0 prevented dispatcher exiting timely manner tasks backlogged (thanks @wlandau #86). Improved memory efficiency stability dispatcher. longer loads ‘promises’ package already loaded (makes ‘mirai’ method available via hook function). Requires nanonext >= 0.11.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0112","dir":"Changelog","previous_headings":"","what":"mirai 0.11.2","title":"mirai 0.11.2","text":"CRAN release: 2023-11-15 make_cluster() specifying ‘url’ now succeeds implied ‘n’ one. Fixes mirai() specifying language object name ‘.expr’ R versions 4.0 earlier. Fixes regression 0.11.1 prevented correct random seed set using dispatcher. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0111","dir":"Changelog","previous_headings":"","what":"mirai 0.11.1","title":"mirai 0.11.1","text":"CRAN release: 2023-11-04 Adds ‘mirai’ method ‘.promise()’ {promises} package (available). functionality merged package {mirai.promises}, allowing use promise pipe %...>% ‘mirai’. Parallel clusters (alternative communications backend R) now work existing R versions, longer requiring R >= 4.4. everywhere() evaluates expression ‘everywhere’ connected daemons compute profile. Resulting changes global environment, loaded pacakges options persisted regardless ‘cleanup’ setting (request @krlmlr #80). host_url() implemented helper function automatically construct host URL using computer’s hostname. daemon() adds argument ‘autoexit’, replaces ‘asyncdial’, govern persistence settings daemon. daemon can now survive host session re-connect another one (request @krlmlr #81). daemons(NULL) implemented variant daemons(0) also sends exit signals connected persistent daemons. dispatcher() argument ‘lock’ removed now applied cases prevent one daemon dialling dispatcher URL one time. daemon() argument ‘cleanup’ simplified logical argument, granular control offered existing integer bitmask (thanks @krlmlr #79). Daemons connecting TLS now perform synchronous dials default (documented). Fixes supplying ssh_config() specifying tunnelling ‘remote’ argument daemons(). Fixes print method subset ‘miraiCluster’ (thanks @HenrikBengtsson #83). Removes deprecated deferred evaluation pipe %>>%. Requires nanonext >= 0.10.4.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0110","dir":"Changelog","previous_headings":"","what":"mirai 0.11.0","title":"mirai 0.11.0","text":"CRAN release: 2023-10-06 Fulfils request R Core R Project Sprint 2023, requires R >= 4.4 (currently R-devel). make_cluster() creates ‘miraiCluster’, compatible existing functions taking ‘cluster’ object, example ‘parallel’ ‘doParallel’ / ‘foreach’ packages. status() can now take ‘miraiCluster’ argument query connection status. Simplified interface single ‘remote’ argument taking remote configuration launch daemons. Returned shell commands now custom print method means may directly copy/pasted remote machine. Can now take ‘miraiCluster’ ‘miraiNode’ return shell commands deployment remote nodes. Adds argument ‘remote’ launching remote daemons directly without recourse separate call launch_remote(). Adds argument ‘resilience’ control behaviour, using dispatcher, whether retry failed tasks daemons. remote_config() added generate configurations directly launching remote daemons, can supplied directly ‘remote’ argument. ssh_config() added convenience method generate launch configurations using SSH, including SSH tunnelling. mirai() adds logical argument ‘.signal’ whether signal condition variable within compute profile upon resolution ‘mirai’. daemon() argument ‘exitlinger’ retired daemons now synchronise host/dispatcher exit soon possible (although default ‘exitlinger’ period still applies ephemeral daemons). Optimises scheduling dispatcher: tasks longer assigned daemon exiting due specified time/task-outs. ‘errorValue’ 19 ‘Connection reset’ now returned ‘mirai’ connection either dispatcher ephemeral daemon drops, example crashed, rather remaining unresolved. Invalid type ‘…’ arguments specified daemons() dispatcher() now raise error early rather attempting launch daemons fail. Eliminates potential crash host process querying status() longer connection dispatcher. Reverts trailing line break added end ‘miraiError’ character string. Moves ‘…’ argument daemons(), dispatcher() daemon() clearly delineate core vs peripheral options. Deprecates Deferred Evaluation Pipe %>>% favour recommendation use package mirai.promises performing side effects upon ‘mirai’ resolution. Deprecated use alias server() daemon() retired. Adds ‘reference’ vignette, incorporating information readme. Requires nanonext >= 0.10.2.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-0100","dir":"Changelog","previous_headings":"","what":"mirai 0.10.0","title":"mirai 0.10.0","text":"CRAN release: 2023-09-16 daemons() gains new argument ‘seed’ set random seed generating streams. daemon() dispatcher() gain argument ‘rs’ takes L’Ecuyer-CMRG random seed. New developer functions nextstream() nextget(), opening interfaces packages extend mirai. Runs R session --vanilla flags efficiency, avoiding lengthy startup configurations (thanks @alexpiper). Straight pass without serialization/unserialization allows higher performance lower memory utilisation. Fixes edge cases status() occasionally failing communicate dispatcher. Fixes edge cases ending session unresolved mirai resulting crash rather clean exit. specifying argument ‘force’ TRUE now immediately regenerates socket returns ongoing mirai ‘errorValue’. allows tasks consistently hang crash cancelled rather repeated new daemon connects. argument ‘’ now required longer defaults 1L. daemons status matrix adds column ‘’ ease use functions saisei() launch_local(). ‘instance’ column now always cumulative - regenerating URL saisei() longer resets counter instead turns negative new daemon connects. Improves shell quoting daemon launch commands, making easier deploy manually via launch_remote(). daemons() dispatcher() gain argument ‘pass’ support password-protected private keys supplying TLS credentials (thanks @wlandau #76). Cryptographic errors using dispatcher TLS now reported user (thanks @wlandau #76). Passing filename ‘tls’ argument daemons(), launch_local() launch_remote() now works correctly documented. Extends clarifies documentation surrounding use certificate authority signed TLS certificates. Certain error messages accurate informative. Increases performance lower resource utilisation due updates nanonext 0.10.0. Requires nanonext >= 0.10.0 R >= 3.5.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-091","dir":"Changelog","previous_headings":"","what":"mirai 0.9.1","title":"mirai 0.9.1","text":"CRAN release: 2023-07-19 Zero-configuration experience - simply specify tls+tcp:// wss:// URL daemons(). Single-use keys certificates automatically generated. Alternatively, custom certificates may passed ‘tls’ argument daemons() daemon(), generated via Ceritficate Signing Request (CSR) Certificate Authority (CA). Example using SSH: launch_remote(\"ws://192.168.0.1:5555\", command = \"ssh\", args = c(\"-p 22 192.168.0.2\", .). Documentation updated refer consistently host daemons (rather client server) clarity. daemon() replaces server(), deprecated (although currently retained alias). launch_local() replaces launch_server() now accepts vector argument ‘url’ well numeric values select relevant dispatcher host URL, returning invisible NULL instead integer value. status() now retrieves connections daemons status, replacing call daemons() arguments (deprecated). return value $daemons now always host URL using dispatcher. Redirection stdout stderr local daemons host process now possible (running without dispatcher) specifying output=TRUE daemons() launch_local(). daemon() accepts new ‘output’ argument. saisei() argument validation now happens prior sending request dispatcher rather dispatcher. ‘miraiError’ now includes trailing line break end character vector. Requires nanonext >= 0.9.1, R requirement relaxed back >= 2.12.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-090","dir":"Changelog","previous_headings":"","what":"mirai 0.9.0","title":"mirai 0.9.0","text":"CRAN release: 2023-06-24 mirai 0.9.0 major release focusing stability improvements. Ensures first URL retains format saisei(= 1L) called. Optimal scheduling tasks submitted prior servers coming online. Fixes rare occasions dispatcher running single server instance get stuck task. daemons() status requests rendered robust. Ensures saisei() always returns NULL ‘tcp://’ URLs used support tokens. Daemons status matrix ‘assigned’ ‘complete’ now cumulative statistics, reset upon new instances. Requires nanonext >= 0.9.0 R >= 3.5.0. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-087","dir":"Changelog","previous_headings":"","what":"mirai 0.8.7","title":"mirai 0.8.7","text":"CRAN release: 2023-05-11 server() dispatcher() argument ‘asyncdial’ now FALSE default, causing functions exit connection immediately available. means distributed computing purposes, daemons() called server() launched remote resources, else server(asyncdial = TRUE) allows servers wait connection. launch_server() now parses passed URL correctness attempting launch server, producing error valid.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-084","dir":"Changelog","previous_headings":"","what":"mirai 0.8.4","title":"mirai 0.8.4","text":"CRAN release: 2023-05-09 .() implemented wrap piped expression, ensuring return either ‘unresolvedExpr’ ‘resolvedExpr’. expressions may tested using unresolved() way ‘mirai’. allows general use contexts, including within functions. Improved error messages top level evaluation errors ‘mirai’. Requires nanonext >= 0.8.3. Internal stability performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-083","dir":"Changelog","previous_headings":"","what":"mirai 0.8.3","title":"mirai 0.8.3","text":"CRAN release: 2023-04-17 accepts language expression object passed ‘.expr’ evaluation. accepts list ‘name = value’ pairs passed ‘.args’ well existing ‘…’. objects specified via ‘…’ now take precedence ‘.args’ named object appears. token appending unique token URL dispatcher listens . lock locking sockets prevent one server connecting unique URL. saisei() implemented regenerate token used given dispatcher socket. launch_server() replaces launch() launching local instances, simpler interface directly mapping server(). Automatically-launched local daemons revised use unique tokens URLs. Daemons status matrix headers updated ‘online’, ‘instance’, ‘assigned’, ‘complete’. Fixes potential issue attempting use mirai() timeouts connection server. Requires nanonext >= 0.8.2. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-082","dir":"Changelog","previous_headings":"","what":"mirai 0.8.2","title":"mirai 0.8.2","text":"CRAN release: 2023-04-03 Arguments ‘pollfreqh’ ‘pollfreql’ removed longer applicable. Server dispatcher processes exit automatically connection client dropped. significantly reduces likelihood orphaned processes. launch() exported utility easily re-launching daemons timed , instance. Correct passthrough ... variables daemons() call. Requires nanonext >= 0.8.1. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-081","dir":"Changelog","previous_headings":"","what":"mirai 0.8.1","title":"mirai 0.8.1","text":"CRAN release: 2023-03-17 Fixes issue daemon processes may launch certain setups (affecting binary package builds).","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-080","dir":"Changelog","previous_headings":"","what":"mirai 0.8.0","title":"mirai 0.8.0","text":"CRAN release: 2023-03-15 mirai 0.8.0 major feature release. Special thanks @wlandau suggestions, discussion testing many new capabilities. daemons() can create new profiles connect different resources e.g. servers GPU, accelerators etc. mirai() tasks can sent using specific compute profile. Supplying client URL zero port number :0 automatically assign free ephemeral port, actual port number subsequently reported daemons(). Calling arguments now provides improved view current number connections / daemons (URL, online busy status, tasks assigned completed, instance), replacing previous daemons(\"view\") functionality. dispatcher() implemented new function dispatcher. asyncdial specify server dials client. maxtasks specifying maximum number tasks exiting. idletime specifying idle time, since completion last task exiting. walltime specifying soft walltime exiting. timerstart specifying minimum number task completions starting timers. Invalid URLs provided daemons() server() now error return immediately instead potentially causing hang. eval_mirai() removed alias mirai(). ‘mirai’ processes longer launched Rscript sessions --vanilla argument enable site / user profile environment files read. Requires nanonext >= 0.8.0. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-072","dir":"Changelog","previous_headings":"","what":"mirai 0.7.2","title":"mirai 0.7.2","text":"CRAN release: 2023-01-17 Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-071","dir":"Changelog","previous_headings":"","what":"mirai 0.7.1","title":"mirai 0.7.1","text":"CRAN release: 2022-11-15 Allow user interrupts call_mirai() (regression 0.7.0), now returning ‘miraiInterrupt’. Adds auxiliary function is_mirai_interrupt() test object ‘miraiInterrupt’. Requires nanonext >= 0.7.0: returned ‘errorValues’ e.g. mirai timeouts longer accompanied warnings. Internal performance enhancements.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-070","dir":"Changelog","previous_headings":"","what":"mirai 0.7.0","title":"mirai 0.7.0","text":"CRAN release: 2022-10-19 daemons() now takes ‘n’ ‘.url’ arguments. ‘.url’ optional client URL allowing mirai tasks distributed across network. Compatibility existing interface retained. server function server() exported creating daemon / ephemeral processes network resources. Mirai errors formatted better now print stdout rather stderr. Improvements performance stability requiring nanonext >= 0.6.0. Internal enhancements error handling mirai / daemon process.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-060","dir":"Changelog","previous_headings":"","what":"mirai 0.6.0","title":"mirai 0.6.0","text":"CRAN release: 2022-09-16 Notice: older package versions longer supported ‘nanonext’ >= 0.6.0. Please ensure using latest version ‘mirai’ else refrain upgrading ‘nanonext’. Internal enhancements daemons() %>>% deferred evaluation pipe.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-053","dir":"Changelog","previous_headings":"","what":"mirai 0.5.3","title":"mirai 0.5.3","text":"CRAN release: 2022-08-16 mirai() gains ‘.args’ argument passing list objects already calling environment, allowing example mirai(func(x, y, z), .args = list(x, y, z)) rather specify mirai(func(x, y, z), x = x, y = y, z = z). Errors inside mirai now return error message character string class ‘miraiError’ ‘errorValue’, rather just nul byte. Utility function is_mirai_error() used place is_nul_byte(), longer re-exported. is_error_value() can used test errors, including timeouts ‘.timeout’ argument used. re-exports ‘nanonext’ brought -package better documentation.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-052","dir":"Changelog","previous_headings":"","what":"mirai 0.5.2","title":"mirai 0.5.2","text":"CRAN release: 2022-07-21 Internal optimisations requiring nanonext >= 0.5.2.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-050","dir":"Changelog","previous_headings":"","what":"mirai 0.5.0","title":"mirai 0.5.0","text":"CRAN release: 2022-06-21 Implements %>>% deferred evaluation pipe. Adds ‘.timeout’ argument mirai() ensure mirai always resolves even child process crashes etc.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-041","dir":"Changelog","previous_headings":"","what":"mirai 0.4.1","title":"mirai 0.4.1","text":"CRAN release: 2022-04-21 Exits cleanly daemons explicitly zeroed prior ending R session. Fixes possible hang Windows shutting daemons.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-040","dir":"Changelog","previous_headings":"","what":"mirai 0.4.0","title":"mirai 0.4.0","text":"CRAN release: 2022-04-14 Back pure R implementation thanks enhanced internal design nanonext. Adds auxiliary function is_mirai() test object mirai. Versioning system synchronise nanonext e.g. v0.4.x requires nanonext >= 0.4.0.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-020","dir":"Changelog","previous_headings":"","what":"mirai 0.2.0","title":"mirai 0.2.0","text":"CRAN release: 2022-03-28 value mirai now stored $data optimally align underlying implementation. Package now contains C code (requires compilation), using weak references simpler management resources. Switch abstract sockets Linux.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-011","dir":"Changelog","previous_headings":"","what":"mirai 0.1.1","title":"mirai 0.1.1","text":"CRAN release: 2022-03-15 mirai() added alias eval_mirai(); supports evaluating arbitrary length expressions wrapped {}. mirai now resolves automatically without requiring call_mirai(). Access $value directly ‘unresolved’ logical NA returned async operation yet complete. stop_mirai() added function stop evaluation ongoing async operation. Auxiliary functions is_nul_byte() unresolved() re-exported {nanonext} test evaluation errors resolution ‘mirai’ respectively. New daemons() interface set manage persistent background processes receiving ‘mirai’ requests.","code":""},{"path":"https://shikokuchuo.net/mirai/news/index.html","id":"mirai-010","dir":"Changelog","previous_headings":"","what":"mirai 0.1.0","title":"mirai 0.1.0","text":"CRAN release: 2022-02-16 Initial release.","code":""}]