Multiple pipeline-runners

[AlessandroLas]

Dear all,

As stated in the "LOCAL_INSTALL.md" for the second option of "Annotating and loading VCF callsets" on local installs,
although it is possible to run the pipeline on a single machine, it is recommended to use multiple machines.
I would like to know how to set up a second machine for running the annotation and then load the index on the elasticsearch instance.

Many thanks in advance.
Best,
Alessandro.

[hanars]

Hi Alessandro, running on multiple machines means using a cloud provider or an on-prem server cluster to distribute workloads. There is no way that I know of to network 2 physical machines together to improve performance.

[AlessandroLas]

Thank you Hana.
Is it possible to network 2 (or more) machines together to improve performance of multiple elasticsearch instances?

[hanars]

the seqr team has no expertise on networking and hardware. If you are interested in reconfiguring how elasticsearch is running, you can look into the documentation for the elasticsearch docker image and tweak the parameters in the docker compose file to try and better utilize your available hardware, but that would be something your team would have to do without support from the seqr team