migration from openshift (CSCS) to kubernetes

[xgui3783]

Dear @ylep @JoelChavas ,

The openshift cluster at CSCS will be discontinued at the end of year. I am currently in the process of migrating all of the existing services (including hbp-spatial-backend) to a kubernetes (k8s) cluster (not openshift). (My understanding was that there were difficulties getting openshift to run on the new sites.

As I have developer access on the OKD project, I believe I should be able to handle the migration.

However, some documentation should be added (re: deployment configuration on the new cluster, updated URL etc).

I would like to use this issue to track the migration progress.

Question

In normal operation how much RAM on average/at max would the process need? (The new cluster is more constrained, and we will need to budget the resources)

Progress

• create namespace on k8s
• mirror static data
• add CI & push built image to docker registry (https://docker-registry.ebrains.eu/)
• create deployment yml files + create deployment

[ylep]

In normal operation how much RAM on average/at max would the process need? (The new cluster is more constrained, and we will need to budget the resources)

The process is not so much CPU-bound, but rather memory and maybe I/O could be a bottleneck. The AimsApplyTransform process needs to load the full deformation fields from disk into RAM, which is essentially the size of the .ima files (about 1 GB for BigBrain, < 100 MB for the other templates). There may be some overheads / suboptimal memory usage e.g. data duplication, but I think 2-3 GB of available RAM should be enough.

[xgui3783]

Thanks for the comment.

I am also curious, regarding the .ima files, they are currently manually curated / managed (I manually downloaded the file from the okd deployment, and uploaded to the new k8s cluster), is the file available programmatically (e.g. via http/ftp somewhere?) I can imagine we can make the deployment process a lot more automatic, and better documented.

[ylep]

@xgui3783 the .ima files should in theory be identical to those in the KnowledgeGraph (DOI 10.25493/GA3Y-9V8). However, I just double-checked and it appears that the curated version is not the most up-to-date:

• it is missing the .minf files, which are used by the backend when transforming non-point data;
• it does not contain the latest fixes (see changelog below).

Here is the change log that appears at the top of my latest version of the graph.yaml file:

# 2021-07-29  Yann Leprince  <[email protected]>
#
#     Important fix of an error in the BigBrain transformation; support for
#     transforming images.
#
#     * *.ima.minf: fix the header transformations in the deformation fields,
#     because they will be used by AimsApplyTransform in order to transform
#     images (since the deformation fields are passed to --reference, see
#     https://github.com/HumanBrainProject/hbp-spatial-backend/blob/515ee51fb6c4f4df06bd6bc02e18e4ccfeaaf1bc/hbp_spatial_backend/api_v1.py#L321).
#     The transformations in *_AIMS_TO_template.trm have been copied into the
#     header of each corresponding deformation field.
#
#     * bigbrain_AIMS_TO_template.trm: fix incorrect value of the
#     transformation (the previous version was offset by about 5 millimetres).
#
# 2021-07-28  Yann Leprince  <[email protected]>
#
#     * graph.yaml: for consistency, restore the "inv:" prefix in the
#     transformation from "MNI 152 ICBM 2009c Nonlinear Asymmetric" to its AIMS
#     referential. Note that this does not change anything to the results, as
#     this transformation is its own inverse.
#
# 2019-03-07  Yann Leprince  <[email protected]>
#
#     Initial release of the cross-template transformations.

I probably should have sent the updated version for curation back in 2021... but maybe it is not too late

[ylep]

Here is the patch file that goes from the current KnowledgeGraph version to the latest: https://gist.github.com/ylep/3614931a52b12ced591f49a2ec7c892b

[denisri]

If this is an issue, it is probably also possible to switch the .ima files to nifti format, which is more standard. But we would need to make a new version of the data in the knowledge graph too.