o#+Title: Virtual Labs Phase 3 Hosting Process

Introduction

Virtual Labs is a mission mode project initiated by the Ministry of Human Resources and Development (MHRD). The objective of this project is to provide laboratory learning experience to the students who do not have access to adequate laboratory infrastructure. Currently, there are around 90 labs developed by various institutes in the consortium. A streamlined software development life cycle process followed for the development of these labs ensures high-quality labs. This document defines the hosting process to be followed by an integration engineer while hosting phase 3 lab units.

Motivation

A well-defined experiment hosting process will help maintain reliable servers. Consolidated information regarding all the deployments will also facilitate reporting.

Audience

The target audience for this document is the hosting team at IIITH who wants to host a unit on receiving the hosting request.

Hosting/Onboarding Process for Phase III Experiment/Labs

Commencing the hosting request

Keeping the deliverables/due date and the current hosting model in mind, the following process will need to be followed for getting a lab hosted on AWS during the Phase III of the Virtual Labs project ( ending on March 31st 2020). This is as per many discussions with IITB and the meeting on [2020-03-03 Tue].

VLEAD and IITB will follow the process detailed below for the hosting of labs which have been developed with each experiment having its own repository. All the Phase 3 labs and experiments hosted on AWS will have a common UI and report analytics at both lab and experiment level.

Step 1: IIITH will create lab repo for the lab that is ready ( marked in green) in the 200224_Deliverable_Status_Phase3 sheet shared by IITB and also populate the lab URLs in the sheet.
Step 2: IITB will raise an issue of type Phase 3 Onboarding Request and fill the lab name and lab URL ( as populated in the excel sheet by IIITH) and attach the R0 of the lab.
Step 3: IIITH will tag the head of the master branch of all the experiments in that lab as mentioned in the 200224_Deliverable_Status_Phase3 as v1.0.0
Step 4: IIITH will populate the received Phase 3 Onboarding Request with experiment details ( name, URL, tag, branch and build command) and Owner details and request for approval from IITB. It will be assumed by IIITH that no special hardware or software is required for the running of the experiments. IITB will have to specially state if this is not the case.
Step 5: IITB will validate the above data and approve the same on the received Phase 3 Onboarding Request .
Step 6: On approval from IITB , IIITH will host all the experiments and populate the lab landing page information (Introduction, Objective, List of experiments, target audience, course alignment ) from the R0 shared by IITB in the Phase 3 Onboarding Request.
Step 7: IIITH will host the lab and share the hosted URL link as part of the onboarding request issue and seek approval from the GitHub handle of the owner of the lab ( as recorded in the onboarding request)
Step 8: On approval from the Lab owner, IIITH will share the link with IIT Delhi.

Pre-Hosting/Rehosting Checklist

In this section, we will look at the instructions to fill or update the lab-descriptor.json file which is used to host a lab. Make Sure that everything in the JSON file is updated correctly before sending it for verification and hosting.
Section I - Lab
1. Name: Should be Discipline name
2. Link: - Crosscheck the discipline link from vlab.co.in and update correctly.
Discipline Should be in the following format:
- Mechanical Engineering
- Computer Science and Engineering
- Electronics and Communication Engineering
- Electrical Engineering
- Civil Engineering
- Chemical Sciences
- Biotechnology and Biomedical Engineering
- Physical Sciences
- Chemical Engineering
- Humanity
- Metallurgical and Materials Engineering
- Design Engineering
- Aerospace Engineering
Lab Name: Should be same in the JSON file, GitHub repo name and in var/www/html
Phase: Should be the number that the lab belongs to Phase 3 or so
Deploy: Should be true if we are hosting lab and exp or only lab. If we want to host only exp then it should be false
College Name: Update from the R0 file (which is given in the request for hosting issue)
Base URL: Crosscheck the lab domain in the reverse proxy and update correctly
Introduction: from R0 file
Section II - Experiment
Name: experiment name from R0 file
Short Name: it should be in small letters and not more than 4 words or can give exp repo name from GitLab (if it has only 4 words)
Repo: give the experiment repo URL from GitLab
Tag: for every request for hosting, a repo should have a new tag
Deploy: Should be true if we are hosting lab and exp or only exp. If we want to host only lab then it should be false
Section III
Target Audience, Objective and Courses Alignment: Update from R0 file

Hosting Lab and Experiments

In this section, we will discuss hosting a Lab and Experiments.

Step 1

Phase-3-Lab-Template to be cloned initially on the server.

Step 2

After cloning the repo the hosting person should go through the README file of Phase-3-Lab-Template repository.

Step 3

Run the following command to initialize a lab descriptor file that contains all the required fields, without any values. In the below command <path/to/lab/repo> should be replaced with the actual path to the local lab repository mentioned in the first step.

	      npm run labgen -- init <path/to/lab/repo>

Step 4

Fill all the values in the <path/to/lab/repo>/lab-descriptor.json file generated above by taking data from R0 file and create a pull-request to get the lab-descriptor approved.

Step 5

After getting approval from the legitimate authority, generate lab and build experiments.

Step 6

From the Phase-3-Lab-Template repository, run the following. This will generate the lab pages using the verified lab-descriptor.json from the lab repository and push the generated content to remote.

	      npm run labgen -- generate <path/to/lab/repo>

Step 7

Finally host lab and experiments. From the Phase-3-Lab-Template repository, run the following. This will copy the lab pages to /var/www/html/<lab-name>, then builds and copy experiments listed in the lab-descriptor to /var/www/html/<lab-name>/<exp>.

	      npm run labgen -- deploy <path/to/lab/repo>

Commencing the (re)hosting request

VLEAD and the institute requesting the rehosting of Phase 3 lab while changing the content of the lab will follow the process detailed below-

Step 1: Institute requesting rehosting will follow the link and locate the Phase III Lab/Experiment(s) OnBoarding Request issue corresponding to the lab that they wish to edit.
Step 2: They will comment on the issue with the content that they like to add to the lab pages following the format - Lab Section Name - New content
Step 3: VLEAD’s hosting engineer will host the lab with the requested changes and share the link on the same issue to seek approval from institute requesting rehosting.
Step 4: Institute requesting rehosting will need to test the hosted link and on their approval the hosted link will be shared with IITD (to be added to vlab.co.in).

(Re)Hosting Lab and Experiments

In this section, we will discuss (re)hosting a Lab and Experiments.

Step 1

Check for Phase-3-Lab-Template and lab repo on the server.

Step 2

Look for the content to be updated(ie., lab or experiments).

Step 3

lab-descriptor.json has fields like ‘deployLab’ under lab name and ‘deploy’ under experiment. When these fields are set true, the particular lab or experiment will be deployed.

Step 4

Mark the values as true or false in the <path/to/lab/repo>/lab-descriptor.json file(present in the lab repo) and create a pull-request to get the lab-descriptor approved.

Step 5

After getting approval from the legitimate authority, generate lab, and build experiments.

Step 6

From the Phase-3-Lab-Template repository, run the following. This will generate the lab pages using the verified lab-descriptor.json from the lab repository and push the generated content to remote.

	      npm run labgen -- generate <path/to/lab/repo>

Step 7

Finally host lab and experiments. From the Phase-3-Lab-Template repository, run the following. This will copy the lab pages to /var/www/html/<lab-name>, then builds and copy experiments listed in the lab-descriptor to /var/www/html/<lab-name>/<exp>.

	      npm run labgen -- deploy <path/to/lab/repo>

Steps for creating a new lab entry(Phase 3)

Build and deploy sources to correct path i.e., (/var/www/html/example-lab) {Hosting Engineer}
Check for example-lab and sources at var/www/html/example-lab {Hosting Engineer}
If step 1 and step 2 are true then go to step 4 else go to step 1 {Hosting Engineer}
Hosting Engineer needs to raise a request to System Engineer
Add entry to virtual host file in apache configuration(Phase3) {Systems Engineer}
Reload the apache server {Systems Engineer}
Add entry to ALB {Raj}
Finally, add entry in DNS server {Systems Engineer}
Reload DNS server {Systems Engineer}

NOTE

Do not add entries without sources as it will create duplicate date in analytics.
When modifying entries, try modifying before 9:00 AM or after 5:00 PM, it will give buffer time if something goes wrong and wont effect live users.

Post Hosting Tasks

In this section, we will look at the tasks to be done after hosting a unit,
Task I - Testing
- A quick SMOKE TEST to be performed on the hosted unit. We will talk about this in the next section.
Task II - Updating hosting request
- After successfuly hosting a unit, go to the hosting request and update the request with the following:
  - Hosted URL
  - Branch
  - Tag
  - Approval
Task III - Updating central hosting sheet
- Final task would be to update the central hosting sheet with the following:
  - Hosted URL
  - Current Tag
  - Previous Tag
  - Hosting request date
  - Hosting Date
  - Link to Hosting Request
  - Requester
  - Status(Approval)
  - Number of Experiment
  - Added to analytics server

Testing

In this section, we will discuss testing of a hosted unit.

Step 1: A quick SMOKE TEST to be performed on the hosted unit.
Step 2: If any issues are found in SMOKE TEST, the hosting unit to be debugged and (re)hosted.

Hosting Outcome

A unit of lab or experiment is hosted on AWS VM.
All the hosted units will have the same UI ie.., UI 3.0 with analytics integrated into them at the time of building with lab generation script.

FAQ

Can you provide a sample JSON file with data?

This is a sample JSON file with data.

{
    "broadArea": {
	"name": "Civil Engineering",
	"link": "http://vlab.co.in/broad-area-civil-engineering"
    },
	"lab": "Fluid Machinery Lab",	
	"phase": "3",
        "deployLab":true,
        "collegeName": "NITS",
        "baseUrl": "http://fmc-nitk.vlabs.ac.in",
	"introduction": "Welcome to the Fluid Machinery Lab",
	"experiments": [
	    {
			"name": "Performance Characteristics of Centrifugal Pump",
			"short-name": "centrifugal-pump",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/performance-characteristics-centrifugal-pump-nitk.git",
	 	        "tag": "v1.0.0",
	         	"deploy": true
	    }, {
			"name": "Performance Characteristics of Hydraulic Ram",
			"short-name": "hydraulic-ram",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/performance-characteristics-hydraulic-ram-nitk.git",
		        "tag": "v1.0.0",
	         	"deploy": true
	    }, {
			"name": "Performance Characteristics of Kaplan Turbine",
			"short-name": "kaplan-turbine",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/performance-characteristics-kaplan-turbine-nitk.git",
		        "tag": "v1.0.0",
		        "deploy": true
	    }, {
			"name": "Double Acting Reciprocating Pump",
			"short-name": "reciprocating-pump",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/double-acting-reciprocating-pump-nitk.git",
		        "tag": "v1.0.0",
		        "deploy": true
	    }, {
			"name": "Performance Characteristics of Pelton Turbine",
			"short-name": "pelton-turbine",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/performance-characteristics-pelton-turbine-nitk.git",
		        "tag": "v1.0.0",
		        "deploy": true
	    }, {
			"name": "Pipe Bursting",
			"short-name": "pipe-bursting",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/pipe-bursting-nitk.git",
		        "tag": "v1.0.0",
		        "deploy": true
	    }, {
			"name": "Performance Characteristics of Francis Turbine",
			"short-name": "francis-turbine",
			"repo": "http://vlabs.iitb.ac.in/gitlab/vlabs-dev-central-hosting/fluid-machinery-nitk/performance-characteristics-francis-turbine-nitk.git",
		        "tag": "v1.0.0",
		        "deploy": true
        }
    ],
    "targetAudience": {
	"UG": [
	    "B. Tech./ B.E in Civil Engineering"
	],
	"PG": [
	    "MS/Ph. D. Beginners in Civil Engineering and related topics"
	]
    },
    "objective": "Many of our daily life activity depends on fluid machinery so it is important to understand the functioning principles and characteristics of such machinery from an engineering point of view. This lab focuses on giving the user an idea about practical behaviour of such machines before performing operations on a real system.",
    "courseAlignment": {
	"description": "The syllabi of this lab aligns to the following universities in India.",
	"universities": [
	    "Visvesvaraya Technological University Karnataka",
	    "National Institute of Technology Karnataka Karnataka",
	    "Anna University Tamil Nadu",
	    "SRM University Delhi - NCR Sonepat Haryana",
	    "University of Kerala Kerala",
	    "APJ Abdul Kalam Technological University Kerala",
	    "CUSAT Kerala"
	]
    }
}

Can you explain about input data attributes in the JSON file?

Field	Type	Purpose	Example
Broad Area Name	String	The Domain of the lab	`Civil Engineering`
		to which the experiments belong
Broad Area Link	URL	Link to the page on vlabs	`http://vlabs.co.in/civil-engineering`
		website that holds all the
		labs in the given domain
Lab Name	String	Name of the lab, as it is displayed	`Fluid Machinery Lab`
		on the lab pages. This field is also
		used to generate urls for lab and
		experiment pages.
Lab Base-url	URL	Domain name part of the lab and	`http://fmc-nitk.vlabs.ac.in`
		experiment URLs.
Experiments	List of Experiments	Each experiment in the lab is stored as a	`[{"name": "Exp1", "short-name": "exp1", "repo": "gitlab.com/exp1", "tag": "v1.0.0", "deploy": "true", }, ...]`
		record containing two fields: experiment
		name and link to the repository

Conclusion

This document highlights the hosting process. It aims at making an integration engineer understand the hosting process of Phase 3 Labs.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

hosting-process-ph3.org

hosting-process-ph3.org

Introduction

Motivation

Audience

Hosting/Onboarding Process for Phase III Experiment/Labs

Commencing the hosting request

Pre-Hosting/Rehosting Checklist

Hosting Lab and Experiments

Commencing the (re)hosting request

(Re)Hosting Lab and Experiments

Steps for creating a new lab entry(Phase 3)

NOTE

Post Hosting Tasks

Testing

Hosting Outcome

FAQ

Can you provide a sample JSON file with data?

Can you explain about input data attributes in the JSON file?

Conclusion

Files

hosting-process-ph3.org

Latest commit

History

hosting-process-ph3.org

File metadata and controls

Introduction

Motivation

Audience

Hosting/Onboarding Process for Phase III Experiment/Labs

Commencing the hosting request

Pre-Hosting/Rehosting Checklist

Hosting Lab and Experiments

Commencing the (re)hosting request

(Re)Hosting Lab and Experiments

Steps for creating a new lab entry(Phase 3)

NOTE

Post Hosting Tasks

Testing

Hosting Outcome

FAQ

Can you provide a sample JSON file with data?

Can you explain about input data attributes in the JSON file?

Conclusion