Update admin documentation #96

prasadtalasila · 2023-09-04T13:53:23Z

Make sure that all the code in deploy/ and script/ have been correctly ported from release-v0.2 branch to feature/distributed-demo. (cross-check PR Feature/distributed demo #94) - (Done)
The "Admin" tab is really "Installation", though there may be administrative (post-install) tasks there too. But it leaps into the detail of the installation without explaining that there are multiple stages and different bits of software that need to be installed in stages. This level of description may make sense for members of the development team, but to someone unfamiliar it is very difficult to follow/understand. - (Done)
There needs to be an overview of the installation in terms of "we're going to do this, then we're going to do that" so that an unfamiliar user can follow it more easily. Ideally, you need to be able to take a smart (but unfamiliar) person and put them in a room with this website, and have them follow your instructions to install the product. As it stands, I think they would be very confused very quickly.(cross-check PR Updates docs #215) - (Done)
The instructions and install.sh need to be updated for basepath. All the ml-workspace containers need to have basepath as well. See issue Update install instructions for basepath #88 for more information
Add installation instructions for Gitlab OAuth integration. - (Done)
Add representative network diagrams on all the installation pages (the docs/developer/system/DTaaS.drawio) (cross-check PR Updates docs #215) - (Done)
Admin --> Installation --> Cookbook page
- create a docs/admin/guides to add notes on modifications to the standard deployment scenarios. Currently known scenarios:
- add more users - (Done)
- add other services
- link services to local ports using SSH port mapping
- restrict user to read only permissions to common/ directory
- hosting the site without SSL certificates
- add self-signed (ssl/) / LetsEncrypt certificates
- add or remove TLS certificates to gateway
Update servers/config/gateway/README.md to include auth in the volume mapping of docker container. - (Done)

Update the servers/config/gateway/README.md for traefik-gateway launch command to use auth. - (Done)

docker run -d \
 --name "traefik-gateway" \
 --network=host -v "$PWD/traefik.yml:/etc/traefik/traefik.yml" \
 -v "$PWD/auth:/etc/traefik/auth" \
 -v "$PWD/dynamic:/etc/traefik/dynamic" \
 -v /var/run/docker.sock:/var/run/docker.sock \
 traefik:v2.5

Document basepath installations and possibility of multiple installations. See issue #88
One Mermaid diagram showing the installation of different software components (cross-check PR Updates docs #215) - (Done)

The text was updated successfully, but these errors were encountered:

prasadtalasila · 2023-09-17T07:13:19Z

The Ubuntu desktop install is a good inspiration for structuring the install documentation.

The openai cookbook indexing and linking is a good structure to have. The linked pages are to be relative links pointing to our mkdocs pages. Another example is Devcookbook.

nichlaes · 2023-09-26T12:54:48Z

Make sure that all the code in deploy/ and script/ have been correctly ported from release-v0.2 branch to feature/distributed-demo. (cross-check PR #94)

a install script is located under /deploy/install.sh but the script uses scripts from the folder /scripts/. e.g.

bash script/env.sh || exit

Won't this be a problem, if you run it from the /deploy/ folder?

prasadtalasila · 2023-09-26T13:12:27Z

The users are need to run it from top-level. Please see this README.

source deploy/install.sh

The installation steps require strict sequence of steps. We will improve upon them to automate most of these steps in near future.

nichlaes · 2023-09-27T11:39:04Z

Update the servers/config/gateway/README.md for traefik-gateway launch command to use auth.

Updated on branch 96-update-admin-documentation

nichlaes · 2023-09-27T11:46:47Z

@prasadtalasila

Comments:

Admin section:

Add a general purpose section
A section which shortly describes the role as an admin for the system and what the goal is. What am I installing? It relates to task/comment (2.), but maybe an even more general intro/overview of the admin role would make sense.

Questions:

Should I run test the 'admin doc install guide' on the DTaaS integration server or do you have another linux server available I can test on? My own PC is running windows machine.

The instructions and install.sh need to be updated for basepath. All the ml-workspace containers need to have basepath as well. See issue Update install instructions for basepath #88 for more information

I know me talked about this one, but I'm not sure what and where to update. Is this already done?

Add representative network diagrams on all the installation pages (the docs/developer/system/DTaaS.drawio)

Are the network diagram already made? If yes which one is it in the draw.io file, the first one?

Admin --> Installation --> Cookbook page

- create a `docs/admin/guides` to add notes on modifications to the standard deployment scenarios. Currently known scenarios:
- add more users
- add other services
- link services to local ports using SSH port mapping
- admin user with read-write permissions to common/ directory
- add self-signed (`ssl/`) / LetsEncrypt certificates
- add or remove TLS certificates to gateway
- hosting the site without SSL certificates

Is this already written somewhere or do I need to write this? If so I think this would take me some time, because I have a knowledge gap, and each scenario would need to be specified further in order for me to understand.

Document basepath installations and possibility of multiple installations. See issue #88

Same question as number 4.

One Mermaid diagram showing the installation of different software components

Not sure how to do this one and what the requirements are. Is it to take one of the draw.io diagrams and convert to Mermaid diagram?

prasadtalasila · 2023-09-29T11:54:43Z

@prasadtalasila

Comments:

Admin section:

1. **Add a general purpose section**
   A section which shortly describes the role as an admin for the system and what the goal is. What am I installing? It relates to task/comment (2.), but maybe an even more general intro/overview of the admin role would make sense.

The Ubuntu install procedure gives a template for what we can write about the installation steps. Let us follow that for now.

Questions:

Should I run test the 'admin doc install guide' on the DTaaS integration server or do you have another linux server available I can test on? My own PC is running windows machine.

The integration server

The instructions and install.sh need to be updated for basepath. All the ml-workspace containers need to have basepath as well. See issue Update install instructions for basepath #88 for more information

I know me talked about this one, but I'm not sure what and where to update. Is this already done?
Not done yet. The places where the changes may be required are:

gateway config in deployment
install script
single script install script
New documentation file for basepath
Minor cleanup of client config in deploy and client/config

Add representative network diagrams on all the installation pages (the docs/developer/system/DTaaS.drawio)

Are the network diagram already made? If yes which one is it in the draw.io file, the first one?
No

Admin --> Installation --> Cookbook page
- create a `docs/admin/guides` to add notes on modifications to the standard deployment scenarios. Currently known scenarios:
- add more users
- add other services
- link services to local ports using SSH port mapping
- admin user with read-write permissions to common/ directory
- add self-signed (`ssl/`) / LetsEncrypt certificates
- add or remove TLS certificates to gateway
- hosting the site without SSL certificates
Is this already written somewhere or do I need to write this? If so I think this would take me some time, because I have a knowledge gap, and each scenario would need to be specified further in order for me to understand.

I have the steps written in a markdown file. We can discuss a way to get this done easily

Document basepath installations and possibility of multiple installations. See issue #88

Same question as number 4.

One Mermaid diagram showing the installation of different software components

Not sure how to do this one and what the requirements are. Is it to take one of the draw.io diagrams and convert to Mermaid diagram?
The mermaid is a JS package to draw UML diagrams. It is already integrated into the documentation pipeline. Please see this example diagram for library microservice. You will probably be making a deployment diagram for this. If mermaid is not capable of doing this, we will use the drawio.

Hope the explanation helps. We will meet soon anyway and so we can discuss more soon. Thanks.

nichlaes · 2023-10-12T08:18:58Z

@prasadtalasila
I have started restructuring some of the documentation.

I have noted some reflections while trying the trial installation.

When the installation is complete (e.g trial), how do I know that it is complete? Should I just access the site, or should I also try out a DigitalTwin/algorithm.
When accessing the website, I am prompted by a username and password, this is not mentioned in the installation guide?
Does the OAuth Provider have to be GitLab or could it be any OAuthProvider?

prasadtalasila · 2023-10-18T21:36:27Z

@nichlaes am not sure if you received answers to these questions. In case you did not receive them, please see below.

@prasadtalasila I have started restructuring some of the documentation.

I have noted some reflections while trying the trial installation.
* When the installation is complete (e.g trial), how do I know that it is complete? Should I just access the site, or should I also try out a DigitalTwin/algorithm.

For now, just access the website and check all the steps outlined in docs/user/website/index.md.

* When accessing the website, I am prompted by a username and password, this is not mentioned in the installation guide?

This is the password check made by Traefik gateway. It should be made obvious in the installation documentation.

Your idea of post-install checklist is a good one. Can you please add a page to the admin documentation section describing the checklist steps and the expected results?

* Does the OAuth Provider have to be GitLab or could it be any OAuthProvider?

It can be any OAuth provider. But the gitlab is also integrated into other components of DTaaS software. Thus having another OAuth provider will be extra work for user. This question and the explanation is a good candidate for admin cookbook section.

prasadtalasila · 2023-10-25T12:53:08Z

Adding more users

Create new account with on gitlab

Within DTaaS codebase

## create required files
cd DTaaS/files
cp -R user2 <username>

## start new workspace for user
cd ..
docker run -d \
 -p <port>:8080 \
  --name "ml-workspace-<username>" \
  -v "${TOP_DIR}/files/<username>:/workspace" \
  -v "${TOP_DIR}/files/<username>:/workspace/common" \
  --env AUTHENTICATE_VIA_JUPYTER="" \
  --env WORKSPACE_BASE_URL="<username>" \
  --shm-size 512m \
  --restart always \
  mltooling/ml-workspace:0.13.2

## add route in gateway
cd servers/config/gateway
htpassword auth <username>

vi dynamic/fileConfig.yml

http:
  routers:
    ....
    <username>:
      entryPoints:
        - http
      rule: 'Host(`foo.com`) && PathPrefix(`/<username>`)'
      middlewares:
        - basic-auth
      service: <username>

  services:
    ...
    <username>:
      loadBalancer:
        servers:
          - url: 'http://localhost:<port>'

Add new services

Provide instructions for installing MongoDB

link services to local ports using SSH port mapping

do SSH-link from workspace to remote services
ssh -fNT -L 15672:localhost:15672 [email protected]
ssh -fNT -L 5672:localhost:5672 [email protected]

Set read only access to common asset area:

docker run -d \
 -p 8091:8080 \
  --name "ml-workspace-user2" \
  -v "${TOP_DIR}/files/user2:/workspace" \
  -v "${TOP_DIR}/files/common:/workspace/common:ro" \
  --env AUTHENTICATE_VIA_JUPYTER="" \
  --env WORKSPACE_BASE_URL="user2" \
  --shm-size 512m \
  --restart always \
  mltooling/ml-workspace:0.13.2 || true

Hosting site without https

correct the following sentence on deploy/admin/overview.md

If you do not have a reverse proxy, please replace https://foo.com with http://foo.com in client .env file and in OAuth registration. Other installation configuration remains the same.

provide correct sample client env.js and OAuth table

prasadtalasila · 2023-11-24T21:39:09Z

PR #300 completes most of this work.

prasadtalasila assigned nichlaes Sep 4, 2023

prasadtalasila added this to the Release 0.3.0 milestone Sep 4, 2023

prasadtalasila mentioned this issue Sep 5, 2023

Port updates from release v0.2 to default branch #93

Closed

prasadtalasila added the documentation Improvements or additions to documentation label Sep 27, 2023

prasadtalasila modified the milestones: Release 0.3.0, Release v0.4.0 Nov 8, 2023

nichlaes linked a pull request Nov 17, 2023 that will close this issue

96 update admin documentation #276

Closed

prasadtalasila mentioned this issue Nov 24, 2023

Swap ml-workspace container and add admin cookbook #300

Merged

prasadtalasila closed this as completed Nov 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update admin documentation #96

Update admin documentation #96

prasadtalasila commented Sep 4, 2023 •

edited by nichlaes

Loading

prasadtalasila commented Sep 17, 2023 •

edited

Loading

nichlaes commented Sep 26, 2023

prasadtalasila commented Sep 26, 2023

nichlaes commented Sep 27, 2023

nichlaes commented Sep 27, 2023 •

edited

Loading

prasadtalasila commented Sep 29, 2023

Comments:

Admin section:

Questions:

nichlaes commented Oct 12, 2023

prasadtalasila commented Oct 18, 2023

prasadtalasila commented Oct 25, 2023 •

edited

Loading

prasadtalasila commented Nov 24, 2023

Update admin documentation #96

Update admin documentation #96

Comments

prasadtalasila commented Sep 4, 2023 • edited by nichlaes Loading

prasadtalasila commented Sep 17, 2023 • edited Loading

nichlaes commented Sep 26, 2023

prasadtalasila commented Sep 26, 2023

nichlaes commented Sep 27, 2023

nichlaes commented Sep 27, 2023 • edited Loading

Comments:

Admin section:

Questions:

prasadtalasila commented Sep 29, 2023

Comments:

Admin section:

Questions:

nichlaes commented Oct 12, 2023

prasadtalasila commented Oct 18, 2023

prasadtalasila commented Oct 25, 2023 • edited Loading

Adding more users

Create new account with on gitlab

Within DTaaS codebase

Add new services

link services to local ports using SSH port mapping

Set read only access to common asset area:

Hosting site without https

prasadtalasila commented Nov 24, 2023

prasadtalasila commented Sep 4, 2023 •

edited by nichlaes

Loading

prasadtalasila commented Sep 17, 2023 •

edited

Loading

nichlaes commented Sep 27, 2023 •

edited

Loading

prasadtalasila commented Oct 25, 2023 •

edited

Loading