Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: tutorial freestanding deployment #4435

Merged
merged 76 commits into from
Feb 13, 2024

Conversation

cdekater
Copy link
Contributor

@cdekater cdekater commented Nov 4, 2023

This tutorial was created at the IETF 118 hackathon. The goal is to make it easier to start tinkering with SCION outside of the complex and somewhat confusing development environment.

This change is Reviewable

Copy link
Contributor

@johnstudarus johnstudarus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 16 of 16 files at r1, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @cdekater)

@johnstudarus
Copy link
Contributor

:lgtm:

Copy link
Contributor

@matzf matzf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 16 files reviewed, 2 unresolved discussions (waiting on @cdekater and @johnstudarus)


doc/tutorials/deploy.rst line 214 at r2 (raw file):

   .. code-block::

      /usr/local/scion/scion-pki testcrypto -t TutorialDeploymentTopology.topo

I've been thinking about this. This testcrypto thing is as opaque as it gets. Even the topo file is confusing. What I would do is to write out a bash script that does everything explicitly. The people can skim through it and get an idea of what's going on. We still make it simple by doing it entirely on one host. We can document that this doesn't make much sense, but is done for convenience...

What do you think?


doc/tutorials/deploy.rst line 327 at r2 (raw file):

   4 Hops:
   [2] Hops: [42-ffaa:1:1 2>1 42-ffaa:1:2 2>2 42-ffaa:1:3 4>2 42-ffaa:1:5] MTU: 1472 NextHop: 127.0.0.1:31002 Status: alive LocalIP: 127.0.0.1
   [3] Hops: [42-ffaa:1:1 3>1 42-ffaa:1:3 2>2 42-ffaa:1:2 3>1 42-ffaa:1:5] MTU: 1472 NextHop: 127.0.0.1:31002 Status: alive LocalIP: 127.0.0.1

Maybe a conclusion would be nice? I tried to write one, but didn't really know what to say. What do we want people to take away from this?

Maybe, this could also be the place to put this into perspective; what is different from a real world deployment, what is missing.

  • certificate renewal. This will stop working after three days.
  • single host ASes
  • gateway
  • ?

Copy link
Contributor

@matzf matzf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 16 files reviewed, 2 unresolved discussions (waiting on @cdekater and @johnstudarus)


doc/tutorials/deploy.rst line 214 at r2 (raw file):
Here's a relatively straight forward script that does this. Note that there is some slight deviation from the .topo file; here only AS1 and AS3 are voting, and only AS1 and AS2 are issuing. Makes it slightly shorter and shows how the roles can be different.
IMO this makes it much more transparent what is happening, and instructing "students" to run this script is at least as easy as the testcrypto command.

The output structure is somewhat flatter than what is expected by the control service. The AS certificates need to be installed with

        for i in {1..5}; do
           scp  gen/AS$i/cp-as.{pem,key} scion0$i:/etc/scion/crypto/as/
           scp  gen/ISD42-B1-S1.trc  scion0$i:/etc/scion/certs/
        done

Note: we don't need the other keys/certificates on the individual hosts (CA / certificate renewal not enabled).

#!/bin/bash

set -euo pipefail

mkdir AS{1..5}

# Create voting and root keys and (self-signed) certificates for core ASes
pushd AS1
scion-pki certificate create --profile=sensitive-voting <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 sensitive voting cert"}') sensitive-voting.pem sensitive-voting.key
scion-pki certificate create --profile=regular-voting <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 regular voting cert"}') regular-voting.pem regular-voting.key
scion-pki certificate create --profile=cp-root <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 cp root cert"}') cp-root.pem cp-root.key
popd

pushd AS2
scion-pki certificate create --profile=cp-root <(echo '{"isd_as": "42-ffaa:1:2", "common_name": "42-ffaa:1:2 cp root cert"}') cp-root.pem cp-root.key
popd

pushd AS3
scion-pki certificate create --profile=sensitive-voting <(echo '{"isd_as": "42-ffaa:1:3", "common_name": "42-ffaa:1:3 sensitive voting cert"}') sensitive-voting.pem sensitive-voting.key
scion-pki certificate create --profile=regular-voting <(echo '{"isd_as": "42-ffaa:1:3", "common_name": "42-ffaa:1:3 regular voting cert"}') regular-voting.pem regular-voting.key
popd

# Create the TRC
mkdir tmp
echo '
isd = 42
description = "Demo ISD 42"
serial_version = 1
base_version = 1
voting_quorum = 2

core_ases = ["ffaa:1:1", "ffaa:1:2", "ffaa:1:3"]
authoritative_ases = ["ffaa:1:1", "ffaa:1:2", "ffaa:1:3"]
cert_files = ["AS1/sensitive-voting.pem", "AS1/regular-voting.pem", "AS1/cp-root.pem", "AS2/cp-root.pem", "AS3/sensitive-voting.pem", "AS3/regular-voting.pem"]

[validity]
not_before = '$(date +%s)'
validity = "365d"' \
> trc-B1-S1-pld.tmpl

scion-pki trc payload --out=tmp/ISD42-B1-S1.pld.der --template trc-B1-S1-pld.tmpl
rm trc-B1-S1-pld.tmpl

# Sign and bundle the TRC
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS1/sensitive-voting.{pem,key} --out tmp/ISD42-B1-S1.AS1-sensitive.trc
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS1/regular-voting.{pem,key} --out tmp/ISD42-B1-S1.AS1-regular.trc
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS3/sensitive-voting.{pem,key} --out tmp/ISD42-B1-S1.AS3-sensitive.trc
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS3/regular-voting.{pem,key} --out tmp/ISD42-B1-S1.AS3-regular.trc

scion-pki trc combine tmp/ISD42-B1-S1.AS{1,3}-{sensitive,regular}.trc --payload tmp/ISD42-B1-S1.pld.der --out ISD42-B1-S1.trc
rm tmp -r

# Create CA key and certificate for issuing ASes
pushd AS1
scion-pki certificate create --profile=cp-ca <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 CA cert"}') cp-ca.pem cp-ca.key --ca cp-root.pem --ca-key cp-root.key
popd
pushd AS2
scion-pki certificate create --profile=cp-ca <(echo '{"isd_as": "42-ffaa:1:2", "common_name": "42-ffaa:1:2 CA cert"}') cp-ca.pem cp-ca.key --ca cp-root.pem --ca-key cp-root.key
popd

# Create AS key and certificate chains
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 AS cert"}') AS1/cp-as.pem AS1/cp-as.key --ca AS1/cp-ca.pem --ca-key AS1/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:2", "common_name": "42-ffaa:1:2 AS cert"}') AS2/cp-as.pem AS2/cp-as.key --ca AS2/cp-ca.pem --ca-key AS2/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:3", "common_name": "42-ffaa:1:3 AS cert"}') AS3/cp-as.pem AS3/cp-as.key --ca AS1/cp-ca.pem --ca-key AS1/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:4", "common_name": "42-ffaa:1:4 AS cert"}') AS4/cp-as.pem AS4/cp-as.key --ca AS1/cp-ca.pem --ca-key AS1/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:5", "common_name": "42-ffaa:1:5 AS cert"}') AS5/cp-as.pem AS5/cp-as.key --ca AS2/cp-ca.pem --ca-key AS2/cp-ca.key --bundle

@johnstudarus
Copy link
Contributor

johnstudarus commented Nov 13, 2023

I like the new expanded out script! @alicecuii is going to test it out and report back.

Copy link
Contributor

@nicorusti nicorusti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 3 of 16 files reviewed, 2 unresolved discussions (waiting on @johnstudarus and @matzf)


doc/tutorials/deploy.rst line 214 at r2 (raw file):

Previously, matzf (Matthias Frei) wrote…

Here's a relatively straight forward script that does this. Note that there is some slight deviation from the .topo file; here only AS1 and AS3 are voting, and only AS1 and AS2 are issuing. Makes it slightly shorter and shows how the roles can be different.
IMO this makes it much more transparent what is happening, and instructing "students" to run this script is at least as easy as the testcrypto command.

The output structure is somewhat flatter than what is expected by the control service. The AS certificates need to be installed with

        for i in {1..5}; do
           scp  gen/AS$i/cp-as.{pem,key} scion0$i:/etc/scion/crypto/as/
           scp  gen/ISD42-B1-S1.trc  scion0$i:/etc/scion/certs/
        done

Note: we don't need the other keys/certificates on the individual hosts (CA / certificate renewal not enabled).

#!/bin/bash

set -euo pipefail

mkdir AS{1..5}

# Create voting and root keys and (self-signed) certificates for core ASes
pushd AS1
scion-pki certificate create --profile=sensitive-voting <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 sensitive voting cert"}') sensitive-voting.pem sensitive-voting.key
scion-pki certificate create --profile=regular-voting <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 regular voting cert"}') regular-voting.pem regular-voting.key
scion-pki certificate create --profile=cp-root <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 cp root cert"}') cp-root.pem cp-root.key
popd

pushd AS2
scion-pki certificate create --profile=cp-root <(echo '{"isd_as": "42-ffaa:1:2", "common_name": "42-ffaa:1:2 cp root cert"}') cp-root.pem cp-root.key
popd

pushd AS3
scion-pki certificate create --profile=sensitive-voting <(echo '{"isd_as": "42-ffaa:1:3", "common_name": "42-ffaa:1:3 sensitive voting cert"}') sensitive-voting.pem sensitive-voting.key
scion-pki certificate create --profile=regular-voting <(echo '{"isd_as": "42-ffaa:1:3", "common_name": "42-ffaa:1:3 regular voting cert"}') regular-voting.pem regular-voting.key
popd

# Create the TRC
mkdir tmp
echo '
isd = 42
description = "Demo ISD 42"
serial_version = 1
base_version = 1
voting_quorum = 2

core_ases = ["ffaa:1:1", "ffaa:1:2", "ffaa:1:3"]
authoritative_ases = ["ffaa:1:1", "ffaa:1:2", "ffaa:1:3"]
cert_files = ["AS1/sensitive-voting.pem", "AS1/regular-voting.pem", "AS1/cp-root.pem", "AS2/cp-root.pem", "AS3/sensitive-voting.pem", "AS3/regular-voting.pem"]

[validity]
not_before = '$(date +%s)'
validity = "365d"' \
> trc-B1-S1-pld.tmpl

scion-pki trc payload --out=tmp/ISD42-B1-S1.pld.der --template trc-B1-S1-pld.tmpl
rm trc-B1-S1-pld.tmpl

# Sign and bundle the TRC
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS1/sensitive-voting.{pem,key} --out tmp/ISD42-B1-S1.AS1-sensitive.trc
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS1/regular-voting.{pem,key} --out tmp/ISD42-B1-S1.AS1-regular.trc
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS3/sensitive-voting.{pem,key} --out tmp/ISD42-B1-S1.AS3-sensitive.trc
scion-pki trc sign tmp/ISD42-B1-S1.pld.der AS3/regular-voting.{pem,key} --out tmp/ISD42-B1-S1.AS3-regular.trc

scion-pki trc combine tmp/ISD42-B1-S1.AS{1,3}-{sensitive,regular}.trc --payload tmp/ISD42-B1-S1.pld.der --out ISD42-B1-S1.trc
rm tmp -r

# Create CA key and certificate for issuing ASes
pushd AS1
scion-pki certificate create --profile=cp-ca <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 CA cert"}') cp-ca.pem cp-ca.key --ca cp-root.pem --ca-key cp-root.key
popd
pushd AS2
scion-pki certificate create --profile=cp-ca <(echo '{"isd_as": "42-ffaa:1:2", "common_name": "42-ffaa:1:2 CA cert"}') cp-ca.pem cp-ca.key --ca cp-root.pem --ca-key cp-root.key
popd

# Create AS key and certificate chains
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:1", "common_name": "42-ffaa:1:1 AS cert"}') AS1/cp-as.pem AS1/cp-as.key --ca AS1/cp-ca.pem --ca-key AS1/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:2", "common_name": "42-ffaa:1:2 AS cert"}') AS2/cp-as.pem AS2/cp-as.key --ca AS2/cp-ca.pem --ca-key AS2/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:3", "common_name": "42-ffaa:1:3 AS cert"}') AS3/cp-as.pem AS3/cp-as.key --ca AS1/cp-ca.pem --ca-key AS1/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:4", "common_name": "42-ffaa:1:4 AS cert"}') AS4/cp-as.pem AS4/cp-as.key --ca AS1/cp-ca.pem --ca-key AS1/cp-ca.key --bundle
scion-pki certificate create --profile=cp-as <(echo '{"isd_as": "42-ffaa:1:5", "common_name": "42-ffaa:1:5 AS cert"}') AS5/cp-as.pem AS5/cp-as.key --ca AS2/cp-ca.pem --ca-key AS2/cp-ca.key --bundle

I like this script, and the fact that you can choose if you want to generate all the material automagically or if you want to go step by step to understand what is going on! I think it will be important to add a couple of lines to day what the script does.

Thanks a lot for proposing this!


doc/tutorials/deploy.rst line 327 at r2 (raw file):

Previously, matzf (Matthias Frei) wrote…

Maybe a conclusion would be nice? I tried to write one, but didn't really know what to say. What do we want people to take away from this?

Maybe, this could also be the place to put this into perspective; what is different from a real world deployment, what is missing.

  • certificate renewal. This will stop working after three days.
  • single host ASes
  • gateway
  • ?

Absolutely. I think this could in the long term be a step by step guide for setting up a laboratory, so I agree that the points above, and perhaps a host with a SICON native application, could be some of the logical next steps.

@nicorusti
Copy link
Contributor

JFYI the compiled documentation is visible here: https://scion--4435.org.readthedocs.build/en/4435/tutorials/deploy.html

@jiceatscion is there a lot missing for this to be merged? It would be a nice addition to the existing doc

@jiceatscion
Copy link
Contributor

jiceatscion commented Jan 29, 2024 via email

@oncilla
Copy link
Contributor

oncilla commented Jan 31, 2024

Maybe it would be worth getting feedback from James (https://scionproto.slack.com/archives/C8ADA9CEP/p1705511756799569) too, given he was using the draft as guidance for his experiments.

Copy link
Contributor

@jiceatscion jiceatscion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the content, either as-is or, preferably with the suggested changes. Since Corine might not be available to incorporate the changes, I don't mind doing that once there's agreement on all the suggestions.
:lgtm:

Reviewed 12 of 13 files at r2, 1 of 1 files at r3, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @cdekater, @matzf, and @nicorusti)


doc/tutorials/deploy.rst line 214 at r2 (raw file):

Previously, nicorusti (Nicola Rustignoli) wrote…

I like this script, and the fact that you can choose if you want to generate all the material automagically or if you want to go step by step to understand what is going on! I think it will be important to add a couple of lines to day what the script does.

Thanks a lot for proposing this!

I agree that referring to a secret command meant for CI testing isn't good. I prefer the script too. Let's just put in a copyable code block and let people use it in the slow or fast way, as they see fit.


doc/tutorials/deploy.rst line 327 at r2 (raw file):

Previously, nicorusti (Nicola Rustignoli) wrote…

Absolutely. I think this could in the long term be a step by step guide for setting up a laboratory, so I agree that the points above, and perhaps a host with a SICON native application, could be some of the logical next steps.

Let's not make a Yak for us to shave: we'd have to write that second tutorial before we can refer to it. How about doing what every academic paper does: remind the reader that we told them exactly what we promised in the intro:

"Congratulations, you now have a working SCION configuration, which consists of a stand-alone complete SCION environment distributed among five computers. This environment contains one SCION Isolation Domain (ISD), with three core ASes and two non-core, leaf ASes. Being a demo, this configuration has some limitations:

  • The certificates are only good for three days unless explicitly renewed using scion-pki help certificate renew
  • Each AS contains a single host running all the SCION services. In a typical deployment, these services would run a separate hosts and include multiple border routers.
  • This environment does not include a SCION-IP gateway.

If you would like to learn more and help develop SCION, consider using the development environment. See :ref:setting-up-the-development-environment.

If you would like to use SCION in a larger deployment, consider using the SCIONLab. For more information, see https://www.scionlab.org/.


doc/tutorials/deploy.rst line 4 at r3 (raw file):

=================================

This document helps you set up a SCION demo configuration, which consists of a stand-alone full-scale SCION environment distributed among five computers. The demo environment contains one SCION Isolation Domain (ISD), with three core ASes and two non-core, leaf ASes.

complete?

Code quote:

 full-scale

- Update instructions for installation from packages

  Simplified: removed choice to download nightly build or release.
  Updated instructions for starting service (using systemd) and
  paths to tools.
  Some of the configuration files are no longer needed as they come with
  defaults from the packages.

- Remove the scion-pki testcrypto invocation in favor of a lengthy but
  more transparent script to generate all the crypto material

  As the "global" .topo file is no longer necessary, simplify the
  terminology around the "normal", "AS specific" topology.json files.

- Misc simplifications of the text where instructions were overly
  repetitive.

- Add cross-references to glossary and manuals

- Add conclusion (text suggested by JC)
Copy link
Contributor

@matzf matzf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the liberty to update this branch as discussed.

Update tutorial to install deb packages, avoid testcrypto command

  • Update instructions for installation from packages

    Simplified: removed choice to download nightly build or release.
    Updated instructions for starting service (using systemd) and
    paths to tools.
    Some of the configuration files are no longer needed as they come with
    defaults from the packages.

  • Remove the scion-pki testcrypto invocation in favor of a lengthy but
    more transparent script to generate all the crypto material

    As the "global" .topo file is no longer necessary, simplify the
    terminology around the "normal", "AS specific" topology.json files.

  • Misc simplifications of the text where instructions were overly
    repetitive.

  • Add cross-references to glossary and manuals

  • Add conclusion (text suggested by JC)

Reviewable status: 12 of 17 files reviewed, 3 unresolved discussions (waiting on @cdekater, @jiceatscion, @johnstudarus, and @nicorusti)

Copy link
Contributor

@matzf matzf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 12 of 17 files reviewed, 1 unresolved discussion (waiting on @cdekater, @jiceatscion, @johnstudarus, and @nicorusti)


doc/tutorials/deploy.rst line 214 at r2 (raw file):

Previously, jiceatscion wrote…

I agree that referring to a secret command meant for CI testing isn't good. I prefer the script too. Let's just put in a copyable code block and let people use it in the slow or fast way, as they see fit.

Done.


doc/tutorials/deploy.rst line 327 at r2 (raw file):

Previously, jiceatscion wrote…

Let's not make a Yak for us to shave: we'd have to write that second tutorial before we can refer to it. How about doing what every academic paper does: remind the reader that we told them exactly what we promised in the intro:

"Congratulations, you now have a working SCION configuration, which consists of a stand-alone complete SCION environment distributed among five computers. This environment contains one SCION Isolation Domain (ISD), with three core ASes and two non-core, leaf ASes. Being a demo, this configuration has some limitations:

  • The certificates are only good for three days unless explicitly renewed using scion-pki help certificate renew
  • Each AS contains a single host running all the SCION services. In a typical deployment, these services would run a separate hosts and include multiple border routers.
  • This environment does not include a SCION-IP gateway.

If you would like to learn more and help develop SCION, consider using the development environment. See :ref:setting-up-the-development-environment.

If you would like to use SCION in a larger deployment, consider using the SCIONLab. For more information, see https://www.scionlab.org/.

Done. Mostly copied your text JC, thanks.

Copy link
Contributor

@matzf matzf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 12 of 17 files reviewed, 1 unresolved discussion (waiting on @jiceatscion, @johnstudarus, and @nicorusti)


doc/tutorials/deploy.rst line 4 at r3 (raw file):

Previously, jiceatscion wrote…

complete?

Done.

@matzf matzf changed the title Ietf118 hackathon doc: tutorial freestanding deploment Feb 13, 2024
@oncilla oncilla changed the title doc: tutorial freestanding deploment doc: tutorial freestanding deployment Feb 13, 2024
Copy link
Contributor

@jiceatscion jiceatscion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 4 of 5 files at r4, 1 of 1 files at r5, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @cdekater)

Copy link
Contributor

@jiceatscion jiceatscion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @cdekater)

@matzf matzf merged commit fce093e into scionproto:master Feb 13, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants