From 4c7fc0d1ed5feaec3ee202832555f9c156a2d66a Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 21 Jun 2024 15:23:00 +0200 Subject: [PATCH 01/18] Scalability of path discovery --- draft-dekater-scion-controlplane.md | 70 +++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 3a3b6ca..bbaf7da 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -127,6 +127,18 @@ informative: RFC6996: RFC9217: RFC9473: + BollRio-2000: + title: The diameter of a scale-free random graph + target: https://kam.mff.cuni.cz/~ksemweb/clanky/BollobasR-scale_free_random.pdf + author: + - + ins: B. Bollobás + name: Béla Bollobás + - + ins: O. Riordan + name: Oliver Riordan + + --- abstract @@ -1297,6 +1309,64 @@ In comparison to these time scales, clock offsets in the order of minutes are im Each administrator of a SCION control service is responsible for maintaining sufficient clock accuracy. No particular method is assumed by this specification. +## Path Discovery Time and Scalability + + +Unreachable goal: discover all policy compliant paths, ~immediately +Resource trade off: reduce propagation interval, reduce candidate set -> not all paths are found, takes some time. + +This resource "costs" for path discovery are communication overhead, processing and storage. Communication is transmitting the PCBs and occasionally obtaining the required PKI material. Processing cost is validating the signatures of the AS entries, signing new AS entries, and, to a lesser extent, evaluating the beaconing policies. +All of these depend on the the number and length of the discovered path segments, that is, on the total number of AS entries of the discovered path segments. + +Interesting metrics for scalability and speed of path discovery are the time until all discoverable path segments have been discovered after a "cold boot" and the time until a new link is usable. + +Generally, the time until a specific PCB is built depends on its length and the propagation interval. +At each AS, the PCB will be processed and propagated at the subsequent propagation interval. +With a propagation interval T, the mean time until the PCB is propgated in one AS is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). + + + +For more specific observations, we distinguish between intra- and inter-ISD beaconing. + +### Intra-ISD Beaconing +In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child links, from core to leaf ASes. Each AS discovers path segments from itself to the core ASes of its ISD. + +Typically, this directed, acyclic graph is narrow at the top, widens towards the leafs, and is relatively shallow; intermediate provider ASes have a large number of children, while they only have a small number of parents. The chain of intermediate providers from a leaf AS to a core AS is typically not long (e.g. local, regional, national provider, then core). + +Each AS potentially receives PCBs for all down paths between core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. +Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily managable even for highly connected ASes. +To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughtly 2.5MB/s, and, processing 10000 signature verifications per second. +If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link, that is at most 50000 signatures per propagation interval. +The total bandwidth for the propagation of these PCBs for all 1000 child links would, again very roughly, be around 25MB/s. +All of these are managable with even modest consumer hardware. + + +On a cold start of the network, path segments to each AS are discovered after a number of propagation steps proportional to the longest path. As mentioned, the longest path is typically not long. With a 5 second propagation period and a generous longest path of length 10, all path segments are discovered after 25 seconds on average. + +When a new parent-child link is added to the network, the parent AS will propagate the available PCBs in the next propagation interval. If the AS on the child side of the new link is a leaf AS, path discovery is thus complete after one single propagation interval. Otherwise, child ASes at distance D below the new link, learn of the new link after D further propagation steps. + +### Inter-ISD Beaconing +In the inter-ISD core beaconing, PCBs are propagated omnidirectionally along core links. Each AS discovers path segments from itself to any other core AS. +The number of distinct paths through the core network is typically very large. To keep the overhead managable, at most 5 path segments to every destination AS are discovered, and the propagation frequency is slower than in the intra-ISD beaconing. + +Without making strong assumptions on the topology of the core network, we can assume that shortest paths through real world, internet-like networks are relatively short; for example, the Barabási-Albert random graph model predicts a diameter below log(N)/log(log(N)) for a network with N nodes {{BollRio-2000}}. +We cannot assume that the selected PCBs are strictly shortest paths through the network, but it's reasonable to assume that they will not be very much longer than the shortest paths either. It seems safe to assume that the PCBs are not longer than 10 hops on average for realistic networks. +<-- TODO find something about _average_ shortest path. Probably can say something smaller than 10 --> + +With N the number of participating core ASes, an AS receives up to 5 * N PCBs per propagation interval per core link interface. +For highly connected ASes, the number of PCBs received thus becomes rather large. In a network of 1000 ASes, an AS with 200 core links receives up to one million PCBs per propagation interval. +This corresponds to roughly 150 thousand signature validations per second. This is just about the achievable throughput on a single core of a present day small server or desktop machine. +In terms of bandwidth, this corresponds to very roughly 40MB/s. + + +On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. With a generous diameter of 10, this corresponds to roughtly 5 minutes on average. + +When a new link is added to the network, it will be available to connect two ASes at distances from the link D1 and D2 from the link, respectively, after a mean time (D1+D2)*T/2. + + # Registration of Path Segments {#path-segment-reg} **Path registration** is the process where an AS transforms selected PCBs into path segments, and adds these segments to the relevant path databases, thus making them available to other ASes. From 760c0e257cc2242bf59c48609c388d53694d4f9a Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Mon, 24 Jun 2024 17:20:55 +0200 Subject: [PATCH 02/18] scalability analysis polish --- draft-dekater-scion-controlplane.md | 29 ++++++++++++----------------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index bbaf7da..38fb6dd 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1311,25 +1311,20 @@ Each administrator of a SCION control service is responsible for maintaining suf ## Path Discovery Time and Scalability - -Unreachable goal: discover all policy compliant paths, ~immediately -Resource trade off: reduce propagation interval, reduce candidate set -> not all paths are found, takes some time. +The path discovery mechanism balances of the number of discovered paths and the time it takes to discover them versus resource overhead of the discovery. -This resource "costs" for path discovery are communication overhead, processing and storage. Communication is transmitting the PCBs and occasionally obtaining the required PKI material. Processing cost is validating the signatures of the AS entries, signing new AS entries, and, to a lesser extent, evaluating the beaconing policies. +The resource costs for path discovery are communication overhead, processing and storage. Communication is transmitting the PCBs and occasionally obtaining the required PKI material. Processing cost is validating the signatures of the AS entries, signing new AS entries, and, to a lesser extent, evaluating the beaconing policies. Storage is both the temporary storage of PCBs before the next propagation interval, and the storage of complete discovered path segments. All of these depend on the the number and length of the discovered path segments, that is, on the total number of AS entries of the discovered path segments. -Interesting metrics for scalability and speed of path discovery are the time until all discoverable path segments have been discovered after a "cold boot" and the time until a new link is usable. - +Interesting metrics for scalability and speed of path discovery are the time until all discoverable path segments have been discovered after a "cold boot", and the time until new link is usable. Generally, the time until a specific PCB is built depends on its length and the propagation interval. -At each AS, the PCB will be processed and propagated at the subsequent propagation interval. -With a propagation interval T, the mean time until the PCB is propgated in one AS is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). +At each AS, the PCB will be processed and propagated at the subsequent propagation event. As propagation events are not synchronized between different ASes, a PCB arrives at a random point in time during the interval and is buffered before potentially being propagated. +With a propagation interval T, the mean time until the PCB is propgated in one AS therefore is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). - +Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, end points route around the failed link by switching to different paths in the data plane. For more specific observations, we distinguish between intra- and inter-ISD beaconing. +As will become apparent, the inter-ISD beaconing results in excessive overhead with very large numbers of participating core ASes. The ideal topology for SCION is to keep the inter-ISD core network to a moderate size, to benefit from the divide-and-conquer partitioning of ASes into ISDs and the efficiency of the intra-ISD beaconing. ### Intra-ISD Beaconing In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child links, from core to leaf ASes. Each AS discovers path segments from itself to the core ASes of its ISD. @@ -1338,19 +1333,19 @@ Typically, this directed, acyclic graph is narrow at the top, widens towards the Each AS potentially receives PCBs for all down paths between core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily managable even for highly connected ASes. + To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughtly 2.5MB/s, and, processing 10000 signature verifications per second. -If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link, that is at most 50000 signatures per propagation interval. +If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link, that is at most 50000 signatures per propagation event. The total bandwidth for the propagation of these PCBs for all 1000 child links would, again very roughly, be around 25MB/s. All of these are managable with even modest consumer hardware. - On a cold start of the network, path segments to each AS are discovered after a number of propagation steps proportional to the longest path. As mentioned, the longest path is typically not long. With a 5 second propagation period and a generous longest path of length 10, all path segments are discovered after 25 seconds on average. -When a new parent-child link is added to the network, the parent AS will propagate the available PCBs in the next propagation interval. If the AS on the child side of the new link is a leaf AS, path discovery is thus complete after one single propagation interval. Otherwise, child ASes at distance D below the new link, learn of the new link after D further propagation steps. +When a new parent-child link is added to the network, the parent AS will propagate the available PCBs in the next propagation event. If the AS on the child side of the new link is a leaf AS, path discovery is thus complete after one single propagation interval. Otherwise, child ASes at distance D below the new link, learn of the new link after D further propagation steps. ### Inter-ISD Beaconing In the inter-ISD core beaconing, PCBs are propagated omnidirectionally along core links. Each AS discovers path segments from itself to any other core AS. -The number of distinct paths through the core network is typically very large. To keep the overhead managable, at most 5 path segments to every destination AS are discovered, and the propagation frequency is slower than in the intra-ISD beaconing. +The number of distinct paths through the core network is typically very large. To keep the overhead managable, at most 5 path segments to every destination AS are discovered, and the propagation frequency is slower than in the intra-ISD beaconing (at least 60 seconds between propagation events). Without making strong assumptions on the topology of the core network, we can assume that shortest paths through real world, internet-like networks are relatively short; for example, the Barabási-Albert random graph model predicts a diameter below log(N)/log(log(N)) for a network with N nodes {{BollRio-2000}}. We cannot assume that the selected PCBs are strictly shortest paths through the network, but it's reasonable to assume that they will not be very much longer than the shortest paths either. It seems safe to assume that the PCBs are not longer than 10 hops on average for realistic networks. @@ -1358,7 +1353,7 @@ We cannot assume that the selected PCBs are strictly shortest paths through the With N the number of participating core ASes, an AS receives up to 5 * N PCBs per propagation interval per core link interface. For highly connected ASes, the number of PCBs received thus becomes rather large. In a network of 1000 ASes, an AS with 200 core links receives up to one million PCBs per propagation interval. -This corresponds to roughly 150 thousand signature validations per second. This is just about the achievable throughput on a single core of a present day small server or desktop machine. +With the shortest propagation interval of 60 seconds, this corresponds to roughly 166 thousand signature validations per second. This is just about the achievable throughput on a single core of a present day small server or desktop machine. In terms of bandwidth, this corresponds to very roughly 40MB/s. From 17ad81bffb96efa32ae86bcbb6246a6fbe775225 Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Mon, 24 Jun 2024 17:59:25 +0200 Subject: [PATCH 03/18] scalability: use different numbers --- draft-dekater-scion-controlplane.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 38fb6dd..fa028a5 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1347,17 +1347,16 @@ When a new parent-child link is added to the network, the parent AS will propaga In the inter-ISD core beaconing, PCBs are propagated omnidirectionally along core links. Each AS discovers path segments from itself to any other core AS. The number of distinct paths through the core network is typically very large. To keep the overhead managable, at most 5 path segments to every destination AS are discovered, and the propagation frequency is slower than in the intra-ISD beaconing (at least 60 seconds between propagation events). -Without making strong assumptions on the topology of the core network, we can assume that shortest paths through real world, internet-like networks are relatively short; for example, the Barabási-Albert random graph model predicts a diameter below log(N)/log(log(N)) for a network with N nodes {{BollRio-2000}}. -We cannot assume that the selected PCBs are strictly shortest paths through the network, but it's reasonable to assume that they will not be very much longer than the shortest paths either. It seems safe to assume that the PCBs are not longer than 10 hops on average for realistic networks. -<-- TODO find something about _average_ shortest path. Probably can say something smaller than 10 --> +Without making strong assumptions on the topology of the core network, we can assume that shortest paths through real world, internet-like networks are relatively short; for example, the Barabási-Albert random graph model predicts a diameter of log(N)/log(log(N)) for a network with N nodes {{BollRio-2000}}. The average distance scales in the same way. +We cannot assume that the selected PCBs are strictly shortest paths through the network, but it's reasonable to assume that they will not be very much longer than the shortest paths either. With N the number of participating core ASes, an AS receives up to 5 * N PCBs per propagation interval per core link interface. -For highly connected ASes, the number of PCBs received thus becomes rather large. In a network of 1000 ASes, an AS with 200 core links receives up to one million PCBs per propagation interval. -With the shortest propagation interval of 60 seconds, this corresponds to roughly 166 thousand signature validations per second. This is just about the achievable throughput on a single core of a present day small server or desktop machine. -In terms of bandwidth, this corresponds to very roughly 40MB/s. +For highly connected ASes, the number of PCBs received thus becomes rather large. In a network of 1000 ASes, a highly connected AS with 300 core links receives up to 1.5 million PCBs per propagation interval. +Assuming an average PCB length of 6 and the shortest propagation interval of 60 seconds, this corresponds to roughly 150 thousand signature validations per second. This throughput can be achieved on a single core of a present day small server or desktop machine. +In terms of bandwidth, this corresponds to very roughly 38MB/s. -On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. With a generous diameter of 10, this corresponds to roughtly 5 minutes on average. +On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. Assuming a network diameter of 6, this corresponds to roughtly 3 minutes on average. When a new link is added to the network, it will be available to connect two ASes at distances from the link D1 and D2 from the link, respectively, after a mean time (D1+D2)*T/2. From 3d16011de5e8937ee44b854b5d934af7f002fa61 Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Mon, 1 Jul 2024 08:28:13 +0200 Subject: [PATCH 04/18] clarify limit of no PCBs per link in example calculation --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index fa028a5..ad601ab 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1335,7 +1335,7 @@ Each AS potentially receives PCBs for all down paths between core to itself. Whi Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily managable even for highly connected ASes. To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughtly 2.5MB/s, and, processing 10000 signature verifications per second. -If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link, that is at most 50000 signatures per propagation event. +If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link (at most 50 per link), that is at most 50000 signatures per propagation event. The total bandwidth for the propagation of these PCBs for all 1000 child links would, again very roughly, be around 25MB/s. All of these are managable with even modest consumer hardware. From 011ef7c0e79e718c4eea4ba1580e19a547680dd7 Mon Sep 17 00:00:00 2001 From: Nicola Rustignoli Date: Wed, 3 Jul 2024 14:59:00 +0200 Subject: [PATCH 05/18] typos --- draft-dekater-scion-controlplane.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index ad601ab..a6ea042 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1319,7 +1319,7 @@ All of these depend on the the number and length of the discovered path segments Interesting metrics for scalability and speed of path discovery are the time until all discoverable path segments have been discovered after a "cold boot", and the time until new link is usable. Generally, the time until a specific PCB is built depends on its length and the propagation interval. At each AS, the PCB will be processed and propagated at the subsequent propagation event. As propagation events are not synchronized between different ASes, a PCB arrives at a random point in time during the interval and is buffered before potentially being propagated. -With a propagation interval T, the mean time until the PCB is propgated in one AS therefore is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). +With a propagation interval T, the mean time until the PCB is propagated in one AS therefore is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, end points route around the failed link by switching to different paths in the data plane. @@ -1332,12 +1332,12 @@ In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child lin Typically, this directed, acyclic graph is narrow at the top, widens towards the leafs, and is relatively shallow; intermediate provider ASes have a large number of children, while they only have a small number of parents. The chain of intermediate providers from a leaf AS to a core AS is typically not long (e.g. local, regional, national provider, then core). Each AS potentially receives PCBs for all down paths between core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. -Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily managable even for highly connected ASes. +Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily manageable even for highly connected ASes. -To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughtly 2.5MB/s, and, processing 10000 signature verifications per second. +To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughly 2.5MB/s, and, processing 10000 signature verifications per second. If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link (at most 50 per link), that is at most 50000 signatures per propagation event. The total bandwidth for the propagation of these PCBs for all 1000 child links would, again very roughly, be around 25MB/s. -All of these are managable with even modest consumer hardware. +All of these are manageable with even modest consumer hardware. On a cold start of the network, path segments to each AS are discovered after a number of propagation steps proportional to the longest path. As mentioned, the longest path is typically not long. With a 5 second propagation period and a generous longest path of length 10, all path segments are discovered after 25 seconds on average. @@ -1345,7 +1345,7 @@ When a new parent-child link is added to the network, the parent AS will propaga ### Inter-ISD Beaconing In the inter-ISD core beaconing, PCBs are propagated omnidirectionally along core links. Each AS discovers path segments from itself to any other core AS. -The number of distinct paths through the core network is typically very large. To keep the overhead managable, at most 5 path segments to every destination AS are discovered, and the propagation frequency is slower than in the intra-ISD beaconing (at least 60 seconds between propagation events). +The number of distinct paths through the core network is typically very large. To keep the overhead manageable, at most 5 path segments to every destination AS are discovered, and the propagation frequency is slower than in the intra-ISD beaconing (at least 60 seconds between propagation events). Without making strong assumptions on the topology of the core network, we can assume that shortest paths through real world, internet-like networks are relatively short; for example, the Barabási-Albert random graph model predicts a diameter of log(N)/log(log(N)) for a network with N nodes {{BollRio-2000}}. The average distance scales in the same way. We cannot assume that the selected PCBs are strictly shortest paths through the network, but it's reasonable to assume that they will not be very much longer than the shortest paths either. @@ -1356,7 +1356,7 @@ Assuming an average PCB length of 6 and the shortest propagation interval of 60 In terms of bandwidth, this corresponds to very roughly 38MB/s. -On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. Assuming a network diameter of 6, this corresponds to roughtly 3 minutes on average. +On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. Assuming a network diameter of 6, this corresponds to roughly 3 minutes on average. When a new link is added to the network, it will be available to connect two ASes at distances from the link D1 and D2 from the link, respectively, after a mean time (D1+D2)*T/2. From 5f7a89eee6523a732ba8fef92afbc0297c772ef9 Mon Sep 17 00:00:00 2001 From: Nicola Rustignoli Date: Thu, 4 Jul 2024 12:36:05 +0200 Subject: [PATCH 06/18] add reference to scalability section in 2.3.1.1. Storing and Selecting Candidate PCBs --- draft-dekater-scion-controlplane.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index a6ea042..945e0f9 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1135,11 +1135,11 @@ PCBs are propagated in batches to each connected downstream AS at a fixed freque - The *best PCBs set size* SHOULD be at most "50" (PCBs) for intra-ISD beaconing and at most "5" (PCBs) for core beaconing. -**Note:** Depending on the selection criteria, it may be necessary to keep more candidate PCBs than the *best PCBs set size* in the beacon store, to be able to determine the best set of PCBs. If this is the case, an AS should have a suitable pre-selection of candidate PCBs in place, in order to keep the beacon store capacity limited. +Depending on the selection criteria, it may be necessary to keep more candidate PCBs than the *best PCBs set size* in the beacon store, to be able to determine the best set of PCBs. If this is the case, an AS should have a suitable pre-selection of candidate PCBs in place, in order to keep the beacon store capacity limited. - The *propagation interval* SHOULD be at least "5" (seconds) for intra-ISD beaconing and at least "60" (seconds) for core beaconing. -**Note:** Note that during bootstrapping and if the AS obtains a PCB containing a previously unknown path, the AS SHOULD forward the PCB immediately, to ensure quick connectivity establishment. +The scalability implications of such parameters are further discussed in [](#scalability). Note that during bootstrapping and if the AS obtains a PCB containing a previously unknown path, the AS SHOULD forward the PCB immediately, to ensure quick connectivity establishment. #### Selection Policy Example @@ -1309,7 +1309,7 @@ In comparison to these time scales, clock offsets in the order of minutes are im Each administrator of a SCION control service is responsible for maintaining sufficient clock accuracy. No particular method is assumed by this specification. -## Path Discovery Time and Scalability +## Path Discovery Time and Scalability {#scalability} The path discovery mechanism balances of the number of discovered paths and the time it takes to discover them versus resource overhead of the discovery. From aa890b107ec07fde622cd3a3ebc218c7aa841d16 Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 12:45:56 +0200 Subject: [PATCH 07/18] Update draft-dekater-scion-controlplane.md Co-authored-by: Nicola Rustignoli --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 945e0f9..0b4abaf 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1311,7 +1311,7 @@ Each administrator of a SCION control service is responsible for maintaining suf ## Path Discovery Time and Scalability {#scalability} -The path discovery mechanism balances of the number of discovered paths and the time it takes to discover them versus resource overhead of the discovery. +The path discovery mechanism balances the number of discovered paths and the time it takes to discover them versus resource overhead of the discovery. The resource costs for path discovery are communication overhead, processing and storage. Communication is transmitting the PCBs and occasionally obtaining the required PKI material. Processing cost is validating the signatures of the AS entries, signing new AS entries, and, to a lesser extent, evaluating the beaconing policies. Storage is both the temporary storage of PCBs before the next propagation interval, and the storage of complete discovered path segments. All of these depend on the the number and length of the discovered path segments, that is, on the total number of AS entries of the discovered path segments. From 8e7ca5264a5c6ab0c220d94550d329aaa8cfdddd Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 12:46:21 +0200 Subject: [PATCH 08/18] Update draft-dekater-scion-controlplane.md Co-authored-by: Nicola Rustignoli --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 0b4abaf..4c7608b 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1319,7 +1319,7 @@ All of these depend on the the number and length of the discovered path segments Interesting metrics for scalability and speed of path discovery are the time until all discoverable path segments have been discovered after a "cold boot", and the time until new link is usable. Generally, the time until a specific PCB is built depends on its length and the propagation interval. At each AS, the PCB will be processed and propagated at the subsequent propagation event. As propagation events are not synchronized between different ASes, a PCB arrives at a random point in time during the interval and is buffered before potentially being propagated. -With a propagation interval T, the mean time until the PCB is propagated in one AS therefore is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). +With a propagation interval T at each AS, the mean time until the PCB is propagated in one AS therefore is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, end points route around the failed link by switching to different paths in the data plane. From 1a6084ff770c618c0223e2281c70cf220c9ac35b Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 12:52:39 +0200 Subject: [PATCH 09/18] fixup down path -> down-path segment and phrasing --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 4c7608b..3cf3d62 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1331,7 +1331,7 @@ In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child lin Typically, this directed, acyclic graph is narrow at the top, widens towards the leafs, and is relatively shallow; intermediate provider ASes have a large number of children, while they only have a small number of parents. The chain of intermediate providers from a leaf AS to a core AS is typically not long (e.g. local, regional, national provider, then core). -Each AS potentially receives PCBs for all down paths between core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. +Each AS potentially receives PCBs for all down-path segments from the core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily manageable even for highly connected ASes. To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughly 2.5MB/s, and, processing 10000 signature verifications per second. From 0c05132608dab81110ce733a924c7e58b59f5e4c Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 13:13:50 +0200 Subject: [PATCH 10/18] Update draft-dekater-scion-controlplane.md Co-authored-by: Nicola Rustignoli --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 3cf3d62..2190c89 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1334,7 +1334,7 @@ Typically, this directed, acyclic graph is narrow at the top, widens towards the Each AS potentially receives PCBs for all down-path segments from the core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily manageable even for highly connected ASes. -To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, allowed propagation period of 5 seconds, this corresponds to a total bandwidth of very roughly 2.5MB/s, and, processing 10000 signature verifications per second. +To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, recommended propagation interval of 5 seconds, this corresponds to an average bandwidth of around 2.5MB/s, and processing 10000 signature verifications per second. If the same AS has 1000 child links, the propagation of the beacons will require signing one new AS entry for each of the propagated PCBs for each link (at most 50 per link), that is at most 50000 signatures per propagation event. The total bandwidth for the propagation of these PCBs for all 1000 child links would, again very roughly, be around 25MB/s. All of these are manageable with even modest consumer hardware. From 7bb67e3a9b37131648c13a4a236c6c139ffd9baf Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 13:23:53 +0200 Subject: [PATCH 11/18] Update draft-dekater-scion-controlplane.md Co-authored-by: Nicola Rustignoli --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 2190c89..7205444 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1341,7 +1341,7 @@ All of these are manageable with even modest consumer hardware. On a cold start of the network, path segments to each AS are discovered after a number of propagation steps proportional to the longest path. As mentioned, the longest path is typically not long. With a 5 second propagation period and a generous longest path of length 10, all path segments are discovered after 25 seconds on average. -When a new parent-child link is added to the network, the parent AS will propagate the available PCBs in the next propagation event. If the AS on the child side of the new link is a leaf AS, path discovery is thus complete after one single propagation interval. Otherwise, child ASes at distance D below the new link, learn of the new link after D further propagation steps. +When a new parent-child link is added to the network, the parent AS will propagate the available PCBs in the next propagation event. If the AS on the child side of the new link is a leaf AS, path discovery is thus complete after one single propagation interval. Otherwise, child ASes at distance D below the new link, learn of the new link after D further propagation intervals. ### Inter-ISD Beaconing In the inter-ISD core beaconing, PCBs are propagated omnidirectionally along core links. Each AS discovers path segments from itself to any other core AS. From 829280506e8c2713418f46d1ab625edd2ef5d848 Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 13:30:43 +0200 Subject: [PATCH 12/18] refer to *recommended* best PCBs set size 50 --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 7205444..487cec7 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1331,7 +1331,7 @@ In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child lin Typically, this directed, acyclic graph is narrow at the top, widens towards the leafs, and is relatively shallow; intermediate provider ASes have a large number of children, while they only have a small number of parents. The chain of intermediate providers from a leaf AS to a core AS is typically not long (e.g. local, regional, national provider, then core). -Each AS potentially receives PCBs for all down-path segments from the core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the limit value of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. +Each AS potentially receives PCBs for all down-path segments from the core to itself. While the number of distinct provider chains to the core is typically moderate, the multiplicity of links between provider ASes has multiplicative effect on the number of PCBs. Once this number grows above the maximum recommended best PCBs set size of 50, ASes trim the set of PCBs propagated. As the choice is among different ways to transit the local AS, operators are well equipped to choose among this set of PCBs. Ultimately, the number of PCBs received by an AS per propagation interval remains bounded by 50 for each parent link of an AS, and at most 50 PCBs per child link are propagated. The length of these PCBs, and thus the number of AS entries to be processed and stored, is expected to be moderate and not grow considerably with network size. The total resource overhead for beacon propagation is easily manageable even for highly connected ASes. To illustrate this with some numbers, an AS with a rather large number of 100 parent links receives at most 5000 PCBs during a propagation interval. Assuming a generous average length of 10 AS entries for these PCBs, this corresponds to 50000 AS entries. Due to the variable length fields in AS entries, the sizes for storage and transmission cannot be predicted exactly, and we'll assume an average of 250 bytes per AS entry. At the shortest, and thus chattiest, recommended propagation interval of 5 seconds, this corresponds to an average bandwidth of around 2.5MB/s, and processing 10000 signature verifications per second. From d8bff0ff77cd4c8e707af3bf40d4bf7b44fd5be3 Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 13:55:50 +0200 Subject: [PATCH 13/18] scaling mention distributed CS --- draft-dekater-scion-controlplane.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 487cec7..abcb26e 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1352,8 +1352,9 @@ We cannot assume that the selected PCBs are strictly shortest paths through the With N the number of participating core ASes, an AS receives up to 5 * N PCBs per propagation interval per core link interface. For highly connected ASes, the number of PCBs received thus becomes rather large. In a network of 1000 ASes, a highly connected AS with 300 core links receives up to 1.5 million PCBs per propagation interval. -Assuming an average PCB length of 6 and the shortest propagation interval of 60 seconds, this corresponds to roughly 150 thousand signature validations per second. This throughput can be achieved on a single core of a present day small server or desktop machine. -In terms of bandwidth, this corresponds to very roughly 38MB/s. +Assuming an average PCB length of 6 and the shortest propagation interval of 60 seconds, this corresponds to roughly 150 thousand signature validations per second. In terms of bandwidth, this corresponds to very roughly 38MB/s. +All of these are managable on a present day small server or desktop machine. +For much larger, more highly connected ASes, the path-discovery tasks of the control service can be distributed over many instances in order to increase the PCB throughput. On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. Assuming a network diameter of 6, this corresponds to roughly 3 minutes on average. From c6f8e1f922525d035e1a3bdbb9bde24013cd9874 Mon Sep 17 00:00:00 2001 From: Matthias Frei Date: Fri, 5 Jul 2024 13:57:57 +0200 Subject: [PATCH 14/18] split unrelated lines --- draft-dekater-scion-controlplane.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index abcb26e..d4716fc 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1139,7 +1139,9 @@ Depending on the selection criteria, it may be necessary to keep more candidate - The *propagation interval* SHOULD be at least "5" (seconds) for intra-ISD beaconing and at least "60" (seconds) for core beaconing. -The scalability implications of such parameters are further discussed in [](#scalability). Note that during bootstrapping and if the AS obtains a PCB containing a previously unknown path, the AS SHOULD forward the PCB immediately, to ensure quick connectivity establishment. +The scalability implications of such parameters are further discussed in [](#scalability). + +Note that during bootstrapping and if the AS obtains a PCB containing a previously unknown path, the AS SHOULD forward the PCB immediately, to ensure quick connectivity establishment. #### Selection Policy Example From 966445bca70e1e55436cd5abb004d779e2ae0e07 Mon Sep 17 00:00:00 2001 From: Nicola Rustignoli Date: Sun, 7 Jul 2024 19:47:43 +0200 Subject: [PATCH 15/18] Remove note about immediate PCB propagation, fix typos --- draft-dekater-scion-controlplane.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index d4716fc..ebbe6e9 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1141,7 +1141,6 @@ Depending on the selection criteria, it may be necessary to keep more candidate The scalability implications of such parameters are further discussed in [](#scalability). -Note that during bootstrapping and if the AS obtains a PCB containing a previously unknown path, the AS SHOULD forward the PCB immediately, to ensure quick connectivity establishment. #### Selection Policy Example @@ -1323,7 +1322,7 @@ Generally, the time until a specific PCB is built depends on its length and the At each AS, the PCB will be processed and propagated at the subsequent propagation event. As propagation events are not synchronized between different ASes, a PCB arrives at a random point in time during the interval and is buffered before potentially being propagated. With a propagation interval T at each AS, the mean time until the PCB is propagated in one AS therefore is T / 2 and the mean total time for the propagation steps of a PCB of length L is L * T / 2 (with a variance of L * T^2 / 12). -Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, end points route around the failed link by switching to different paths in the data plane. +Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, endpoints route around the failed link by switching to different paths in the data plane. For more specific observations, we distinguish between intra- and inter-ISD beaconing. As will become apparent, the inter-ISD beaconing results in excessive overhead with very large numbers of participating core ASes. The ideal topology for SCION is to keep the inter-ISD core network to a moderate size, to benefit from the divide-and-conquer partitioning of ASes into ISDs and the efficiency of the intra-ISD beaconing. @@ -1355,7 +1354,7 @@ We cannot assume that the selected PCBs are strictly shortest paths through the With N the number of participating core ASes, an AS receives up to 5 * N PCBs per propagation interval per core link interface. For highly connected ASes, the number of PCBs received thus becomes rather large. In a network of 1000 ASes, a highly connected AS with 300 core links receives up to 1.5 million PCBs per propagation interval. Assuming an average PCB length of 6 and the shortest propagation interval of 60 seconds, this corresponds to roughly 150 thousand signature validations per second. In terms of bandwidth, this corresponds to very roughly 38MB/s. -All of these are managable on a present day small server or desktop machine. +All of these are manageable on a present day small server or desktop machine. For much larger, more highly connected ASes, the path-discovery tasks of the control service can be distributed over many instances in order to increase the PCB throughput. From 2b2b7683c0fbb23e9c06f8f05ca7a03374e91824 Mon Sep 17 00:00:00 2001 From: Nicola Rustignoli Date: Sun, 7 Jul 2024 23:03:11 +0200 Subject: [PATCH 16/18] scalability: implement feedback Tilman --- draft-dekater-scion-controlplane.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index ebbe6e9..6fec103 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1301,10 +1301,9 @@ A PCB originated by a given control service is validated by all the control serv * A fast clock at origination or a slow clock at reception will yield a lengthened expiration time for hops, and possibly an origination time in the future. * A slow clock at origination or a fast clock at reception will yield a shortened expiration time for hops, and possibly an expiration time in the past. -This bias comes in addition to a structural delay: PCBs are propagated at a configurable interval (typically, one minute). As a result of this and the way they are iteratively constructed, PCBs with N hops may be validated up to N intervals (so typically N minutes) after origination. This creates a constraint on the expiration of hops. Hops of the minimal expiration time (337.5 seconds - see [](#hopfield)) would render useless any PCB describing a path longer than 5 hops. For this reason, it is unadvisable to create hops with a short expiration time. The norm is 6 hours. +This bias comes in addition to a structural delay: PCBs are propagated at a configurable interval (typically, one minute). As a result of this and the way they are iteratively constructed, PCBs with N hops may be validated up to N intervals (so maximally N minutes) after origination. This creates a constraint on the expiration of hops. Hops of the minimal expiration time (337.5 seconds - see [](#hopfield)) would render useless any PCB describing a path longer than 5 hops. For this reason, it is unadvisable to create hops with a short expiration time, that should be around 6 hours. The control service and its clients authenticate each-other according to their respective AS's certificate. Path segments are authenticated based on the certificates of the ASes that they refer to. The expiration of a SCION AS certificate typically ranges from 3h to 5 years. - In comparison to these time scales, clock offsets in the order of minutes are immaterial. Each administrator of a SCION control service is responsible for maintaining sufficient clock accuracy. No particular method is assumed by this specification. @@ -1324,8 +1323,7 @@ With a propagation interval T at each AS, the mean time until the PCB is propaga Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, endpoints route around the failed link by switching to different paths in the data plane. -For more specific observations, we distinguish between intra- and inter-ISD beaconing. -As will become apparent, the inter-ISD beaconing results in excessive overhead with very large numbers of participating core ASes. The ideal topology for SCION is to keep the inter-ISD core network to a moderate size, to benefit from the divide-and-conquer partitioning of ASes into ISDs and the efficiency of the intra-ISD beaconing. +To achieve scalability in its routing process, SCION uses a divide-and-conquer approach, partitioning ASes into ISDs. In order to benefit from this, an ideal topology SCION should keep the inter-ISD core network to a moderate size. For more specific observations, we distinguish between intra- and inter-ISD beaconing. ### Intra-ISD Beaconing In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child links, from core to leaf ASes. Each AS discovers path segments from itself to the core ASes of its ISD. @@ -1360,7 +1358,7 @@ For much larger, more highly connected ASes, the path-discovery tasks of the con On a cold start of the network, full connectivity is obtained after a number of propagation steps corresponding to the diameter of the network. Assuming a network diameter of 6, this corresponds to roughly 3 minutes on average. -When a new link is added to the network, it will be available to connect two ASes at distances from the link D1 and D2 from the link, respectively, after a mean time (D1+D2)*T/2. +When a new link is added to the network, it will be available to connect two ASes at distances D1 and D2 from the link, respectively, after a mean time (D1+D2)*T/2. # Registration of Path Segments {#path-segment-reg} From 4e2e591454a653a64a7d27b1ddb94dd55e32c1fb Mon Sep 17 00:00:00 2001 From: Nicola Rustignoli Date: Sun, 7 Jul 2024 23:07:59 +0200 Subject: [PATCH 17/18] fix lint --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index 6fec103..033b4f0 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1323,7 +1323,7 @@ With a propagation interval T at each AS, the mean time until the PCB is propaga Note that link removal is not part of path discovery in SCION. For scheduled removal of links, operators let path segments expire. On link failures, endpoints route around the failed link by switching to different paths in the data plane. -To achieve scalability in its routing process, SCION uses a divide-and-conquer approach, partitioning ASes into ISDs. In order to benefit from this, an ideal topology SCION should keep the inter-ISD core network to a moderate size. For more specific observations, we distinguish between intra- and inter-ISD beaconing. +To achieve scalability in its routing process, SCION uses a divide-and-conquer approach, partitioning ASes into ISDs. In order to benefit from this, an ideal topology SCION should keep the inter-ISD core network to a moderate size. For more specific observations, we distinguish between intra- and inter-ISD beaconing. ### Intra-ISD Beaconing In the intra-ISD beaconing, PCBs are propagated top-down, along parent-child links, from core to leaf ASes. Each AS discovers path segments from itself to the core ASes of its ISD. From 8b8d3dd4313fd3a427379cfb7311fb82105bcb6f Mon Sep 17 00:00:00 2001 From: Nicola Rustignoli Date: Mon, 8 Jul 2024 17:05:27 +0200 Subject: [PATCH 18/18] mention propagation time is around one minute --- draft-dekater-scion-controlplane.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/draft-dekater-scion-controlplane.md b/draft-dekater-scion-controlplane.md index e206230..66f3154 100644 --- a/draft-dekater-scion-controlplane.md +++ b/draft-dekater-scion-controlplane.md @@ -1301,7 +1301,7 @@ A PCB originated by a given control service is validated by all the control serv * A fast clock at origination or a slow clock at reception will yield a lengthened expiration time for hops, and possibly an origination time in the future. * A slow clock at origination or a fast clock at reception will yield a shortened expiration time for hops, and possibly an expiration time in the past. -This bias comes in addition to a structural delay: PCBs are propagated at a configurable interval (typically, one minute). As a result of this and the way they are iteratively constructed, PCBs with N hops may be validated up to N intervals (so maximally N minutes) after origination. This creates a constraint on the expiration of hops. Hops of the minimal expiration time (337.5 seconds - see [](#hopfield)) would render useless any PCB describing a path longer than 5 hops. For this reason, it is unadvisable to create hops with a short expiration time, that should be around 6 hours. +This bias comes in addition to a structural delay: PCBs are propagated at a configurable interval (typically, around one minute). As a result of this and the way they are iteratively constructed, PCBs with N hops may be validated up to N intervals (so maximally N minutes) after origination. This creates a constraint on the expiration of hops. Hops of the minimal expiration time (337.5 seconds - see [](#hopfield)) would render useless any PCB describing a path longer than 5 hops. For this reason, it is unadvisable to create hops with a short expiration time, that should be around 6 hours. The control service and its clients authenticate each-other according to their respective AS's certificate. Path segments are authenticated based on the certificates of the ASes that they refer to. The expiration of a SCION AS certificate typically ranges from 3h to 5 years. In comparison to these time scales, clock offsets in the order of minutes are immaterial.