Dev port disaggregation #320

anjan-keysight · 2023-05-25T04:08:48Z

Support to address issue #319 (Metrics response disaggregation based on Rx ports).

This is Rx port level disaggregation of metrics response required for multi Rx port device (and raw) traffic. Also addressed is support for specifying Rx/Tx frames ratio.

Example code snippet

config.Ports().Add().SetName("p1").SetLocation("localhost:5555")
config.Ports().Add().SetName("p2").SetLocation("localhost:5555")
config.Ports().Add().SetName("p3").SetLocation("localhost:5555")

For control plane BGP sessions configured on P1, P2 and P3 with device names
p1d1peer1rrv4, p2d1peer1rrv4 and p3d1peer1rrv4 respectively.
Following is the data plane configuration

flow := config.Flows().Add()
flow.Metrics().SetEnable(true).PredefinedMetricTags().SetRxName(true)

We are expecting Tx and Rx packets to match in this example and hence the Tx Rx ratio is set to "one"

flow.Metrics().RxTxRatio().SetChoice("value")
flow.Duration().FixedPackets().SetPackets(1000)

flow.SetName("f1").TxRx().Device().
	SetTxNames([]string{"p1d1peer1rrv4"}).
	SetRxNames([]string{"p2d1peer1rrv4", "p3d1peer1rrv4"})

flowEth := flow.Packet().Add().Ethernet()
flowEth.Src().SetValue("00:00:01:01:01:01")
flowEth.Dst().SetChoice("auto")

flowIp := flow.Packet().Add().Ipv4()
flowIp.Src().SetValue("10.10.10.1")
flowIp.Dst().SetValues([]string{"20.20.20.1", "30.30.30.1"})

client.SetConfig(config)

Query for metrics results

client.GetFlowMetrics(nil, nil)

Flow      Frames Tx      Bytes Tx      Frames Rx     Bytes Rx      Tags
f1        1000           64000         1000          64000
f1                                     500           32000         rx_name: p2
f1                                     500           32000         rx_name: p3

Note: The “rx_name” would be in-built tag name for Rx ports and cannot be used for any egress tracking name.

…ffic-generator/models into dev-port-disaggregation

parthpower · 2023-05-30T17:22:37Z

@anjan-keysight

The response format has been captured here.

linked document is not accessible to public

anjan-keysight · 2023-05-31T04:40:01Z

@parthpower I have shared the document.

flow/metrics.yaml

apratimmukherjee · 2023-05-31T10:10:05Z

flow/metrics.yaml

@@ -30,6 +37,12 @@ components:
            Latency metrics.
          $ref: '#/components/schemas/Flow.Latency.Metrics'
          x-field-uid: 4
+        rx_per_port:
+          description: |-
+            Enables Rx port level disaggregation with metrics tag name set as "rx_port_name".


... metrics tag name automatically set by implementation as "rx_port_name" in the flow metric_response ( use exact attribute names from metric response )

Also , do we need updates in otg gnmi model to specify that response can be filtered for specific port by specifing the filter for rx_port_name: in this exact format . Otherwise, how will gnmi user know what filter to use or expect which has to be parsed to set expectation in tests when checking stats using gnmi ( or otg metrics)

For gNMI query path: “/flows/flow[name=f1]/”
expected gNMI response (for the metric response example in PR description):

"updates": [ { "Path": "flows/flow[name=f1]", "values": { "flows/flow": { "open-traffic-generator-flow:name": "f1", "open-traffic-generator-flow:state": { // Contains the aggregated per-flow stats "counters": { "in-pkts": "1000", "in-octets": "8000", "out-pkts": "1000", "out-octets": "8000" }, "in-frame-rate": "QKAAAA==", "name": "f1", "out-frame-rate": "QCAAAA==", "transmit": true }, "open-traffic-generator-flow:tag-metrics": { // Contains the disaggregated per-flow stats "tag-metric": [ { "name-value": "rx_name=p2", "state": { "counters": { "in-octets": "4000", "in-pkts": "500" }, "name-value": "rx_name=p2", "tags": [ { "tag-name": "rx_name", "tag-value": { "value-as-string": "p2", "value-type": "STRING" } } ] } }, { "name-value": "rx_name=p3", "state": { "counters": { "in-octets": "4000", "in-pkts": "500" }, "name-value": "rx_name=p3", "tags": [ { "tag-name": "rx_name", "tag-value": { "value-as-string": "p3", "value-type": "STRING" } } ] } } ] } } } } ]

Description modified to reflect Rx port name location in tagged response.

flow/metrics.yaml

bortok

I have a more general question. A device's ethernet can have a connection to a port, lag or vxlan tunnel. As a result, a flow that is received by multiple devices can come through all of these types of connections. Shall we disaggregate Rx metrics accordingly, per lag or vxlan when configured as such, instead of port-level disaggregation?

bortok · 2023-06-07T22:27:31Z

flow/metrics.yaml

@@ -30,6 +35,13 @@ components:
            Latency metrics.
          $ref: '#/components/schemas/Flow.Latency.Metrics'
          x-field-uid: 4
+        rx_per_port:


should we also have rx_per_lag and rx_per_vxlan? Or maybe if we want to support those in future, shall we do rx_per_connection?

Currently we do not have support for drill down stats for lag or vxlan in TE. So we will support ports level disaggregation for now. But the model is very extensible and when we have the underlying support for those, we can extend the model without any breaking change.

could you give an example of how the model would be extended? it would be good to see how that would be aligned with rx_per_port parameter. i'm thinking maybe we should use choice here and set it as port first and then extend to lag and vxlan later?

I have restructured this part, now all additional tracking will come under "predefined_metric_tags" and we can add new properties under this. Current property "rx_name" will apply for both raw port and lag.

bortok · 2023-06-07T22:32:13Z

flow/metrics.yaml

@@ -69,3 +81,44 @@ components:
              x-field-uid: 1
            cut_through:
              x-field-uid: 2
+    Flow.TxRxReplication:


Wouldn't Replication imply additional copies? Meaning, if we have Replication = 1 it means we expect to receive twice more packets that we sent? Maybe we could use TxRxRatio instead?

Agree, I have updated it to TxRxRatio.

should we change the reference as well from $ref: '#/components/schemas/Flow.TxRxReplication' to $ref: '#/components/schemas/Flow.TxRxRatio'?

flow/metrics.yaml

apratimmukherjee

One minor typo.

apratimmukherjee · 2023-06-13T02:28:40Z

flow/metrics.yaml

+          x-field-uid: 3
+    Flow.RxTxRatio.RxCount:
+      description: |-
+        This is for cases where one copy each Tx packet is received on all Rx ports and so the sum total of Rx packets


one copy of

PrasenjitAdhikary · 2023-06-16T06:18:00Z

flow/endpoint.yaml

+          type: array
+          items:
+            description: |-
+              The unique name of a port that is the intended receive port.


Should we correct 'port' to 'port or lag'?

Not seeing the updated change?

PrasenjitAdhikary · 2023-06-16T06:24:33Z

flow/metrics.yaml

+          x-field-uid: 2
+        value:
+          description: |-
+            Should be a positive, non-zero value. A custom integer value (>1) can be specified for


We might need to explain the scenario with the default value (i.e. == 1)

Added description for default value.

PrasenjitAdhikary

LGTM

jasdeep-hundal

I'm still thinking about GNMI outputs a bit here, but I'm wondering if we should just use the port name directly as the tag ("" vs. "rx_port_name=".) (It should be a unique name in the config, but I'm also not sure if there are any other future 'built-in' ones that could not be specified this way.)

Unrelated clarification question: These tags will be included in disaggregated GNMI stats in the same way/alongside the user-defined ones correct?

jasdeep-hundal · 2023-06-21T22:55:25Z

flow/metrics.yaml

+      properties:
+        rx_name:
+          description: |-
+            Enables Rx port or lag level disaggregation with predefined metrics tag name set as "rx_name".


Does this need an update from rx_name->rx_port_name?

Both otg and gnmi tag name is going to be rx_name (rx_port_name was the earlier proposal). The rx_name is generic tag name, referring to both port and lag as rx endpoints.
So if we have a flow configured from lag L1 to lag L2 and a separate port P5, the metrics response will look like:
F1: L1 => L2, P5

Flow Frames Tx Bytes Tx Frames Rx Bytes Rx Tags f1 1000 64000 1000 64000 f1 400 25600 rx_name: p5 f1 600 38400 rx_name: l2

There can be other built-in tag names in future, like "mem_name" (as an example), if we want to disaggregate based on lag member ports also.
And yes, these tag names will be alongside user defined ones.

jasdeep-hundal · 2023-06-21T22:58:16Z

flow/metrics.yaml

+          format: float
+          default: 1.0
+          x-field-uid: 3
+    Flow.RxTxRatio.RxCount:


In my current understanding, I'm wondering if we need this option. I'm not sure it's more convenient/explicit for test writer to use vs. just setting the port count explicitly.

User can use RxCount, so even if the rx port count changes in config, user does not have to update it.
But I leave it to you, if you feel otherwise.

jasdeep-hundal · 2023-06-21T23:04:03Z

flow/endpoint.yaml

+          type: array
+          items:
+            description: |-
+              The unique name of a port that is the intended receive port.


Not seeing the updated change?

anjan-keysight and others added 2 commits May 24, 2023 14:57

Support for rx port drill down stats

6c86bc9

Update auto generated content

116dd1e

anjan-keysight requested review from alakendu, ashutshkumr, apratimmukherjee and PrasenjitAdhikary May 25, 2023 04:08

anjan-keysight and others added 5 commits May 25, 2023 07:08

Metrics tag name description update

5e9cc3d

Merge branch 'dev-port-disaggregation' of https://github.com/open-tra…

c629ca4

…ffic-generator/models into dev-port-disaggregation

Update auto generated content

f705555

Added rx tx ratio for loss calculation

9f662fd

Update auto generated content

2ea6fdc

anjan-keysight and others added 4 commits May 31, 2023 08:06

Minor update

bd3ffbf

Update auto generated content

79d594b

Modified default value

4705302

Update auto generated content

5d4988f

apratimmukherjee reviewed May 31, 2023

View reviewed changes

anjan-keysight requested review from bortok, winstonliu1111, ankur-sheth and hashwini-keysight June 7, 2023 10:39

anjan-keysight and others added 2 commits June 7, 2023 11:54

Replication factor made into enum

2931551

Update auto generated content

c571502

anjan-keysight requested a review from parthpower June 7, 2023 12:14

ankur-sheth requested changes Jun 7, 2023

View reviewed changes

flow/metrics.yaml Show resolved Hide resolved

flow/metrics.yaml Outdated Show resolved Hide resolved

flow/metrics.yaml Outdated Show resolved Hide resolved

bortok reviewed Jun 7, 2023

View reviewed changes

bortok requested changes Jun 7, 2023

View reviewed changes

anjan-keysight and others added 2 commits June 8, 2023 12:30

Name and desc update based on PR comments

72dd3d8

Update auto generated content

eb10839

anjan-keysight and others added 2 commits June 8, 2023 12:40

Minor name update

e0b2326

Update auto generated content

acd187a

anjan-keysight requested a review from ajbalogh June 8, 2023 13:29

anjan-keysight and others added 4 commits June 8, 2023 16:52

Description and object reference update

a04e2ed

Update auto generated content

6c7d3b6

Changes based on review comments

f12703c

Update auto generated content

276d21a

bortok approved these changes Jun 9, 2023

View reviewed changes

ajbalogh approved these changes Jun 12, 2023

View reviewed changes

ankur-sheth approved these changes Jun 12, 2023

View reviewed changes

apratimmukherjee approved these changes Jun 13, 2023

View reviewed changes

anjan-keysight and others added 2 commits June 13, 2023 04:35

Minor description update

7669e51

Update auto generated content

e2a08f9

anjan-keysight assigned jasdeep-hundal Jun 13, 2023

anjan-keysight requested a review from jasdeep-hundal June 13, 2023 08:48

PrasenjitAdhikary reviewed Jun 16, 2023

View reviewed changes

PrasenjitAdhikary approved these changes Jun 16, 2023

View reviewed changes

anjan-keysight and others added 4 commits June 16, 2023 08:31

Description update

fa722dd

Update auto generated content

579ad62

Update of enum type from string to str

1356b0e

Update auto generated content

83e23fa

jasdeep-hundal reviewed Jun 21, 2023

View reviewed changes

anjan-keysight and others added 2 commits June 22, 2023 09:09

Description update

d3e252c

Update auto generated content

cbbaa0c

anjan-keysight merged commit 6c6d5f6 into master Jun 23, 2023

anjan-keysight deleted the dev-port-disaggregation branch June 23, 2023 08:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev port disaggregation #320

Dev port disaggregation #320

anjan-keysight commented May 25, 2023 •

edited

Loading

parthpower commented May 30, 2023

anjan-keysight commented May 31, 2023

apratimmukherjee May 31, 2023

apratimmukherjee May 31, 2023

PrasenjitAdhikary Jun 7, 2023 •

edited by anjan-keysight

Loading

anjan-keysight Jun 7, 2023

bortok left a comment

bortok Jun 7, 2023

anjan-keysight Jun 8, 2023

bortok Jun 8, 2023

anjan-keysight Jun 9, 2023

bortok Jun 7, 2023

anjan-keysight Jun 8, 2023

bortok Jun 8, 2023

anjan-keysight Jun 9, 2023

apratimmukherjee left a comment

apratimmukherjee Jun 13, 2023

PrasenjitAdhikary Jun 16, 2023

anjan-keysight Jun 16, 2023

jasdeep-hundal Jun 21, 2023

PrasenjitAdhikary Jun 16, 2023

anjan-keysight Jun 16, 2023

PrasenjitAdhikary left a comment

jasdeep-hundal left a comment

jasdeep-hundal Jun 21, 2023

anjan-keysight Jun 22, 2023 •

edited

Loading

jasdeep-hundal Jun 21, 2023

anjan-keysight Jun 22, 2023

jasdeep-hundal Jun 21, 2023

Dev port disaggregation #320

Dev port disaggregation #320

Conversation

anjan-keysight commented May 25, 2023 • edited Loading

parthpower commented May 30, 2023

anjan-keysight commented May 31, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PrasenjitAdhikary Jun 7, 2023 • edited by anjan-keysight Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bortok left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

apratimmukherjee left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PrasenjitAdhikary left a comment

Choose a reason for hiding this comment

jasdeep-hundal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anjan-keysight Jun 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anjan-keysight commented May 25, 2023 •

edited

Loading

PrasenjitAdhikary Jun 7, 2023 •

edited by anjan-keysight

Loading

anjan-keysight Jun 22, 2023 •

edited

Loading