-
Notifications
You must be signed in to change notification settings - Fork 374
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
loadBalancerSourceRanges not supported in AntreaProxy #5493
Comments
@hongliangl @wenyingd could you please take a look from Linux and Windows perspectives? |
After discussing @wenyingd, we have two designs. Take an example of two LoadBalancer Services
For design 1:
For design 2:
Could you give some suggestions @tnqn? Thanks a lot. |
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days |
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days |
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days |
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Key changes include: - Added a new reg mark, `LoadBalancerSourceRangesRegMark` (bit 29 of reg4), which is used to mark packets sourced from the CIDRs included in `loadBalancerSourceRanges`. - Renamed register mark `SvcNoEpRegMark` (bit 14 of reg0) to `SvcRejectRegMark` to generalize its use. It is used mark packets not sourced from CIDRs included in `loadBalancerSourceRanges`. - Renamed table `NodePortMark` to `ServiceMark` to generalize its use for installing flows for LoadBalancer Services with `loadBalancerSourceRanges` configured. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.152 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x20000000/0x20000000->reg4" 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x20000000/0x20000000->reg4" 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x4000/0x4000->reg0" 4. table=ServiceLB, priority=200,tcp,reg4=0x20010000/0x20070000,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0x14->reg7,group:20 5. table=EndpointDNAT, priority=200,reg0=0x4000/0x4000 actions=controller(reason=no_match,id=62373,userdata=04) ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `SvcRejectRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesRegMark` and performs load balancing. - Flow 5 is to match not allowed packets with `SvcRejectRegMark`, resulting in a reject reply. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Key changes include: - Added a new reg mark, `LoadBalancerSourceRangesRegMark` (bit 29 of reg4), which is used to mark packets sourced from the CIDRs included in `loadBalancerSourceRanges`. - Renamed register mark `SvcNoEpRegMark` (bit 14 of reg0) to `SvcRejectRegMark` to generalize its use. It marks packets not sourced from CIDRs included in `loadBalancerSourceRanges`. - Renamed table `NodePortMark` to `ServiceMark` to generalize its use for installing flows for LoadBalancer Services with `loadBalancerSourceRanges` configured. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.152 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x20000000/0x20000000->reg4" 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x20000000/0x20000000->reg4" 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x4000/0x4000->reg0" 4. table=ServiceLB, priority=200,tcp,reg4=0x20010000/0x20070000,nw_dst=192.168.77.152,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0x14->reg7,group:20 5. table=EndpointDNAT, priority=200,reg0=0x4000/0x4000 actions=controller(reason=no_match,id=62373,userdata=04) ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `SvcRejectRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesRegMark` and performs load balancing. - Flow 5 is to match not allowed packets with `SvcRejectRegMark`, resulting in a reject reply. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.152 ``` To implement `loadBalancerSourceRanges`, a new table LoadBalancerSourceRanges is introduced after table PreRoutingClassifier. Here are the corresponding flows: ```text 1. table=LoadBalancerSourceRanges, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=goto_table:SessionAffinity", 2. table=LoadBalancerSourceRanges, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=goto_table:SessionAffinity", 3. table=LoadBalancerSourceRanges, priority=190,tcp,nw_dst=192.168.77.152,tp_dst=80 actions=drop", 4. table=LoadBalancerSourceRanges, priority=0 actions=goto_table:SessionAffinity ``` Flows 1-2 allow packets destined for the for sample [LoadBalancer] from CIDRs specified in the `loadBalancerSourceRanges` of the Service. Flow 3, with lower priority, drops packets destined for the sample [LoadBalancer] that don't match any CIDRs within the `loadBalancerSourceRanges`. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.152 ``` To implement `loadBalancerSourceRanges`, a new table LoadBalancerSourceRanges is introduced after table PreRoutingClassifier. Here are the corresponding flows: ```text 1. table=LoadBalancerSourceRanges, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=goto_table:SessionAffinity", 2. table=LoadBalancerSourceRanges, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.152,tp_dst=80 actions=goto_table:SessionAffinity", 3. table=LoadBalancerSourceRanges, priority=190,tcp,nw_dst=192.168.77.152,tp_dst=80 actions=drop", 4. table=LoadBalancerSourceRanges, priority=0 actions=goto_table:SessionAffinity ``` Flows 1-2 allow packets destined for the for sample [LoadBalancer] from CIDRs specified in the `loadBalancerSourceRanges` of the Service. Flow 3, with lower priority, drops packets destined for the sample [LoadBalancer] that don't match any CIDRs within the `loadBalancerSourceRanges`. Signed-off-by: Hongliang Liu <[email protected]> Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.150 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x40000000/0x60000000->reg4", 4. table=ServiceLB, priority=200,tcp,reg4=0x0x20010000/0x0x60070000,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0xe->reg7,group:14 5. table=ServiceLB, priority=190,reg4=0x40000000/0x60000000 actions=drop ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesAllowRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `LoadBalancerSourceRangesDropRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesAllowRegMark` and perform load balancing. - Flow 5 is to match not allowed packets with `LoadBalancerSourceRangesDropRegMark` and drop. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.150 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x40000000/0x60000000->reg4", 4. table=ServiceLB, priority=200,tcp,reg4=0x0x20010000/0x0x60070000,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0xe->reg7,group:14 5. table=ServiceLB, priority=190,reg4=0x40000000/0x60000000 actions=drop ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesAllowRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `LoadBalancerSourceRangesDropRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesAllowRegMark` and perform load balancing. - Flow 5 is to match not allowed packets with `LoadBalancerSourceRangesDropRegMark` and drop. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.150 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x40000000/0x60000000->reg4", 4. table=ServiceLB, priority=200,tcp,reg4=0x0x20010000/0x0x60070000,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0xe->reg7,group:14 5. table=ServiceLB, priority=190,reg4=0x40000000/0x60000000 actions=drop ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesAllowRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `LoadBalancerSourceRangesDropRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesAllowRegMark` and perform load balancing. - Flow 5 is to match not allowed packets with `LoadBalancerSourceRangesDropRegMark` and drop. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.150 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x40000000/0x60000000->reg4", 4. table=ServiceLB, priority=200,tcp,reg4=0x0x20010000/0x0x60070000,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0xe->reg7,group:14 5. table=ServiceLB, priority=190,reg4=0x40000000/0x60000000 actions=drop ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesAllowRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `LoadBalancerSourceRangesDropRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesAllowRegMark` and perform load balancing. - Flow 5 is to match not allowed packets with `LoadBalancerSourceRangesDropRegMark` and drop. Signed-off-by: Hongliang Liu <[email protected]>
For antrea-io#5493 This commit introduces support for loadBalancerSourceRanges for LoadBalancer Services. Here is an example of a LoadBalancer Service configuration allowing access from specific CIDRs: ```yaml apiVersion: v1 kind: Service metadata: name: sample-loadbalancer-source-ranges spec: selector: app: web ports: - protocol: TCP port: 80 targetPort: 80 type: LoadBalancer loadBalancerSourceRanges: - "192.168.77.0/24" - "192.168.78.0/24" status: loadBalancer: ingress: - ip: 192.168.77.150 ``` [New] Here are the corresponding flows: ```text 1. table=ServiceMark, priority=200,tcp,nw_src=192.168.77.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 2. table=ServiceMark, priority=200,tcp,nw_src=192.168.78.0/24,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x20000000/0x60000000->reg4", 3. table=ServiceMark, priority=190,tcp,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x40000000/0x60000000->reg4", 4. table=ServiceLB, priority=200,tcp,reg4=0x0x20010000/0x0x60070000,nw_dst=192.168.77.150,tp_dst=80 actions=set_field:0x200/0x200->reg0,set_field:0x20000/0x70000->reg4,set_field:0xe->reg7,group:14 5. table=ServiceLB, priority=190,reg4=0x40000000/0x60000000 actions=drop ``` - Flow 1 is to match packets from allowed CIDR `192.168.77.0/24`, marking them with `LoadBalancerSourceRangesAllowRegMark`. - Flow 2 is similar to flow 1 but for CIDR `192.168.78.0/24`. - Flow 3 is to match packets not from allowed CIDRs, marking with `LoadBalancerSourceRangesDropRegMark`. - Flow 4 is to match allowed packets with `LoadBalancerSourceRangesAllowRegMark` and perform load balancing. - Flow 5 is to match not allowed packets with `LoadBalancerSourceRangesDropRegMark` and drop. Signed-off-by: Hongliang Liu <[email protected]>
For the current design implemented in #6181, see the git commit message. In an offline discussion, @wenyingd proposed a new design for the implementation: Here are the flows of the latest Antrea main branch code with proxyAll enabled. I believe that everyone is very familiar with them.
Here are the draft flows of the design to implement loadBalancerSourceRanges:
We can see key changes:
Look forwarding to your suggestions @tnqn @antoninbas |
Describe the bug
As of now antrea doesnt support loadBalancerSourceRanges in the antreaProxy. Since this is also not supported in the kube-proxy, we have no way to provdide this to Windows users.
To Reproduce
Similar to kubernetes/kubernetes#120033 ...
Expected
AntreaProxy would fully support the Kubernetes service spec...
Actual behavior
LoadbalancerSourceRanges that are outside of an packet's IP are allowed into antrea clusters where antreaproxy is used.
Note this isnt a HUGE bug b/c alas, even the windows service proxy doesnt yet implement this
QUESTION:
Could this be done by reusing HNS packet filtering from the HNS ACLs ? Or does it require OVS?
The text was updated successfully, but these errors were encountered: