Skip to content

Commit

Permalink
Update benchmark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
naqvis authored Nov 10, 2023
1 parent 825a4c0 commit 7a1cfe3
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions content/en/guides/benchmark/benchmark.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,39 @@
---
title: "Data plane benchmarking"
description: "Benchmarking FSM and fsm data planes"
title: "Service Mesh Data Plane Benchmark"
description: "Benchmarking FSM and Istio data planes"
type: docs
weight: 1
draft: false
---

FSM aims to provide service mesh functionality along with high performance and low resource consumption, so that resource-constrained edge environments can also use the service mesh functionality used in the cloud.
Flomesh Service Mesh (FSM) aims to provide service mesh functionality with a focus on high performance and low resource consumption. This allows resource-constrained edge environments to leverage service mesh functionality similar to the cloud.

In this test, benchmarks were conducted for **FSM (v1.1.4)** and **Istio (v1.19.3)**. The main focus is on the service latency distribution when using two different meshes, and on monitoring the resource overhead of the data plane.
In this test, benchmarks were conducted for **FSM (v1.1.4)** and **Istio (v1.19.3)**. The primary focus is on the service latency distribution when using two different meshes and monitoring the resource overhead of the data plane.

FSM uses Pipy as the data plane and Istio uses Envoy.
FSM uses Pipy as the data plane, while Istio uses Envoy.

**Before testing, we clarify that the focus is on comparing latency and resource consumption between them, instead of on extreme performance.**
**Before testing, it is important to note that the focus is on comparing latency and resource consumption between them, rather than extreme performance.**

## Testing Environment

The benchmark was tested in a Kubernetes cluster running on Azure Cloud VM. There are 2 *Standard_D8_v3* nodes in the cluster. FSM and Istio are both **on loose traffic mode and mTLS, and the other settings are default**.
The benchmark was tested in a Kubernetes cluster running on Azure Cloud VM. The cluster consists of 2 *Standard_D8_v3* nodes. FSM and Istio are both configured with **loose traffic mode and mTLS, while other settings are set to default**.

* Kubernetes: K3s v1.24.17+k3s1
* OS: Ubuntu 20.04
* Nodes: 8c32g * 2
* Sidecar: 1c512Mi

The test tool is on the branch `fsm` of [this repository](https://github.com/addozhang/istio-tools) which is forked from [istio/tools](https://github.com/istio/tools).
The test tool is located on the branch `fsm` of [this repository](https://github.com/addozhang/istio-tools), which is forked from [istio/tools](https://github.com/istio/tools).

## Procedure

The procedure was marked down in [this doc](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/fsm/README.md).
The procedure is documented in [this file](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/fsm/README.md).

In test tool, there are two applications: fortioclient and fortioserver. The load is generated by fortioclient by triggered with `kubectl exec`.
In the test tool, there are two applications: fortioclient and fortioserver. The load is generated by fortioclient triggered with `kubectl exec`.

For both meshes, we proceed the test for baseline (no sidecar) and both (two sidecars) modes. And generate load with 2, 4, 8, 16, 32, 64 concurrencies on QPS 1000. You can check the benchmark configs of [FSM](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/configs/fsm/fsm_latency.yaml) and [Istio](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/configs/istio/istio_latency.yaml).
For both meshes, tests are conducted for baseline (no sidecar) and both (two sidecars) modes. Load is generated with 2, 4, 8, 16, 32, 64 concurrencies at QPS 1000. You can review the benchmark configs for [FSM](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/configs/fsm/fsm_latency.yaml) and [Istio](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/configs/istio/istio_latency.yaml).

The most important aspect is setting [the request and limit resource](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/values.yaml#L9) on `1000m` and `512Mi`.
An essential aspect is setting [the request and limit resource](https://github.com/addozhang/istio-tools/blob/fsm/perf/benchmark/values.yaml#L9) to `1000m` and `512Mi`.

## Latency

Expand Down Expand Up @@ -80,4 +80,4 @@ This time, we benchmarked FSM and Istio data planes with limited sidecar resourc

From the results, FSM can still maintain high performance with low resource usage and more efficient use of resources. So FSM is particularly suitable for resource-constrained and large-scale scenarios, reducing costs effectively.. These are made possible by [Pipy](https://flomesh.io/pipy)'s low-resource, high-performance features.

Of course, while FSM is suitable for cloud, it can also be applied to edge computing scenarios.
Of course, while FSM is suitable for cloud, it can also be applied to edge computing scenarios.

0 comments on commit 7a1cfe3

Please sign in to comment.