With the policy portion of securing our application complete, we need a way to report that our application is in compliance going forward. There are two main tools for this within Calico Cloud:
By default, flow logs will be flushed from the hosts and stored in Elasticsearch every 300s (5 minutes). During normal operation this default value will be adequate for troubleshooting and auditing. For this workshop, we will reduce our flow and dns log flush interval to 30s to make it easier to see the results of our policies.
*Note - Increasing the frequency of logs sent to Elasticsearch will increase the storage usage.
kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"flowLogsFlushInterval":"30s"}}'
kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"dnsLogsFlushInterval":"30s"}}'In Calico Cloud, there are two tools that help us provide visualizations of our cluster and traffic:Service Graph and Flow Visualizations.
Service Graph is a graph of pod and service communication for all applications within the cluster. Allows for customized views to show relationships between namespaces, services, and deployments
Calico Cloud logs all network flows including source and destination namespaces, pods, labels, and the policies that evaluate each flow. Logs of all connection attempts (inside and outside the cluster) are automatically generated so you can quickly identify source of connectivity issues.
The Flow Visualizer lets you quickly drill down and pinpoint which policies are allowing and denying traffic between their services.
Calico Cloud includes a fully-integrated deployment of Elasticsearch to collect flow log data that drives key features like the Flow Visualizer, metrics in the dashboard and Policy Board, policy automation and testing features, and security.
Calico Cloud also embeds Kibana to view raw log data for the traffic within your cluster. Kibana provides its own set of powerful filtering capabilities to quickly drilling into log data. For example, use filters to drill into flow log data for specific namespaces and pods. Or view details and metadata for a single flow log entry.
Now that we've seen our traffic in the flow logs, lets reset our flow and dns log flush interval to the default:
kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"flowLogsFlushInterval":"300s"}}'
kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"dnsLogsFlushInterval":"300s"}}'Using the reporting feature of Calico Cloud we can create a number of reports to satisfy the various SOC 2 reporting requirements.
Calico Cloud supports the following built-in report types:
- Inventory
- Network Access
- Policy-Audit
- CIS Benchmark
These reports can be customized to report against a certain set of endpoints (for example SOC2 endpoints).
Compliance reports provide the following high-level information:
- Protection
- Endpoints explicitly protected using ingress or egress policy
- Endpoints with Envoy enabled
- Policies and services
- Policies and services associated with endpoints
- Policy audit logs
- Traffic
- Allowed ingress/egress traffic to/from namespaces
- Allowed ingress/egress traffic to/from the internet
The following report schedules daily inventory reports for all endpoints that have the 'soc2=true' label.
kubectl apply -f -<<EOF
apiVersion: projectcalico.org/v3
kind: GlobalReport
metadata:
name: daily-soc2-inventory
spec:
reportType: inventory
endpoints:
namespaces:
names:
- hipstershop
schedule: 8 6 * * *
EOFSOC2 Inventory Endpoints Example
SOC2 Inventory Summary Example
The following report schedules daily network-access reports for all endpoints that have the 'soc2=true' label.
kubectl apply -f -<<EOF
apiVersion: projectcalico.org/v3
kind: GlobalReport
metadata:
name: daily-soc2-network-access
spec:
reportType: network-access
endpoints:
namespaces:
names:
- hipstershop
schedule: 0 1 * * *
EOFSOC2 Network Access Endpoints Example
SOC2 Network Access Summary Example
The following report schedules a policy audit of the cluster at the frequency of your choosing
kubectl apply -f -<<EOF
apiVersion: projectcalico.org/v3
kind: GlobalReport
metadata:
name: daily-hipstershop-policy-audit
spec:
reportType: policy-audit
endpoints:
namespaces:
names:
- hipstershop
schedule: 0 0 * * *
EOFThe following report schedules a CIS benchmark report of the cluster at the frequency of your choosing
kubectl apply -f -<<EOF
apiVersion: projectcalico.org/v3
kind: GlobalReport
metadata:
name: daily-cis-benchmark
spec:
reportType: cis-benchmark
schedule: 0 0 * * *
cis:
highThreshold: 100
medThreshold: 50
includeUnscoredTests: true
numFailedTests: 5
resultsFilters:
- benchmarkSelection: { kubernetesVersion: "1.28" }
exclude: ["1.1.4", "1.2.5"]
EOF# for managed clusters you must set ELASTIC_INDEX_SUFFIX var to cluster name in the reporter pod template YAML
ELASTIC_INDEX_SUFFIX=$(kubectl get deployment -n tigera-intrusion-detection intrusion-detection-controller -ojson | jq -r '.spec.template.spec.containers[0].env[] | select(.name == "CLUSTER_NAME").value')
# on MacOS
START_TIME=$(date -v -2H -u +'%Y-%m-%dT%H:%M:%SZ')
END_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
# on Linux
START_TIME=$(date -d '-2 hours' -u +'%Y-%m-%dT%H:%M:%SZ')
END_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
# replace variables in YAML and deploy reporter jobs
sed -e "s?<ELASTIC_INDEX_SUFFIX>?$ELASTIC_INDEX_SUFFIX?g" \
-e "s?<TIGERA_COMPLIANCE_REPORT_START_TIME>?$START_TIME?g" \
-e "s?<TIGERA_COMPLIANCE_REPORT_START_TIME>?$END_TIME?g" \
5.\ Reports/manifests/compliance-reporter-pod.yaml | kubectl apply -f -
Calico Enterprise Documentation for Compliance Reports