Support custom backendRefs via extensions

[muwaqar]

Description:
Currently Envoy Gateway implements support for the following BackendReference.

• Service (support: Core)
• ServiceImport (support: Experimental)
• Backend (support: Envoy Gateway custom)

However, vendors may need to support specific backends (S3, EC2, Lambda, ...) which EG does not support natively. In that case, the ask is to enhance the Extensions sub-system to invoke a custom extension which can program the filter chain, route and upstream cluster for the configured backend correctly.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

I could find some previous discussion in #4423, specifically the comment at #4423 (comment)

Envoy Gateway extensions doc also mentions support for custom backendRefs is expected in future, but I could not find an issue tracking this, hence raising this.

Allow Extension Developers to introduce their own custom resources for extending the Gateway-API via ExtensionRefs, policyAttachments (future) and backendRefs (future).

[arkodg]

+1 @muwaqar, @guydc and @zhaohuabing have already +1'd this idea in #4423
would you be interested in driving this work ?

[arkodg]

reg AWS backends, there's this issue #688 tracking the work, although aws is a vendor specific backend, its fine to add a aws setting in the Backend resource since a Envoy filter exists for this in Envoy Proxy (we established this precedence when adding the datadog tracing field, which also is a native option in envoy proxy)

[zhaohuabing]

+1 native support for aws backend as they're widely used, starting with lambda could be a great first step.

[muwaqar]

great to hear there is broader consensus on this, is there a extension hook/api design sketched out or discussed for this already?

[zhaohuabing]

Native support for aws lambda can be added to https://github.com/envoyproxy/gateway/blob/main/api/v1alpha1/backend_types.go and implemented in the Gateway API translator and xDS translator.

For custom Backend, there's no extension hook/api design in plance, and this will require furtuer discussion.

[arkodg]

Here are some links related to the extension manager route

• Support for custom policies feat: Adding extension server policy handling. #3371
• Original Design Document https://github.com/envoyproxy/gateway/blob/main/site/content/en/contributions/design/extending-envoy-gateway.md
  cc @liorokman