Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update pubsub docs #3962

Merged
merged 3 commits into from
Jan 10, 2025
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 42 additions & 4 deletions docs/content/docs/reference/pubsub.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@ top = false

FTL has first-class support for PubSub, modelled on the concepts of topics (where events are sent) and subscribers (a verb which consumes events). Subscribers are, as you would expect, sinks. Each subscriber is a cursor over the topic it is associated with. Each topic may have multiple subscriptions. Each published event has an at least once delivery guarantee for each subscription.

A topic can be exported to allow other module to subscribe to it. Subscriptions are always private to their module.
wesbillman marked this conversation as resolved.
Show resolved Hide resolved

When a subscription is first created in an environment, it can start consuming from the beginning of the topic or only consume events published afterwards.

Topics allow configuring the number of partitions and how each event should be mapped to a partition, allowing for greater throughput. Subscriptions will consume in order within each partition. There are cases where a small amount of progress on a subscription will be lost, so subscriptions should be able to handle receiving some events that have already been consumed.

{% code_selector() %}
<!-- go -->

Expand All @@ -26,21 +32,37 @@ package payments
import (
"github.com/block/ftl/go-runtime/ftl"
)

// Define an event type
type Invoice struct {
InvoiceNo string
}

//ftl:export
type Invoices = ftl.TopicHandle[Invoice, ftl.SinglePartitionMap[Invoice]]
// ftl.TopicPartitionMap is an interface for mapping each event to a partition in the topic.
//
// If creating a topic with multiple partitions, you'll need to define a partition mapper for your event type.
// Otherwise you can use ftl.SingpePartitionMap[Event]
wesbillman marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Preview

Copilot AI Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type name 'ftl.SingpePartitionMap[Event]' is incorrect. It should be 'ftl.SinglePartitionMap[Event]'.

Suggested change
// Otherwise you can use ftl.SingpePartitionMap[Event]
// Otherwise you can use ftl.SinglePartitionMap[Event]

Copilot is powered by AI, so mistakes are possible. Review output carefully before use.

Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
type PartitionMapper struct{}

var _ ftl.TopicPartitionMap[PubSubEvent] = PartitionMapper{}

func (PartitionMapper) PartitionKey(event PubSubEvent) string {
return event.Time.String()
}

//ftl:topic export partitions=10
type Invoices = ftl.TopicHandle[Invoice, PartitionMapper]
```

Note that the name of the topic as represented in the FTL schema is the lower camel case version of the type name.

The `Invoices` type is a handle to the topic. It is a generic type that takes two arguments: the event type and the partition map type. The partition map type is used to map events to partitions. In this case, we are using a single partition map, which means that all events are sent to the same partition.
The `Invoices` type is a handle to the topic. It is a generic type that takes two arguments: the event type and the partition map type. The partition map type is used to map events to partitions.

Then define a Sink to consume from the topic:

```go
// Configure initial event consumption with either from=beginning or from=latest
//
//ftl:subscribe payments.invoices from=beginning
func SendInvoiceEmail(ctx context.Context, in Invoice) error {
// ...
Expand All @@ -59,14 +81,22 @@ func PublishInvoice(ctx context.Context, topic Invoices) error {

<!-- kotlin -->

First, declare a new topic :
First, declare a new topic:

```kotlin

import com.block.ftl.WriteableTopic

// Define the event type for the topic
data class Invoice(val invoiceNo: String)

// PartitionMapper maps each to a partition in the topic
class PartitionMapper : TopicPartitionMapper<Invoice> {
override fun getPartitionKey(invoice: Invoice): String {
return invoice.getInvoiceNo()
}
}

@Export
@Topic(name = "invoices", partitions = 8)
internal interface InvoiceTopic : WriteableTopic<Invoice>
Expand Down Expand Up @@ -105,8 +135,16 @@ First, declare a new topic:
```java
import com.block.ftl.WriteableTopic;

// Define the event type for the topic
record Invoice(String invoiceNo) {}

// PartitionMapper maps each to a partition in the topic
class PartitionMapper implements TopicPartitionMapper<Invoice> {
public String getPartitionKey(Invoice invoice) {
return invoice.getInvoiceNo();
}
}

@Export
@Topic(name = "invoices", partitions = 8)
public interface InvoiceTopic extends WriteableTopic<Invoice> {}
Expand Down
Loading