diff --git a/2pc.html b/2pc.html index 8767cde..3143f2f 100644 --- a/2pc.html +++ b/2pc.html @@ -8,7 +8,7 @@ - + @@ -170,7 +170,7 @@ @@ -256,41 +256,10 @@

Transaction branch operation - - - Do nothing: reservations were already made - External API - Send a message or call and endpoint of another service - - - Do nothing: the guarantee charge was already made - Internal service - Send a message to another service - - - Update customer details in the internal database - Database - Perform the row update and unlock - - - Update the “recent bookings” cache and release the semaphore - In-memory resource - Edit and unlock an in-memory resource - - - Do nothing: the reminder notifications were already scheduled - Actor cluster - Send a command to an actor - - - Do nothing: the bookings log was already updated - File - Persist a change in a file - - -
Consistency
-

It’s up to the implementer to decide the level of consistency in the execution of the commit. Transaction failure is also a valid state and can be signaled by raising an exception in the target effect. This will lead to some degree of inconsistency in the overall system state, but that can be an acceptable compromise in some use cases.

-

For instance, in our imaginary example the “in-memory cache” could be locked only for a short time to preserve throughput. Because it’s not essential to the journey process, updating it could be skipped in case of delays. On the other hand, if the “internal database” update would still fail despite the lock, this could be signaled by raising an exception. This would conclude the transaction in a failed state, which could be surfaced in the UI to allow for manual remediation.

+ +

| Do nothing: reservations were already made | External API | - | | Do nothing: the guarantee charge was already made | Internal service | - | | Update customer details in the internal database | Database | Perform the row update and unlock | | Update the “recent bookings” cache and release the semaphore | In-memory resource | Edit and unlock an in-memory resource | | Do nothing: the reminder notifications were already scheduled | Actor cluster | - | | Do nothing: the bookings log was already updated | File | - |

Commit consistency
+

It’s up to the implementer to decide the level of consistency in the execution of the commit. Transaction failure is also valid and can be signaled by raising an exception in the target effect. Failure will lead to inconsistency in the overall system state, which can be an acceptable compromise in some use cases.

+

For instance, in our imaginary example, the “in-memory cache” could be locked only briefly to preserve throughput. Because it’s optional to the journey process, updating it could be skipped in case of delays. On the other hand, if the “internal database” update still fails despite the lock, the commit expression could signal this by raising an exception. The exception would conclude the transaction in a failed state, allowing surfacing in the UI for manual remediation.

Abort

This is triggered by the coordinator after at least one branch has voted for abort. The branch is expected to effect the local context to roll back the changes and return a confirmation value. The same flexibility as for prepare and commit applies here.

Some examples of abort operations in the same imaginary process:

@@ -334,10 +303,10 @@

Roll back the change in a file - -

The same flexibility applies here as for the commit operation: it’s up to the implementer to decide the level of consistency in the execution of the abort. In this dummy example, let’s suppose traveler reminders have already been sent, a compensation action could be to schedule a new notification inviting the customer to ignore previous messages. On the other hand, failing to cancel the hotel and flight reservations would be a more serious issue and should be surfaced as a failed transaction, to elicit manual remediation.

+
Abort consistency
+

The same flexibility applies here as for the commit operation: it’s up to the implementer to decide the level of consistency in the execution of the abort. In this dummy example, let’s suppose traveler reminders have already been sent: a compensation action could be to schedule a new notification inviting the customer to ignore previous messages. On the other hand, failing to cancel the hotel and flight reservations would be a more serious issue and should be considered a failed transaction to elicit manual remediation.

State diagram

-

Protocol state throughout the aforementioned phases is tracked by an event-sourced entity, with events representing transitions of the state. The transaction state machine diagram is depicted below. As usual, side-effects are invoked after successful event persistence, and repeated in case of recovery (at least once delivery characteristics).

+

Protocol state throughout the phases mentioned above is tracked by an event-sourced entity, with events representing state transitions. The transaction state machine diagram is depicted below. As usual, side-effects are invoked after successful event persistence and repeated in case of recovery (at least once delivery characteristics).

Consider the use case carefully

Certain use cases call for different techniques: for instance, the classical challenge of publishing events to a broker atomically with a change in a database is best solved by the simpler transaction outbox pattern or event sourcing.

2PC is often used in distributed technologies
-0.0.3 +0.1.0
@@ -396,7 +365,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/abstractions.html b/abstractions.html index 22de770..dded2e1 100644 --- a/abstractions.html +++ b/abstractions.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@ @@ -181,13 +181,13 @@

+ Edit this page

-0.0.3 +0.1.0
@@ -231,7 +231,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/branch.html b/branch.html index 8dc24fc..af63a93 100644 --- a/branch.html +++ b/branch.html @@ -8,7 +8,7 @@ - + @@ -169,7 +169,7 @@ @@ -214,13 +214,13 @@

An exception raised in one of these operations transitions the transaction to failed state (once all branches return).

- + Edit this page
-0.0.3 +0.1.0
@@ -264,7 +264,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/coordinator.html b/coordinator.html index cef9baa..4953ba2 100644 --- a/coordinator.html +++ b/coordinator.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@ @@ -192,13 +192,13 @@

+ Edit this page

-0.0.3 +0.1.0
@@ -242,7 +242,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/diagrams/TransactionEntity.png b/diagrams/TransactionEntity.png index 35efc67..458f645 100644 Binary files a/diagrams/TransactionEntity.png and b/diagrams/TransactionEntity.png differ diff --git a/diagrams/TransactionEntity.puml b/diagrams/TransactionEntity.puml index 1eaa5fc..88570c6 100644 --- a/diagrams/TransactionEntity.puml +++ b/diagrams/TransactionEntity.puml @@ -1,5 +1,4 @@ @startuml -skinparam handwritten true skinparam defaultFontName Chalkboard title Transaction entity state diagram @@ -12,6 +11,12 @@ Committing --> Committed : all branches confirm commit Aborting --> Aborted : all branches confirm abort note right of Preparing: side-effect: call prepare() on branches -note left of Committing: side-effect: call commit() on branches +note bottom of Committing: side-effect: call commit() on branches note right of Aborting: side-effect: call abort() on branches + +state Preparing #LightBlue +state Committing #LightGreen +state Aborting #LightCoral +state Committed #Green +state Aborted #Coral @enduml \ No newline at end of file diff --git a/example.html b/example.html index 7ad692f..ccdff16 100644 --- a/example.html +++ b/example.html @@ -8,7 +8,7 @@ - + @@ -169,7 +169,7 @@ @@ -238,7 +238,7 @@

Coordinator

A coordinator is created with TransferID as the transaction identifier type, AccountID as the branch identifier, Transfer as the query payload (the amount, origin, and destination), and TransferFailure as the error coproduct.

-
sourcetransactor.coordinator[TransferID, AccountID, Transfer, TransferFailure](
+
sourcetransactor.coordinator[TransferID, AccountID, Transfer, TransferFailure](
   "transfer",
   { accountID =>
     val account = sharding.entityFor(accountID)
@@ -249,7 +249,7 @@ 
Transaction

 
The snippet below shows the logic: the code creates a transfer with two branches, one for the origin account, and the other for the destination account.

-
sourcedef transfer(from: AccountID, to: AccountID, amount: PosAmount): F[TransferFailure \/ Unit] =
+
sourcedef transfer(from: AccountID, to: AccountID, amount: PosAmount): F[TransferFailure \/ Unit] =
   coordinator
     .create(TransferID.random, Transfer(from, to, amount), from, to)
     .use(_.pollForFinalStatus())
@@ -274,10 +274,12 @@ 
Branch

 
An account’s involvement in a transfer is described by TransferBranch, as exemplified below with the implementation of the prepare method: 

-
sourceclass TransferBranch[F[_]: Logger](accountID: AccountID, account: Account[F])(implicit
+
sourceclass TransferBranch[F[_]: Logger](accountID: AccountID, account: Account[F])(implicit
+    retryParameters: TransferParameters.BranchRetryParameters,
     temporal: Temporal[F]
 ) extends Branch[F, TransferID, Transfer, TransferFailure] {
   import temporal.*
+  private implicit val onErrorRetryParameters: RetryParameters = retryParameters.onError
 
   def prepare(transferID: TransferID, transfer: Transfer): F[Branch.Vote[TransferFailure]] = {
     if (accountID === transfer.origin)
@@ -286,44 +288,47 @@
- + Edit this page
-0.0.3 +0.1.0
@@ -367,7 +372,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/getting-started.html b/getting-started.html index c758564..e238b3c 100644 --- a/getting-started.html +++ b/getting-started.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@
@@ -172,20 +172,20 @@

Getting Started

Add the following dependency to your build.sbt file:

-
libraryDependencies += "io.github.endless4s" %% "endless-transaction" % "0.0.3"
+
libraryDependencies += "io.github.endless4s" %% "endless-transaction" % "0.1.0"
 

 
This will pull in the module containing the abstractions. You should add this dependency to the project that contains your business domain logic (typically “domain”).

 
The Pekko runtime is available in endless-transaction-pekko and provides PekkoTransactor, an implementation of Transactor which is the entry-point to create a transaction coordinator (for Akka, use endless-transaction-akka). 
Compatibility

-
Since Pekko/Akka do not allow mixed versions in a project, dependencies of endless-transaction-pekko (and endless-transaction-akka respectively) are marked a Provided. This means that your application libraryDependencies needs to directly include Pekko or Akka as a direct dependency. The minimal supported Pekko version is 1.0.0, and Akka version is 2.6.20. 

+
Since Pekko/Akka do not allow mixed versions in a project, dependencies of endless-transaction-pekko (and endless-transaction-akka respectively) are marked a Provided. This means that your application libraryDependencies needs to directly include Pekko or Akka as a direct dependency. The minimal supported Pekko version is 1.0.2, and Akka version is 2.6.20.
-0.0.3 +0.1.0
@@ -228,7 +228,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/index.html b/index.html index cd37840..bf63895 100644 --- a/index.html +++ b/index.html @@ -8,7 +8,7 @@ - + @@ -155,7 +155,7 @@
@@ -174,13 +174,13 @@
-0.0.3 +0.1.0
@@ -211,7 +211,7 @@ Powered by Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/nutshell.html b/nutshell.html index 2bf9948..cc9758d 100644 --- a/nutshell.html +++ b/nutshell.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@
@@ -174,16 +174,17 @@

two-phase commit protocol (2PC) is synchronous and blocking, endless-transaction is designed to be asynchronous and non-blocking and features fine-grained failure and retry semantics. It can be used as a ready-made tool to achieve various degrees of consistency in distributed systems, relying behind the scenes on an actor cluster for scalability and resilience.

The three tenants of the 2PC protocol are prepare, commit, and abort: in endless-transaction, participating branches have full flexibility in defining effectful expressions for the three operations. For certain domains, strong consistency and locking might be required. For others, some degree of inconsistency can be tolerated. It can even be a mix of both in a single transaction. The lifetime of the transaction isn’t limited either, transaction timeout tracking is optional.

Diverse forms of two-phase consensus appear frequently in service-oriented systems, especially when involving complex business workflows and heterogeneous data stores. Its usefulness goes beyond the specific area of strongly consistent, atomic transactions such as XA. Long-running transactions, also known as sagas, typically make use of rollback or undo operations in the abort phase, also called compensating transactions.

-

Abstractions in the library are implemented by one of the two available runtimes, Pekko or Akka. Internally, transactions are materialized with a persistent sharded entity implementing the two-phase protocol asynchronously with at least once delivery guarantee.

+

Abstractions in the library are implemented by one of the two available runtimes, Pekko or Akka. Internally, transactions are materialized with a persistent sharded entity implementing the two-phase protocol asynchronously with at least once delivery guarantee.

For more info
+

Check out the blog article Two-phase consensus with functional Scala

-0.0.3 +0.1.0
@@ -227,7 +228,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/paradox.json b/paradox.json index 267a004..808b49f 100644 --- a/paradox.json +++ b/paradox.json @@ -1,4 +1,4 @@ { "name" : "documentation", - "version" : "0.0.3" + "version" : "0.1.0" } \ No newline at end of file diff --git a/reference.html b/reference.html index f4c278a..64ca180 100644 --- a/reference.html +++ b/reference.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@
@@ -178,13 +178,13 @@

+ Edit this page

-0.0.3 +0.1.0
@@ -215,7 +215,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/search/search_index.json b/search/search_index.json index f6dcad6..1d66f78 100644 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"docs":[{"location":"/paradox.json","text":"","title":""},{"location":"/index.html","text":"endless-transaction is a Scala library that provides a flexible functional abstraction to orchestrate distributed transactions based on cats-effect and the endless library. It is a simple tool to simplify the process of coordinating transactions using a two-phase commit protocol.\nThis is not “old-school” 2PC however: the library abstractions can be used to describe any form of two-phase consensus, allowing for both short and long-lived transactions (aka sagas).","title":""},{"location":"/getting-started.html","text":"","title":"Getting Started ·"},{"location":"/getting-started.html#getting-started","text":"Add the following dependency to your build.sbt file:\nlibraryDependencies += \"io.github.endless4s\" %% \"endless-transaction\" % \"0.0.3\"\nThis will pull in the module containing the abstractions. You should add this dependency to the project that contains your business domain logic (typically “domain”).\nThe Pekko runtime is available in endless-transaction-pekko and provides PekkoTransactor, an implementation of Transactor which is the entry-point to create a transaction coordinator (for Akka, use endless-transaction-akka).\nCompatibility Since Pekko/Akka do not allow mixed versions in a project, dependencies of endless-transaction-pekko (and endless-transaction-akka respectively) are marked a Provided. This means that your application libraryDependencies needs to directly include Pekko or Akka as a direct dependency. The minimal supported Pekko version is 1.0.0, and Akka version is 2.6.20.","title":"Getting Started"},{"location":"/nutshell.html","text":"","title":"In a nutshell ·"},{"location":"/nutshell.html#in-a-nutshell","text":"While the traditional two-phase commit protocol (2PC) is synchronous and blocking, endless-transaction is designed to be asynchronous and non-blocking and features fine-grained failure and retry semantics. It can be used as a ready-made tool to achieve various degrees of consistency in distributed systems, relying behind the scenes on an actor cluster for scalability and resilience.\nThe three tenants of the 2PC protocol are prepare, commit, and abort: in endless-transaction, participating branches have full flexibility in defining effectful expressions for the three operations. For certain domains, strong consistency and locking might be required. For others, some degree of inconsistency can be tolerated. It can even be a mix of both in a single transaction. The lifetime of the transaction isn’t limited either, transaction timeout tracking is optional.\nDiverse forms of two-phase consensus appear frequently in service-oriented systems, especially when involving complex business workflows and heterogeneous data stores. Its usefulness goes beyond the specific area of strongly consistent, atomic transactions such as XA. Long-running transactions, also known as sagas, typically make use of rollback or undo operations in the abort phase, also called compensating transactions.\nAbstractions in the library are implemented by one of the two available runtimes, Pekko or Akka. Internally, transactions are materialized with a persistent sharded entity implementing the two-phase protocol asynchronously with at least once delivery guarantee.","title":"In a nutshell"},{"location":"/2pc.html","text":"","title":"Two-phase commit protocol ·"},{"location":"/2pc.html#two-phase-commit-protocol","text":"The three tenants of the 2PC protocol are prepare, commit, and abort: in endless-transaction, participating branches define effectful expressions for the three operations.","title":"Two-phase commit protocol"},{"location":"/2pc.html#prepare","text":"Validate a query value, possibly effect the local branch context, and return a vote value. The vote is a signal for the coordinator to either commit or abort the transaction. This expression can involve any kind of asynchronous process, and there is no limit to the time it takes for the vote to be delivered to the coordinator (unless a timeout is configured). Below are some examples of prepare operations in an imaginary (and somewhat convoluted) touristic journey booking process:\nExample: orchestration of the booking process for a journey Data store Transaction branch operation Create cancelable hotel & flight reservations External API Make a reversible synchronous HTTP POST/PUT request Request the billing back-end for credit card guarantee charge and await confirmation Internal service Send a message or call an endpoint and wait for an event to occur Add details to the customer row in the database Database Acquire an exclusive lock on a database row or use built-in XA support Grab a semaphore to update the “recent bookings” cache In-memory resource Lock an in-memory resource Schedule traveller reminder notifications Actor cluster Send a command to an actor Add an entry in a bookings log File Persist a rollback-ready change in a file","title":"Prepare"},{"location":"/2pc.html#commit","text":"This is triggered by the coordinator after all branches have voted for commit. The branch is expected to effect the local context to make at least part of the change durable and return a confirmation value.\nLike for prepare above, this expression can involve any kind of asynchronous process, and the toolkit does not impose a limit on the time it takes for the confirmation to be delivered to the coordinator.\nSome examples of commit operations in the same imaginary touristic journey booking process:\nExample: orchestration of the booking process for a journey Data store Transaction branch operation Do nothing: reservations were already made External API Send a message or call and endpoint of another service Do nothing: the guarantee charge was already made Internal service Send a message to another service Update customer details in the internal database Database Perform the row update and unlock Update the “recent bookings” cache and release the semaphore In-memory resource Edit and unlock an in-memory resource Do nothing: the reminder notifications were already scheduled Actor cluster Send a command to an actor Do nothing: the bookings log was already updated File Persist a change in a file\nConsistency It’s up to the implementer to decide the level of consistency in the execution of the commit. Transaction failure is also a valid state and can be signaled by raising an exception in the target effect. This will lead to some degree of inconsistency in the overall system state, but that can be an acceptable compromise in some use cases. For instance, in our imaginary example the “in-memory cache” could be locked only for a short time to preserve throughput. Because it’s not essential to the journey process, updating it could be skipped in case of delays. On the other hand, if the “internal database” update would still fail despite the lock, this could be signaled by raising an exception. This would conclude the transaction in a failed state, which could be surfaced in the UI to allow for manual remediation.","title":"Commit"},{"location":"/2pc.html#abort","text":"This is triggered by the coordinator after at least one branch has voted for abort. The branch is expected to effect the local context to roll back the changes and return a confirmation value. The same flexibility as for prepare and commit applies here.\nSome examples of abort operations in the same imaginary process:\nExample: orchestration of the booking process for a journey Data store Transaction branch operation Cancel the hotel & flight reservations External API Send a message or call and endpoint of another service Cancel the credit guarantee charge Internal service Send a message to another service Do nothing: the customer details do not need to be updated Database Unlock the row and do nothing Release the semaphore and do nothing: the cache does not need to be updated In-memory resource Unlock an in-memory resource Cancel the reminder notifications Actor cluster Send a command to an actor Roll back the booking log entry File Roll back the change in a file\nThe same flexibility applies here as for the commit operation: it’s up to the implementer to decide the level of consistency in the execution of the abort. In this dummy example, let’s suppose traveler reminders have already been sent, a compensation action could be to schedule a new notification inviting the customer to ignore previous messages. On the other hand, failing to cancel the hotel and flight reservations would be a more serious issue and should be surfaced as a failed transaction, to elicit manual remediation.","title":"Abort"},{"location":"/2pc.html#state-diagram","text":"Protocol state throughout the aforementioned phases is tracked by an event-sourced entity, with events representing transitions of the state. The transaction state machine diagram is depicted below. As usual, side-effects are invoked after successful event persistence, and repeated in case of recovery (at least once delivery characteristics).\nConsider the use case carefully Certain use cases call for different techniques: for instance, the classical challenge of publishing events to a broker atomically with a change in a database is best solved by the simpler transaction outbox pattern or event sourcing.\n2PC is often used in distributed technologies Distributed databases such as CockroachDB, MongoDB, and others implement 2PC to atomically store values across partitions. Apache Kafka allows producing messages across multiple partitions atomically with an implementation similar to 2PC.","title":"State diagram"},{"location":"/abstractions.html","text":"","title":"Abstractions ·"},{"location":"/abstractions.html#abstractions","text":"All definitions make use of the following type parameters:\nF[_]: abstract effectful context F encapsulating all values, e.g. IO[Boolean] BID: the branch identifier type, requiring an instance of BinaryCodec for serialization of the identifier in protobuf messages and events and Show for the built-in logging. TID: the transaction identifier type, requiring an instance of BinaryCodec for serialization of the identifier in protobuf messages and events and StringCodec as an entity identifier (this is also used for the built-in logging). Q: the query type (the payload sent to branches), requiring an instance of BinaryCodec for serialization in protobuf messages and events. R: the abort reason type (provided by branches when voting for transaction abort), requiring an instance of BinaryCodec for serialization in protobuf messages and events.","title":"Abstractions"},{"location":"/transactor.html","text":"","title":"Transactor ·"},{"location":"/transactor.html#transactor","text":"trait Transactor[F[_]] \n def coordinator[\n TID: StringCodec: BinaryCodec,\n BID: BinaryCodec: Show,\n Q: BinaryCodec,\n R: BinaryCodec\n ](transactionName: String,\n branchForID: BID => Branch[F, TID, Q, R],\n timeout: Option[FiniteDuration] = None\n ): Resource[F, Coordinator[F, TID, BID, Q, R]]\nTransactor is the entry point provided by the runtime to create distributed transaction coordinators for any type of transaction. Particular transaction types are defined by the type parameters of the coordinator method: TID is the transaction identifier, BID is the branch identifier, Q is the query type, and R is the abort reason type.\nThe coordinator method creates a Coordinator for a given transaction type. Its parameters are:\ntransactionName: used as the entity name for persistence. branchForID: a function that describes the branch behavior for a given branch ID. A Branch describes what to do in the prepare, commit, and abort phases of the transaction. In other words, this defines the various “sides” of the transaction. Branch behavior can be differentiated by branch ID. No constraints are set on the effects a branch can have: it can describe interactions with heterogeneous systems via HTTP, multiple entities in a cluster, etc. timeout: optional timeout for transaction preparation. When defined, elapsed time is tracked internally during the prepare phase, and timing out leads to transaction abort with timeout reason.","title":"Transactor"},{"location":"/coordinator.html","text":"","title":"Coordinator ·"},{"location":"/coordinator.html#coordinator","text":"trait Coordinator[F[_], TID, BID, Q, R] \n def create(\n id: TID,\n query: Q,\n branch1: BID,\n branch2: BID,\n others: BID*\n ): Resource[F, Transaction[F, BID, Q, R]]\nCoordinator allows for creating and recovering transactions of a certain type. Internally, it implements the two-phase commit protocol to guarantee a final transaction state of either committed or aborted.\nThe create method creates a new transaction with the given id, query and branches, and returns a Transaction wrapped in a Resource. Resource release leads to transaction abort if it is still pending (or a no-op if the transaction is already completed).\nParameters of create are:\nid: the transaction identifier query: the payload sent to branches branch...: identifiers for the branches involved in the transaction\nCompatibility Behavior for each branch is defined by the branchForID function passed to the Coordinator upon creation. This is invoked upon node restart for any pending transaction. It is therefore important to consider compatibility aspects if any change is done on this behavior. The transaction might be in a partial preparation readiness state, and rollout of the new version will pick up preparation where it left off. As long as the new behavior is compatible with the old, this is not a problem.","title":"Coordinator"},{"location":"/transaction.html","text":"","title":"Transaction ·"},{"location":"/transaction.html#transaction","text":"trait Transaction[F[_], BID, Q, R] {\n def query: F[Unknown.type \\/ Q]\n def branches: F[Unknown.type \\/ Set[BID]]\n def status: F[Unknown.type \\/ Status[R]]\n def abort(reason: Option[R] = None): F[AbortError \\/ Unit]\n}\nTransaction is the handle to the transaction sharded entity, which can be used to inspect its status, retrieve its query and participating branches, and trigger a client abort.\nThe various states of the transaction are:\nState Description Side-effect (upon transition into state or recovery) Preparing Prepares have been issued by the coordinator, which is now waiting for votes from the participating branches. Send prepare messages (to branches that have yet to vote). Committing All participating branches have voted in favor of transaction commit. Branch commits have been issued by the coordinator, which is now waiting for branch commit confirmations. Send commit messages (to branches that have yet to confirm committed) Committed Transaction has successfully completed, all participating branches have confirmed the commit. N/A Aborting At least one participating branch voted against transaction commit, or an abort has been requested by the client in the meantime. Branch aborts have been issued by the coordinator which is now waiting for confirmation from the branches. Send abort messages (to branches that have yet to confirm aborted). Aborted Transaction has been aborted and all participating branches have confirmed the abort N/A Failed Transaction has failed due to an exception raised by one of the participating branches N/A","title":"Transaction"},{"location":"/branch.html","text":"","title":"Branch ·"},{"location":"/branch.html#branch","text":"trait Branch[F[_], TID, Q, R] \n def prepare(id: TID, query: Q): F[Vote[R]]\n def commit(id: TID): F[Unit]\n def abort(id: TID): F[Unit]\nA branch defines the behavior of the various phases of the 2PC protocol for a certain transaction type. The branch is responsible for preparing, committing, and aborting the transaction. The coordinator instantiates a branch for each transaction and branch ID. The branch provides an indirection to the actual effectful operations that are to be performed in the various phases of the protocol, such as sending messages to other services, updating databases, etc.","title":"Branch"},{"location":"/branch.html#prepare","text":"Prepare the branch for a transaction. This is the first step in the 2PC protocol. This operation supports an optional timeout (this is tracked by the coordinator), in which case the transaction will be aborted. However, there is no intrinsic limitation in the duration this operation may take. It could span days, as long as it returns a vote at some point and the coordinator is kept alive.","title":"Prepare"},{"location":"/branch.html#commit","text":"Commit the transaction branch. This is the second step in the 2PC protocol. This operation should in principle not fail, all branches are expected to commit at this point to ensure consistency. It’s up to the branch to organize retries if necessary. As for prepare, there is no intrinsic limitation in the duration this operation may take. Confirmation is expected by the coordinator at some point to transition the transaction to committed, however.","title":"Commit"},{"location":"/branch.html#abort","text":"Abort the transaction branch. This is the second step in the 2PC protocol, in case of preparation timeout or any branch voting to abort. All branches are expected to abort at this point, failure could lead to local inconsistency. It’s up to the branch to decide about that though, and organize retries if necessary. As for prepare and commit, there is no intrinsic limitation in the duration this operation may take. Confirmation is expected by the coordinator at some point to transition the transaction to aborted, however.\nIdempotency These operations are expected to be idempotent, as they may be retried by the coordinator.\nExceptions An exception raised in one of these operations transitions the transaction to failed state (once all branches return).","title":"Abort"},{"location":"/example.html","text":"","title":"Example ·"},{"location":"/example.html#example","text":"The example app is a dummy API for managing bank accounts and transferring amounts between accounts. Accounts are implemented with sharded entities, and transfers with transactions.\nThe example app can be found in the example module and can be run directly with sbt run.\nStress-test The module also contains a multi-jvm test that demonstrates the resilience of the system, by transferring amounts between 1000 accounts while restarting nodes.","title":"Example"},{"location":"/example.html#api","text":"The API has a simple CRUD for accounts and a dedicated endpoint to perform transfers between two accounts.\nHttpRoutes.of[IO] {\n case GET -> Root / \"account\" / id / \"balance\" => server.balanceFor(AccountID(id))\n case POST -> Root / \"account\" / id => server.open(AccountID(id))\n case POST -> Root / \"account\" / id / \"deposit\" / IntVar(amount) => server.deposit(AccountID(id), amount)\n case POST -> Root / \"account\" / id / \"withdraw\" / IntVar(amount) => server.withdraw(AccountID(id), amount)\n case POST -> Root / \"account\" / origin / \"transfer\" / \"to\" / destination / IntVar(amount) => server.transfer(AccountID(origin), AccountID(destination), PosAmount(amount))\n }\n .orNotFound\nThe HTTP server makes uses of the Accounts and Account algebras to satisfy the requests.","title":"API"},{"location":"/example.html#algebras","text":"","title":"Algebras"},{"location":"/example.html#accounts","text":"This trait represents the repository, i.e. the ability to retrieve a handle on a specific account and orchestrate transfers between accounts.\ntrait Accounts[F[_]] \n def accountFor(name: AccountID): Account[F]\n def transfer(from: AccountID, to: AccountID, amount: PosAmount): F[TransferFailure \\/ Unit]","title":"Accounts"},{"location":"/example.html#account","text":"This trait represents the account entity, i.e. the ability to query and perform operations on a specific account.\ntrait Account[F[_]] \n def open: F[AlreadyExists.type \\/ Unit]\n def balance: F[Unknown.type \\/ NonNegAmount]\n def deposit(amount: PosAmount): F[Unknown.type \\/ PosAmount]\n def withdraw(amount: PosAmount): F[WithdrawFailure \\/ NonNegAmount]\n\n def prepareOutgoingTransfer(id: TransferID, transfer: Transfer): F[WithdrawFailure \\/ Unit]\n def prepareIncomingTransfer(id: TransferID, transfer: Transfer): F[IncomingTransferFailure \\/ Unit]\n def commitTransfer(id: TransferID): F[TransferFailure \\/ Unit]\n def abortTransfer(id: TransferID): F[TransferFailure \\/ Unit]\nThe latter four methods are of particular interest as they are used by the transactional branch logic to carry out the various phases of the transfer.","title":"Account"},{"location":"/example.html#transfers","text":"Transfers are implemented using an endless-transaction transaction coordinator.","title":"Transfers"},{"location":"/example.html#coordinator","text":"A coordinator is created with TransferID as the transaction identifier type, AccountID as the branch identifier, Transfer as the query payload (the amount, origin, and destination), and TransferFailure as the error coproduct.\ncopysourcetransactor.coordinator[TransferID, AccountID, Transfer, TransferFailure](\n \"transfer\",\n { accountID =>\n val account = sharding.entityFor(accountID)\n new TransferBranch(accountID, account)\n },\n Some(transferParameters.timeout)\n)\nThe coordinator is used in the implementation of the transfer method in ShardedAccounts, i.e. the implementation of Accounts.","title":"Coordinator"},{"location":"/example.html#transaction","text":"The snippet below shows the logic: the code creates a transfer with two branches, one for the origin account, and the other for the destination account.\ncopysourcedef transfer(from: AccountID, to: AccountID, amount: PosAmount): F[TransferFailure \\/ Unit] =\n coordinator\n .create(TransferID.random, Transfer(from, to, amount), from, to)\n .use(_.pollForFinalStatus())\n .flatMap {\n case Status.Committed => ().asRight[TransferFailure].pure\n case Status.Aborted(reason) =>\n reason match {\n case AbortReason.Timeout =>\n EitherT.leftT(TransferFailure.Timeout: TransferFailure).value\n case AbortReason.Branches(reasons) => EitherT.leftT(reasons.head).value\n case AbortReason.Client(Some(reason)) => EitherT.leftT(reason).value\n case AbortReason.Client(None) =>\n new Exception(\"Transaction aborted by client without justification\")\n .raiseError[F, TransferFailure \\/ Unit]\n }\n case Status.Failed(errors) =>\n Logger[F].error(show\"Transaction failed: $errors\") *> new Exception(\n \"Transaction failed due to branch error\"\n ).raiseError[F, TransferFailure \\/ Unit]\n }\nThe transaction entity is sharded, so it can be running on a separate node. We therefore need to regularly check for completion so that we can respond to the API call (it’s synchronous here for the sake of simplicity).\nTo implement this polling operation, we use the pollForFinalStatus() built-in extension method (defined for Transaction): this method retrieves the status of the transaction at configurable intervals and semantically sleeps in between.","title":"Transaction"},{"location":"/example.html#branch","text":"An account’s involvement in a transfer is described by TransferBranch, as exemplified below with the implementation of the prepare method:\ncopysourceclass TransferBranch[F[_]: Logger](accountID: AccountID, account: Account[F])(implicit\n temporal: Temporal[F]\n) extends Branch[F, TransferID, Transfer, TransferFailure] {\n import temporal.*\n\n def prepare(transferID: TransferID, transfer: Transfer): F[Branch.Vote[TransferFailure]] = {\n if (accountID === transfer.origin)\n Logger[F].debug(\n show\"Preparing outgoing transfer $transferID: $transfer for account $accountID\"\n ) >>\n account\n .prepareOutgoingTransfer(transferID, transfer)\n .map {\n case Left(Account.Unknown) =>\n Branch.Vote.Abort(TransferFailure.AccountNotFound(accountID))\n case Left(InsufficientFunds(missing)) =>\n Branch.Vote.Abort(TransferFailure.InsufficientFunds(missing))\n case Left(Account.PendingOutgoingTransfer) =>\n Branch.Vote.Abort(TransferFailure.OtherPendingTransfer)\n case Right(_) => Branch.Vote.Commit: Branch.Vote[TransferFailure]\n }\n .retryWithBackoff(\n Logger[F]\n .warn(_)(show\"Error preparing outgoing transfer $transferID, retrying in a bit\")\n )\n else\n Logger[F].debug(show\"Preparing incoming $transferID: $transfer for account $accountID\") >>\n account\n .prepareIncomingTransfer(transferID, transfer)\n .map {\n case Left(Account.Unknown) =>\n Branch.Vote.Abort(TransferFailure.AccountNotFound(accountID))\n case Left(Account.PendingIncomingTransfer) =>\n Branch.Vote.Abort(TransferFailure.OtherPendingTransfer)\n case Right(_) => Branch.Vote.Commit: Branch.Vote[TransferFailure]\n }\n .retryWithBackoff(\n Logger[F]\n .warn(_)(show\"Error preparing incoming transfer $transferID, retrying in a bit\")\n )\n }","title":"Branch"},{"location":"/reference.html","text":"","title":"Reference ·"},{"location":"/reference.html#reference","text":"Lib API documentation Akka Runtime API documentation Pekko Runtime API documentation","title":"Reference"}]} \ No newline at end of file +{"docs":[{"location":"paradox.json","text":"","title":""},{"location":"index.html","text":"endless-transaction is a Scala library that provides a flexible functional abstraction to orchestrate distributed transactions based on cats-effect and the endless library. It is a simple tool to simplify the process of coordinating transactions using a two-phase commit protocol.\nThis is not “old-school” 2PC however: the library abstractions can be used to describe any form of two-phase consensus, allowing for both short and long-lived transactions (aka sagas).","title":""},{"location":"getting-started.html","text":"","title":"Getting Started ·"},{"location":"getting-started.html#getting-started","text":"Add the following dependency to your build.sbt file:\nlibraryDependencies += \"io.github.endless4s\" %% \"endless-transaction\" % \"0.1.0\"\nThis will pull in the module containing the abstractions. You should add this dependency to the project that contains your business domain logic (typically “domain”).\nThe Pekko runtime is available in endless-transaction-pekko and provides PekkoTransactor, an implementation of Transactor which is the entry-point to create a transaction coordinator (for Akka, use endless-transaction-akka).\nCompatibility Since Pekko/Akka do not allow mixed versions in a project, dependencies of endless-transaction-pekko (and endless-transaction-akka respectively) are marked a Provided. This means that your application libraryDependencies needs to directly include Pekko or Akka as a direct dependency. The minimal supported Pekko version is 1.0.2, and Akka version is 2.6.20.","title":"Getting Started"},{"location":"nutshell.html","text":"","title":"In a nutshell ·"},{"location":"nutshell.html#in-a-nutshell","text":"While the traditional two-phase commit protocol (2PC) is synchronous and blocking, endless-transaction is designed to be asynchronous and non-blocking and features fine-grained failure and retry semantics. It can be used as a ready-made tool to achieve various degrees of consistency in distributed systems, relying behind the scenes on an actor cluster for scalability and resilience.\nThe three tenants of the 2PC protocol are prepare, commit, and abort: in endless-transaction, participating branches have full flexibility in defining effectful expressions for the three operations. For certain domains, strong consistency and locking might be required. For others, some degree of inconsistency can be tolerated. It can even be a mix of both in a single transaction. The lifetime of the transaction isn’t limited either, transaction timeout tracking is optional.\nDiverse forms of two-phase consensus appear frequently in service-oriented systems, especially when involving complex business workflows and heterogeneous data stores. Its usefulness goes beyond the specific area of strongly consistent, atomic transactions such as XA. Long-running transactions, also known as sagas, typically make use of rollback or undo operations in the abort phase, also called compensating transactions.\nAbstractions in the library are implemented by one of the two available runtimes, Pekko or Akka. Internally, transactions are materialized with a persistent sharded entity implementing the two-phase protocol asynchronously with at least once delivery guarantee.\nFor more info Check out the blog article Two-phase consensus with functional Scala","title":"In a nutshell"},{"location":"2pc.html","text":"","title":"Two-phase commit protocol ·"},{"location":"2pc.html#two-phase-commit-protocol","text":"The three tenants of the 2PC protocol are prepare, commit, and abort: in endless-transaction, participating branches define effectful expressions for the three operations.","title":"Two-phase commit protocol"},{"location":"2pc.html#prepare","text":"Validate a query value, possibly effect the local branch context, and return a vote value. The vote is a signal for the coordinator to either commit or abort the transaction. This expression can involve any kind of asynchronous process, and there is no limit to the time it takes for the vote to be delivered to the coordinator (unless a timeout is configured). Below are some examples of prepare operations in an imaginary (and somewhat convoluted) touristic journey booking process:\nExample: orchestration of the booking process for a journey Data store Transaction branch operation Create cancelable hotel & flight reservations External API Make a reversible synchronous HTTP POST/PUT request Request the billing back-end for credit card guarantee charge and await confirmation Internal service Send a message or call an endpoint and wait for an event to occur Add details to the customer row in the database Database Acquire an exclusive lock on a database row or use built-in XA support Grab a semaphore to update the “recent bookings” cache In-memory resource Lock an in-memory resource Schedule traveller reminder notifications Actor cluster Send a command to an actor Add an entry in a bookings log File Persist a rollback-ready change in a file","title":"Prepare"},{"location":"2pc.html#commit","text":"This is triggered by the coordinator after all branches have voted for commit. The branch is expected to effect the local context to make at least part of the change durable and return a confirmation value.\nLike for prepare above, this expression can involve any kind of asynchronous process, and the toolkit does not impose a limit on the time it takes for the confirmation to be delivered to the coordinator.\nSome examples of commit operations in the same imaginary touristic journey booking process:\nExample: orchestration of the booking process for a journey Data store Transaction branch operation\n| Do nothing: reservations were already made | External API | - | | Do nothing: the guarantee charge was already made | Internal service | - | | Update customer details in the internal database | Database | Perform the row update and unlock | | Update the “recent bookings” cache and release the semaphore | In-memory resource | Edit and unlock an in-memory resource | | Do nothing: the reminder notifications were already scheduled | Actor cluster | - | | Do nothing: the bookings log was already updated | File | - |\nCommit consistency It’s up to the implementer to decide the level of consistency in the execution of the commit. Transaction failure is also valid and can be signaled by raising an exception in the target effect. Failure will lead to inconsistency in the overall system state, which can be an acceptable compromise in some use cases. For instance, in our imaginary example, the “in-memory cache” could be locked only briefly to preserve throughput. Because it’s optional to the journey process, updating it could be skipped in case of delays. On the other hand, if the “internal database” update still fails despite the lock, the commit expression could signal this by raising an exception. The exception would conclude the transaction in a failed state, allowing surfacing in the UI for manual remediation.","title":"Commit"},{"location":"2pc.html#abort","text":"This is triggered by the coordinator after at least one branch has voted for abort. The branch is expected to effect the local context to roll back the changes and return a confirmation value. The same flexibility as for prepare and commit applies here.\nSome examples of abort operations in the same imaginary process:\nExample: orchestration of the booking process for a journey Data store Transaction branch operation Cancel the hotel & flight reservations External API Send a message or call and endpoint of another service Cancel the credit guarantee charge Internal service Send a message to another service Do nothing: the customer details do not need to be updated Database Unlock the row and do nothing Release the semaphore and do nothing: the cache does not need to be updated In-memory resource Unlock an in-memory resource Cancel the reminder notifications Actor cluster Send a command to an actor Roll back the booking log entry File Roll back the change in a file\nAbort consistency The same flexibility applies here as for the commit operation: it’s up to the implementer to decide the level of consistency in the execution of the abort. In this dummy example, let’s suppose traveler reminders have already been sent: a compensation action could be to schedule a new notification inviting the customer to ignore previous messages. On the other hand, failing to cancel the hotel and flight reservations would be a more serious issue and should be considered a failed transaction to elicit manual remediation.","title":"Abort"},{"location":"2pc.html#state-diagram","text":"Protocol state throughout the phases mentioned above is tracked by an event-sourced entity, with events representing state transitions. The transaction state machine diagram is depicted below. As usual, side-effects are invoked after successful event persistence and repeated in case of recovery (at least once delivery characteristics).\nConsider the use case carefully Certain use cases call for different techniques: for instance, the classical challenge of publishing events to a broker atomically with a change in a database is best solved by the simpler transaction outbox pattern or event sourcing.\n2PC is often used in distributed technologies Distributed databases such as CockroachDB, MongoDB, and others implement 2PC to atomically store values across partitions. Apache Kafka allows producing messages across multiple partitions atomically with an implementation similar to 2PC.","title":"State diagram"},{"location":"abstractions.html","text":"","title":"Abstractions ·"},{"location":"abstractions.html#abstractions","text":"All definitions make use of the following type parameters:\nF[_]: abstract effectful context F encapsulating all values, e.g. IO[Boolean] BID: the branch identifier type, requiring an instance of BinaryCodec for serialization of the identifier in protobuf messages and events and Show for the built-in logging. TID: the transaction identifier type, requiring an instance of BinaryCodec for serialization of the identifier in protobuf messages and events and StringCodec as an entity identifier (this is also used for the built-in logging). Q: the query type (the payload sent to branches), requiring an instance of BinaryCodec for serialization in protobuf messages and events. R: the abort reason type (provided by branches when voting for transaction abort), requiring an instance of BinaryCodec for serialization in protobuf messages and events.","title":"Abstractions"},{"location":"transactor.html","text":"","title":"Transactor ·"},{"location":"transactor.html#transactor","text":"trait Transactor[F[_]] \n def coordinator[\n TID: StringCodec: BinaryCodec,\n BID: BinaryCodec: Show,\n Q: BinaryCodec,\n R: BinaryCodec\n ](transactionName: String,\n branchForID: BID => Branch[F, TID, Q, R],\n timeout: Option[FiniteDuration] = None\n ): Resource[F, Coordinator[F, TID, BID, Q, R]]\nTransactor is the entry point provided by the runtime to create distributed transaction coordinators for any type of transaction. Particular transaction types are defined by the type parameters of the coordinator method: TID is the transaction identifier, BID is the branch identifier, Q is the query type, and R is the abort reason type.\nThe coordinator method creates a Coordinator for a given transaction type. Its parameters are:\ntransactionName: used as the entity name for persistence. branchForID: a function that describes the branch behavior for a given branch ID. A Branch describes what to do in the prepare, commit, and abort phases of the transaction. In other words, this defines the various “sides” of the transaction. Branch behavior can be differentiated by branch ID. No constraints are set on the effects a branch can have: it can describe interactions with heterogeneous systems via HTTP, multiple entities in a cluster, etc. timeout: optional timeout for transaction preparation. When defined, elapsed time is tracked internally during the prepare phase, and timing out leads to transaction abort with timeout reason.","title":"Transactor"},{"location":"coordinator.html","text":"","title":"Coordinator ·"},{"location":"coordinator.html#coordinator","text":"trait Coordinator[F[_], TID, BID, Q, R] \n def create(\n id: TID,\n query: Q,\n branch1: BID,\n branch2: BID,\n others: BID*\n ): Resource[F, Transaction[F, BID, Q, R]]\nCoordinator allows for creating and recovering transactions of a certain type. Internally, it implements the two-phase commit protocol to guarantee a final transaction state of either committed or aborted.\nThe create method creates a new transaction with the given id, query and branches, and returns a Transaction wrapped in a Resource. Resource release leads to transaction abort if it is still pending (or a no-op if the transaction is already completed).\nParameters of create are:\nid: the transaction identifier query: the payload sent to branches branch...: identifiers for the branches involved in the transaction\nCompatibility Behavior for each branch is defined by the branchForID function passed to the Coordinator upon creation. This is invoked upon node restart for any pending transaction. It is therefore important to consider compatibility aspects if any change is done on this behavior. The transaction might be in a partial preparation readiness state, and rollout of the new version will pick up preparation where it left off. As long as the new behavior is compatible with the old, this is not a problem.","title":"Coordinator"},{"location":"transaction.html","text":"","title":"Transaction ·"},{"location":"transaction.html#transaction","text":"trait Transaction[F[_], BID, Q, R] {\n def query: F[Unknown.type \\/ Q]\n def branches: F[Unknown.type \\/ Set[BID]]\n def status: F[Unknown.type \\/ Status[R]]\n def abort(reason: Option[R] = None): F[AbortError \\/ Unit]\n}\nTransaction is the handle to the transaction sharded entity, which can be used to inspect its status, retrieve its query and participating branches, and trigger a client abort.\nThe various states of the transaction are:\nState Description Side-effect (upon transition into state or recovery) Preparing Prepares have been issued by the coordinator, which is now waiting for votes from the participating branches. Send prepare messages (to branches that have yet to vote). Committing All participating branches have voted in favor of transaction commit. Branch commits have been issued by the coordinator, which is now waiting for branch commit confirmations. Send commit messages (to branches that have yet to confirm committed) Committed Transaction has successfully completed, all participating branches have confirmed the commit. N/A Aborting At least one participating branch voted against transaction commit, or an abort has been requested by the client in the meantime. Branch aborts have been issued by the coordinator which is now waiting for confirmation from the branches. Send abort messages (to branches that have yet to confirm aborted). Aborted Transaction has been aborted and all participating branches have confirmed the abort N/A Failed Transaction has failed due to an exception raised by one of the participating branches N/A","title":"Transaction"},{"location":"branch.html","text":"","title":"Branch ·"},{"location":"branch.html#branch","text":"trait Branch[F[_], TID, Q, R] \n def prepare(id: TID, query: Q): F[Vote[R]]\n def commit(id: TID): F[Unit]\n def abort(id: TID): F[Unit]\nA branch defines the behavior of the various phases of the 2PC protocol for a certain transaction type. The branch is responsible for preparing, committing, and aborting the transaction. The coordinator instantiates a branch for each transaction and branch ID. The branch provides an indirection to the actual effectful operations that are to be performed in the various phases of the protocol, such as sending messages to other services, updating databases, etc.","title":"Branch"},{"location":"branch.html#prepare","text":"Prepare the branch for a transaction. This is the first step in the 2PC protocol. This operation supports an optional timeout (this is tracked by the coordinator), in which case the transaction will be aborted. However, there is no intrinsic limitation in the duration this operation may take. It could span days, as long as it returns a vote at some point and the coordinator is kept alive.","title":"Prepare"},{"location":"branch.html#commit","text":"Commit the transaction branch. This is the second step in the 2PC protocol. This operation should in principle not fail, all branches are expected to commit at this point to ensure consistency. It’s up to the branch to organize retries if necessary. As for prepare, there is no intrinsic limitation in the duration this operation may take. Confirmation is expected by the coordinator at some point to transition the transaction to committed, however.","title":"Commit"},{"location":"branch.html#abort","text":"Abort the transaction branch. This is the second step in the 2PC protocol, in case of preparation timeout or any branch voting to abort. All branches are expected to abort at this point, failure could lead to local inconsistency. It’s up to the branch to decide about that though, and organize retries if necessary. As for prepare and commit, there is no intrinsic limitation in the duration this operation may take. Confirmation is expected by the coordinator at some point to transition the transaction to aborted, however.\nIdempotency These operations are expected to be idempotent, as they may be retried by the coordinator.\nExceptions An exception raised in one of these operations transitions the transaction to failed state (once all branches return).","title":"Abort"},{"location":"example.html","text":"","title":"Example ·"},{"location":"example.html#example","text":"The example app is a dummy API for managing bank accounts and transferring amounts between accounts. Accounts are implemented with sharded entities, and transfers with transactions.\nThe example app can be found in the example module and can be run directly with sbt run.\nStress-test The module also contains a multi-jvm test that demonstrates the resilience of the system, by transferring amounts between 1000 accounts while restarting nodes.","title":"Example"},{"location":"example.html#api","text":"The API has a simple CRUD for accounts and a dedicated endpoint to perform transfers between two accounts.\nHttpRoutes.of[IO] {\n case GET -> Root / \"account\" / id / \"balance\" => server.balanceFor(AccountID(id))\n case POST -> Root / \"account\" / id => server.open(AccountID(id))\n case POST -> Root / \"account\" / id / \"deposit\" / IntVar(amount) => server.deposit(AccountID(id), amount)\n case POST -> Root / \"account\" / id / \"withdraw\" / IntVar(amount) => server.withdraw(AccountID(id), amount)\n case POST -> Root / \"account\" / origin / \"transfer\" / \"to\" / destination / IntVar(amount) => server.transfer(AccountID(origin), AccountID(destination), PosAmount(amount))\n }\n .orNotFound\nThe HTTP server makes uses of the Accounts and Account algebras to satisfy the requests.","title":"API"},{"location":"example.html#algebras","text":"","title":"Algebras"},{"location":"example.html#accounts","text":"This trait represents the repository, i.e. the ability to retrieve a handle on a specific account and orchestrate transfers between accounts.\ntrait Accounts[F[_]] \n def accountFor(name: AccountID): Account[F]\n def transfer(from: AccountID, to: AccountID, amount: PosAmount): F[TransferFailure \\/ Unit]","title":"Accounts"},{"location":"example.html#account","text":"This trait represents the account entity, i.e. the ability to query and perform operations on a specific account.\ntrait Account[F[_]] \n def open: F[AlreadyExists.type \\/ Unit]\n def balance: F[Unknown.type \\/ NonNegAmount]\n def deposit(amount: PosAmount): F[Unknown.type \\/ PosAmount]\n def withdraw(amount: PosAmount): F[WithdrawFailure \\/ NonNegAmount]\n\n def prepareOutgoingTransfer(id: TransferID, transfer: Transfer): F[WithdrawFailure \\/ Unit]\n def prepareIncomingTransfer(id: TransferID, transfer: Transfer): F[IncomingTransferFailure \\/ Unit]\n def commitTransfer(id: TransferID): F[TransferFailure \\/ Unit]\n def abortTransfer(id: TransferID): F[TransferFailure \\/ Unit]\nThe latter four methods are of particular interest as they are used by the transactional branch logic to carry out the various phases of the transfer.","title":"Account"},{"location":"example.html#transfers","text":"Transfers are implemented using an endless-transaction transaction coordinator.","title":"Transfers"},{"location":"example.html#coordinator","text":"A coordinator is created with TransferID as the transaction identifier type, AccountID as the branch identifier, Transfer as the query payload (the amount, origin, and destination), and TransferFailure as the error coproduct.\ncopysourcetransactor.coordinator[TransferID, AccountID, Transfer, TransferFailure](\n \"transfer\",\n { accountID =>\n val account = sharding.entityFor(accountID)\n new TransferBranch(accountID, account)\n },\n Some(transferParameters.timeout)\n)\nThe coordinator is used in the implementation of the transfer method in ShardedAccounts, i.e. the implementation of Accounts.","title":"Coordinator"},{"location":"example.html#transaction","text":"The snippet below shows the logic: the code creates a transfer with two branches, one for the origin account, and the other for the destination account.\ncopysourcedef transfer(from: AccountID, to: AccountID, amount: PosAmount): F[TransferFailure \\/ Unit] =\n coordinator\n .create(TransferID.random, Transfer(from, to, amount), from, to)\n .use(_.pollForFinalStatus())\n .flatMap {\n case Status.Committed => ().asRight[TransferFailure].pure\n case Status.Aborted(reason) =>\n reason match {\n case AbortReason.Timeout =>\n EitherT.leftT(TransferFailure.Timeout: TransferFailure).value\n case AbortReason.Branches(reasons) => EitherT.leftT(reasons.head).value\n case AbortReason.Client(Some(reason)) => EitherT.leftT(reason).value\n case AbortReason.Client(None) =>\n new Exception(\"Transaction aborted by client without justification\")\n .raiseError[F, TransferFailure \\/ Unit]\n }\n case Status.Failed(errors) =>\n Logger[F].error(show\"Transaction failed: $errors\") *> new Exception(\n \"Transaction failed due to branch error\"\n ).raiseError[F, TransferFailure \\/ Unit]\n }\nThe transaction entity is sharded, so it can be running on a separate node. We therefore need to regularly check for completion so that we can respond to the API call (it’s synchronous here for the sake of simplicity).\nTo implement this polling operation, we use the pollForFinalStatus() built-in extension method (defined for Transaction): this method retrieves the status of the transaction at configurable intervals and semantically sleeps in between.","title":"Transaction"},{"location":"example.html#branch","text":"An account’s involvement in a transfer is described by TransferBranch, as exemplified below with the implementation of the prepare method:\ncopysourceclass TransferBranch[F[_]: Logger](accountID: AccountID, account: Account[F])(implicit\n retryParameters: TransferParameters.BranchRetryParameters,\n temporal: Temporal[F]\n) extends Branch[F, TransferID, Transfer, TransferFailure] {\n import temporal.*\n private implicit val onErrorRetryParameters: RetryParameters = retryParameters.onError\n\n def prepare(transferID: TransferID, transfer: Transfer): F[Branch.Vote[TransferFailure]] = {\n if (accountID === transfer.origin)\n Logger[F].debug(\n show\"Preparing outgoing transfer $transferID: $transfer for account $accountID\"\n ) >>\n account\n .prepareOutgoingTransfer(transferID, transfer)\n .onErrorRetryWithBackoff(\n Logger[F]\n .warn(_)(show\"Error preparing outgoing transfer $transferID, retrying in a bit\")\n )\n .onLeftRetryWithBackoff { case Account.PendingOutgoingTransfer =>\n Logger[F].warn(\n show\"Account $accountID has a pending outgoing transfer, retrying in a bit\"\n )\n }(retryParameters.onPendingTransfer)\n .flatMap {\n case Left(Account.Unknown) =>\n Branch.Vote.Abort(TransferFailure.AccountNotFound(accountID)).pure[F]\n case Left(InsufficientFunds(missing)) =>\n Branch.Vote.Abort(TransferFailure.InsufficientFunds(missing)).pure[F]\n case Left(Account.PendingOutgoingTransfer) =>\n Branch.Vote.Abort(TransferFailure.OtherPendingTransfer).pure[F]\n case Right(_) => Branch.Vote.Commit.pure[F]\n }\n else\n Logger[F].debug(show\"Preparing incoming $transferID: $transfer for account $accountID\") >>\n account\n .prepareIncomingTransfer(transferID, transfer)\n .onErrorRetryWithBackoff(\n Logger[F]\n .warn(_)(show\"Error preparing incoming transfer $transferID, retrying in a bit\")\n )\n .map {\n case Left(Account.Unknown) =>\n Branch.Vote.Abort(TransferFailure.AccountNotFound(accountID))\n case Right(_) => Branch.Vote.Commit\n }\n }","title":"Branch"},{"location":"reference.html","text":"","title":"Reference ·"},{"location":"reference.html#reference","text":"Lib API documentation Akka Runtime API documentation Pekko Runtime API documentation","title":"Reference"}]} \ No newline at end of file diff --git a/transaction.html b/transaction.html index fc4a213..aedd455 100644 --- a/transaction.html +++ b/transaction.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@
@@ -224,13 +224,13 @@

-0.0.3 +0.1.0
@@ -274,7 +274,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme
diff --git a/transactor.html b/transactor.html index 81e063e..f192894 100644 --- a/transactor.html +++ b/transactor.html @@ -8,7 +8,7 @@ - + @@ -160,7 +160,7 @@
@@ -191,13 +191,13 @@

+ Edit this page

-0.0.3 +0.1.0
@@ -241,7 +241,7 @@

Paradox and -Paradox Material Theme +Paradox Material Theme