当前位置:网站首页>How to use the Outbox mode to realize saga choreography of micro services

How to use the Outbox mode to realize saga choreography of micro services

2021-07-16 17:13:13 InfoQ

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":" Key points "}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Saga Be able to run for a long time 、 Distributed business transactions , Such a transaction performs a set of operations across multiple microservices , Realize the consistent semantics of all or nothing ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In order to decouple , Communication between microservices is best done asynchronously , For example, with the help of Apache Kafka Using distributed commit logs ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The Outbox pattern provides a solution for service authors , They can write to the local database , At the same time through Apache Kafka Send a message , Avoid relying on unsafe “ Double write (dual writes)”."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Debezium Is a distributed open source data change capture platform , To use the layout of the Outbox mode Saga Streaming provides a robust and flexible foundation ."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" When it comes to micro Services , The first thing we realize is that no single service exists in isolation . Although our goal is to create loose coupling 、 Independent service , The less interaction between them, the better , But it's very likely that one service needs the data set held by another service , Or multiple services need to work together to achieve consistent operational results in the business domain ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" With the help of "},{"type":"link","attrs":{"href":"https:\/\/debezium.io\/blog\/2019\/02\/19\/reliable-microservices-data-exchange-with-the-outbox-pattern\/","title":"","type":null},"content":[{"type":"text","text":" Change data capture "}]},{"type":"text","text":" The implementation of the Outbox mode is an effective way to solve the problem of data exchange between microservices , This model can avoid the damage to multiple resources ( Such as database and message agent ) It's not safe “ Double write ”, So as to achieve the final consistent data exchange , In this process, we don't rely on the synchronization availability of all participants , There's no need for complex protocols , Such as XA( from "},{"type":"link","attrs":{"href":"https:\/\/www.opengroup.org\/","title":"","type":null},"content":[{"type":"text","text":"The Open Group"}]},{"type":"text","text":" The definition is widely used in distributed transaction processing "},{"type":"link","attrs":{"href":"https:\/\/pubs.opengroup.org\/onlinepubs\/009680699\/toc.pdf","title":"","type":null},"content":[{"type":"text","text":" standard "}]},{"type":"text","text":")."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In this paper , I will explore how to further use the Outbox mode , That is to use it to realize Saga, That is, long-running transactions that may span multiple microservices . A common example is booking a multi part itinerary : Or all flights and accommodations are reserved , Or cancel all the reservations .Saga Divide such a whole business transaction into a series of local database transactions , These transactions are executed in related services ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Saga introduction "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In case of failure “ Roll back ” Overall business affairs ,Saga The idea of relying on compensation matters : Every local transaction that has been applied before must be able to run another transaction “ revoke ”, This transaction cancels changes that have been completed before ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Saga Not at all "},{"type":"link","attrs":{"href":"https:\/\/www.infoq.com\/articles\/History-of-Extended-Transactions\/","title":"","type":null},"content":[{"type":"text","text":" What new concept "}]},{"type":"text","text":": As early as 1987 year ,Hector Garcia-Molina and Kenneth Salem In their SIGMOD "},{"type":"link","attrs":{"href":"https:\/\/www.cs.cornell.edu\/andru\/cs711\/2002fa\/reading\/sagas.pdf","title":"","type":null},"content":[{"type":"text","text":"Sagas"}]},{"type":"text","text":" This idea is discussed for the first time in this paper . however , Under the background of the continuous evolution of the industry to the microservice Architecture ,Saga As a solution supported by local transactions in related services, it is more and more popular , For example, currently in active development "},{"type":"link","attrs":{"href":"https:\/\/github.com\/eclipse\/microprofile-lra","title":"","type":null},"content":[{"type":"text","text":" A long-running operation MicroProfile standard "}]},{"type":"text","text":", These questions are usually "},{"type":"link","attrs":{"href":"https:\/\/www.theserverside.com\/news\/1365143\/ACID-is-Good-Take-it-in-Short-Doses","title":"","type":null},"content":[{"type":"text","text":" Out of commission ACID Semantics to solve "}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" To make the statement more specific , Let's consider an example of an e-commerce business , It's made up of three services : Order 、 Consumers and payments . When a new purchase order is submitted to the order service , The following process will be performed , It includes two other services :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/31\/01\/3157f599243f365a8d783dc7106ae801.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 1: Order status transition "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" First , We need to check whether the incoming order matches the consumer's credit limit through the consumer service ( Because we don't want the user's pending orders to exceed a certain threshold ). If the consumer's credit limit is 500 dollar , The new order is 300 dollar , Then this order is in line with the current limit , The remaining amount will become 200 dollar . If there's another one after that 259 Orders in US dollars , Then it will be rejected accordingly , Because it's beyond the credit limit that consumers are currently open to ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" If the credit limit check is successful , Then you need to pay for the order through the payment service application . If both the credit limit check and the payment request are successful , The order will be transferred to "},{"type":"codeinline","content":[{"type":"text","text":"Accepted"}]},{"type":"text","text":" state , So that the delivery of the order can begin ( This step is not in the process we are discussing here )."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" however , If the credit limit check fails , The order will be transferred immediately to "},{"type":"codeinline","content":[{"type":"text","text":"Rejected"}]},{"type":"text","text":" state . If this step succeeds , But subsequent payment requests failed , Transfer the order to "},{"type":"codeinline","content":[{"type":"text","text":"Rejected"}]},{"type":"text","text":" Before status , You need to release the credit line allocated in the previous step ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":" Optional implementation "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In the implementation of distributed Saga When , There are two common ways , It's collaborative (choreography) And choreography (orchestration). In Collaborative Saga in , Each participating service sends a message to the next service after it has completed the local transaction . And in choreography Saga in , There will be a coordination service , It calls each service involved one by one ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Both ways have their advantages and disadvantages ( Please see the Chris Richardson Of "},{"type":"link","attrs":{"href":"https:\/\/chrisrichardson.net\/post\/sagas\/2019\/08\/04\/developing-sagas-part-2.html","title":"","type":null},"content":[{"type":"text","text":" Blog posts "}]},{"type":"text","text":" as well as Yves do Régo Of "},{"type":"link","attrs":{"href":"https:\/\/medium.com\/@ydorego\/microservices-orchestration-vs-choreography-the-eternal-saga-d58c35e07d81","title":"","type":null},"content":[{"type":"text","text":" article "}]},{"type":"text","text":" For a more detailed discussion ). As far as I'm concerned , I prefer choreography , Because it defines a central point ( Choreographer , Otherwise known as “Saga Executive Coordinator ”, abbreviation SEC), Through it, we can get specific Saga Current state . It can avoid point-to-point communication between various participants ( Except for the choreographer ), It also allows additional intermediate steps to be added to the process , There is no need to adjust each participant in the process ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In depth to achieve this Saga Before the process , We need to spend some time discussing Saga The transaction semantics provided . Let's take a look first Saga How to satisfy the four classics of transaction ACID attribute , This is a Theo Härder and Andreas Reuter( be based on Jim Gray Earlier "},{"type":"link","attrs":{"href":"http:\/\/jimgray.azurewebsites.net\/papers\/thetransactionconcept.pdf","title":"","type":null},"content":[{"type":"text","text":" Results of work "}]},{"type":"text","text":") In their basic thesis "},{"type":"link","attrs":{"href":"https:\/\/citeseerx.ist.psu.edu\/viewdoc\/summary?doi=10.1.1.115.8124","title":"","type":null},"content":[{"type":"text","text":"Principles of Transaction-Oriented Database Recovery"}]},{"type":"text","text":" As defined in :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Atomicity (Atomicity): This pattern ensures that all services either apply local transactions , Or when something goes wrong , All executed local transactions are compensated , So there won't be any valid data changes ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Uniformity (Consistency): In the successful execution of the composition Saga After all the business of , All local constraints are guaranteed to be met , So that the whole system from a consistent state to another consistent state ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Isolation, (Isolation): Even though Saga There is a possibility of eventual failure , This will cause all previously executed transactions to be compensated , But in view of Saga During the operation of , The local transaction has been committed , So their changes are already visible to other concurrent transactions , let me put it another way , From the whole Saga From the perspective of , The isolation level can be compared to ” Read uncommitted data (read uncommitted)“."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" persistence (Durability): once Saga Local transactions are committed , Their changes will persist , And it will last forever , Such as being able to experience service failure or restart ."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" From the perspective of service consumers , For example, a user submits a purchase order through the order service , The system is ultimately consistent , in other words , According to the logic of different services involved , It takes a certain amount of time for the purchase order to be in the correct state ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" As for communication between the services involved , It can be synchronous , Such as through HTTP or "},{"type":"link","attrs":{"href":"https:\/\/grpc.io\/","title":"","type":null},"content":[{"type":"text","text":"gRPC"}]},{"type":"text","text":", It can also be done asynchronously , For example, through message broker or distributed log , Such as "},{"type":"link","attrs":{"href":"https:\/\/kafka.apache.org\/","title":"","type":null},"content":[{"type":"text","text":"Apache Kafka"}]},{"type":"text","text":". Whenever possible , We should give priority to asynchronous communication between services , Because it combines the availability of sending services with that of consuming services . As we see in the next section , With the help of change data capture , even Kafka Its usability is no longer a problem ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":" Review the Outbox mode "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" that , Outbox mode and change data capture ( from "},{"type":"link","attrs":{"href":"https:\/\/debezium.io\/","title":"","type":null},"content":[{"type":"text","text":"Debezium"}]},{"type":"text","text":" Provide ) How does it all come together ? As mentioned earlier ,Saga It is better for coordinator to communicate with related services asynchronously through request and reply message channels .Apache Kafka It's a very popular option to implement this kind of channel . however , Choreographer ( And every service involved ) You also need to apply the transaction to its specific database , So that the entire Saga The part of the stream that belongs to them ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Although it's easy to perform a database transaction , And send a corresponding message to Kafka It's a very tempting approach , But it's not a good idea . These two actions span databases and Kafka, So it's not done in the same transaction . Sooner or later, we will encounter the problem of inconsistency , For example, the database transaction has been committed , But write to Kafka The process failed . however ,"},{"type":"link","attrs":{"href":"https:\/\/speakerdeck.com\/gunnarmorling\/practical-change-data-streaming-use-cases-with-apache-kafka-and-debezium-qcon-san-francisco-2019?slide=10","title":"","type":null},"content":[{"type":"text","text":" Good friends don't let their friends do double writing "}]},{"type":"text","text":", The Outbox model provides a very elegant way to solve this problem :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/53\/96\/53e86e8a829bc082c703f3a038411196.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 2: Safely update the database and send messages to Kafka"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We don't send messages directly after updating the data , Instead, let the service perform normal updates based on the same transaction and insert messages into a specific outbox table in the database . Because this operation is done in the same database transaction , We'll have two results , Either changes to the service model will be persisted and messages can be safely saved to the Outbox table , Or neither of them will be implemented . After the transaction is written to the transaction log of the database ,Debezium This is where the data change capture process gets messages from the Outbox , And send it to Apache Kafka."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" This is by using “ At least once (at-least-once)” The semantic implementation of : Under certain circumstances , The same outbox message may be sent multiple times to Kafka in . In order for consumers to detect and ignore duplicate messages , Every message should have a unique id. for example , It can be a UUID, It can also be a monotonically increasing sequence , This is closely related to every message producer , This id It should be through Kafka The header information of the message is transmitted ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":" Through the Outbox mode Saga"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" When the Outbox mode in the toolbox is ready , The next thing is clearer . Order service will serve as Saga The coordinator , After receiving a request to place an order ( Usually through REST API Realization ), It will update the local state ( Including persistent order model and Saga Execution log ) To trigger the whole process , And send messages to the other two participating services in turn ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" These two services are through Kafka Respond to received messages , Execute local transactions to update their data status and send a reply message to the coordinator through their own outbox table . The whole solution looks like this :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/2e\/df\/2ed6d0284d82a6b3e3d67c9c14b2ecdf.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 3: Using outbox mode Saga layout "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" stay Debezium Of GitHub"},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga","title":"","type":null},"content":[{"type":"text","text":" Sample Repository "}]},{"type":"text","text":" in , You can see the complete proof of concept for this architecture (proof-of-concept,PoC) Realization . The main components of the architecture are as follows :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Three services , Namely "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga\/order-service","title":"","type":null},"content":[{"type":"text","text":" Order "}]},{"type":"text","text":"( Used to manage purchase orders and as Saga The coordinator )、"},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga\/customer-service","title":"","type":null},"content":[{"type":"text","text":" consumer "}]},{"type":"text","text":"( Credit restrictions to manage consumers ) and "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga\/payment-service","title":"","type":null},"content":[{"type":"text","text":" payment "}]},{"type":"text","text":"( To handle credit card payments ), Each service has its own database (Postgres)."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Apache Kafka As the backbone of message transmission "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Debezium Running on the Kafka Connect above , It subscribes to changes in these three different databases , And pass Debezium Of "},{"type":"link","attrs":{"href":"https:\/\/debezium.io\/documentation\/reference\/configuration\/outbox-event-router.html","title":"","type":null},"content":[{"type":"text","text":" Outbox event routing (outbox event routing)"}]},{"type":"text","text":" Component sends them to the corresponding Kafka The theme ."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" These three services are using "},{"type":"link","attrs":{"href":"https:\/\/quarkus.io\/","title":"","type":null},"content":[{"type":"text","text":"Quarkus"}]},{"type":"text","text":" Realized , This is a technology stack for building cloud native microservices , The built application can run in JVM On , It can also be compiled into native binary ( adopt GraalVM Realization ). Of course , This pattern can also be implemented by other technology stacks or languages , As long as they provide consumption from Kafka The ability to write messages to the database . in addition , It's also possible to combine different implementation techniques ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Here are four Kafka The theme : Request and response topics of credit approval messages and payment messages . stay Saga In the case of successful execution , Just four messages will be exchanged . If one of these steps fails , A compensation transaction is needed , At each step, there are additional request and response message pairs to compensate ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":" Make sure the order "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In order to expand ,Kafka Topics can be organized into multiple partitions ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Only within a partition , To ensure that the order in which consumers receive messages is exactly the same as the order in which producers send messages . By default , Have the same key All messages in the same partition are sent , therefore Saga The only id yes Kafka news key The natural selection of . In this way , The same Saga Instance messages are guaranteed to be processed in the correct order ."}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" If we have more than one Saga example , They are used for Saga The topic of message exchange appears in different partitions , So they can be processed in parallel ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/6b\/83\/6b3735535c9881d6a35090f592742c83.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 4: success Saga The execution sequence of the stream "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Each service sends messages through the Outbox table in its own database . ad locum , The news is from Debezium Capture and send to Kafka, It is ultimately consumed by the service that receives the message . When sending and sending messages , Order service as a choreographer will also Saga Persistence of the progress to the local state table ( Later, ). in addition , All participants will share the information they consume id A record of journal In the table , So as to identify the possible repetition in the future ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" that , Let's think about it now , What happens if a step in the flow fails ? Suppose the payment step fails because the consumer's credit card has expired . under these circumstances , In the previous consumer service, the reserved credit card limit needs to be released again . To achieve this , The order service sends a compensation request to the consumer service . Scale it up a little bit ( It's like what I said before Debezium and Kafka That's the details ), Then the message exchange will look like this :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/89\/d8\/89921c2f16b35bf3808a70214584cdd8.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 5: With compensation Saga The execution sequence of the stream "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" After discussing the message flow between services , Next, let's go into some implementation details of order service . The proof of concept implementation provides a general purpose in the form of a simple state machine Saga Choreographer and for order scenarios Saga Realization , We will discuss it in depth later . The realization of order service ” frame “ Part in "},{"type":"codeinline","content":[{"type":"text","text":"sagastate"}]},{"type":"text","text":" The table tracks Saga The current state of execution , The pattern is as follows :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/0a\/85\/0a958bf73091926c7ff113c9fd122785.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 6:Saga The pattern of the state table "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" This table satisfies Saga The requirements of the log . Each of its columns is as follows :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"id"}]},{"type":"text","text":": Given Saga Unique identifier of the instance , Represents the creation of a specific purchase order ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"currentStep"}]},{"type":"text","text":":Saga The current steps , Such as “credit-approval” or “payment”."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"payload"}]},{"type":"text","text":": With specific Saga Any data structure associated with an instance , for example , stay Saga In the life cycle , Containing the corresponding purchase order id And other useful information ; Although in the sample implementation we use JSON As the format of the load , But you can also consider using other formats , such as "},{"type":"link","attrs":{"href":"https:\/\/avro.apache.org\/","title":"","type":null},"content":[{"type":"text","text":"Apache Avro"}]},{"type":"text","text":", And store the load mode in the mode registry ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"status"}]},{"type":"text","text":":Saga Current state , It can be "},{"type":"codeinline","content":[{"type":"text","text":"STARTED"}]},{"type":"text","text":"、"},{"type":"codeinline","content":[{"type":"text","text":"SUCCEEDED"}]},{"type":"text","text":"、"},{"type":"codeinline","content":[{"type":"text","text":"ABORTING"}]},{"type":"text","text":" or "},{"type":"codeinline","content":[{"type":"text","text":"ABORTED"}]},{"type":"text","text":" One of them ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"stepState"}]},{"type":"text","text":": After stringing JSON structure , Describes the status of each step , such as \"{\"credit-approval\":\"SUCCEEDED\",\"payment\":\"STARTED\"}\""}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"type"}]},{"type":"text","text":":Saga Named types , such as “order-placement”, It is used to distinguish the different types of Saga."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"version"}]},{"type":"text","text":": A version based on optimistic locking , Used to detect and reject a Saga Concurrent updates of instances ( under these circumstances , Messages that trigger failed updates need to be retried , from Saga Reload the current state in the log )"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" When the order service sends the request to the consumer and the payment service through Kafka When you receive their reply ,Saga The status will be updated to this table . By building Debezium connector To keep track of "},{"type":"codeinline","content":[{"type":"text","text":"sagastate"}]},{"type":"text","text":" surface , We can check it very well Kafka in Saga Implementation progress of ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The following shows the status transition of a purchase order that failed to pay , First of all, the order is passed in ,“credit-approval” Step start :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"{\n \"id\": \"73707ad2-0732-4592-b7e2-79b07c745e45\",\n \"currentstep\": null,\n \"payload\": \"\\\"order-id\\\": 2, \\\"customer-id\\\": 456, \\\"payment-due\\\": 4999, \\\"credit-card-no\\\": \\\"xxxx-yyyy-dddd-9999\\\"}\",\n \"sagastatus\": \"STARTED\",\n \"stepstatus\": \"{}\",\n \"type\": \"order-placement\",\n \"version\": 0\n}\n{\n \"id\": \"73707ad2-0732-4592-b7e2-79b07c745e45\",\n \"currentstep\": \"credit-approval\",\n \"payload\": \"{ \\\"order-id\\\": 2, \\\"customer-id\\\": 456, ... }\",\n \"sagastatus\": \"STARTED\",\n \"stepstatus\": \"{\\\"credit-approval\\\": \\\"STARTED\\\"}\",\n \"type\": \"order-placement\",\n \"version\": 1\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" here , One “credit-approval” The request message is also persisted to the Outbox table . Message sent to Kafka after , The consumer service will process it and send a reply message . The order service will be updated Saga Status and start the payment step :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"{\n \"id\": \"73707ad2-0732-4592-b7e2-79b07c745e45\",\n \"currentstep\": \"payment\",\n \"payload\": \"{ \\\"order-id\\\": 2, \\\"customer-id\\\": 456, ... }\",\n \"sagastatus\": \"STARTED\",\n \"stepstatus\": \"{\\\"payment\\\": \\\"STARTED\\\", \\\"credit-approval\\\": \\\"SUCCEEDED\\\"}\",\n \"type\": \"order-placement\",\n \"version\": 2\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The message is sent again through the Outbox table , But now it's “payment” request . If this step fails , The payment system will send a reply message in response , And show what happened . That means “credit-approval” Steps need to be compensated through the consumer system :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"{\n \"id\": \"73707ad2-0732-4592-b7e2-79b07c745e45\",\n \"currentstep\": \"credit-approval\",\n \"payload\": \"{ \\\"order-id\\\": 2, \\\"customer-id\\\": 456, ... }\",\n \"sagastatus\": \"ABORTING\",\n \"stepstatus\": \"{\\\"payment\\\": \\\"FAILED\\\", \\\"credit-approval\\\": \\\"COMPENSATING\\\"}\",\n \"type\": \"order-placement\",\n \"version\": 3\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" When this step is finished ,Saga Will enter the final state , That is to say "},{"type":"codeinline","content":[{"type":"text","text":"ABORTED"}]},{"type":"text","text":":"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"{\n \"id\": \"73707ad2-0732-4592-b7e2-79b07c745e45\",\n \"currentstep\": null,\n \"payload\": \"{ \\\"order-id\\\": 2, \\\"customer-id\\\": 456, ... }\",\n \"sagastatus\": \"ABORTED\",\n \"stepstatus\": \"{\\\"payment\\\": \\\"FAILED\\\", \\\"credit-approval\\\": \\\"COMPENSATED\\\"}\",\n \"type\": \"order-placement\",\n \"version\": 4\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" You can follow the example README In the document "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga#running-the-example","title":"","type":null},"content":[{"type":"text","text":" explain "}]},{"type":"text","text":" Try it yourself , Here you can find create order "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/blob\/master\/saga\/requests\/place-order.json","title":"","type":null},"content":[{"type":"text","text":" success "}]},{"type":"text","text":" and "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/blob\/master\/saga\/requests\/place-invalid-order2.json","title":"","type":null},"content":[{"type":"text","text":" Failure "}]},{"type":"text","text":" Request . It also includes how to check Kafka A guide to exchanging messages in topics , These messages come from the Outbox tables of different services ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Now? , Let's take a look at some of the specific implementations of this use case .Saga The flow is started in the order service , Its REST The endpoint implementation looks like this :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"@POST\n@Transactional\npublic PlaceOrderResponse placeOrder(PlaceOrderRequest req) {\n PurchaseOrder order = req.toPurchaseOrder();\n order.persist(); \n\n sagaManager.begin(OrderPlacementSaga.class, OrderPlacementSaga.payloadFor(order)); \n\n return PlaceOrderResponse.fromPurchaseOrder(order);\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Persistent incoming purchase orders "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Open the order for the incoming order Saga flow "}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"SagaMananger.begin()"}]},{"type":"text","text":" Will be in "},{"type":"codeinline","content":[{"type":"text","text":"sagastate"}]},{"type":"text","text":" Create a new record in the table , adopt "},{"type":"codeinline","content":[{"type":"text","text":"OrderPlacementSaga"}]},{"type":"text","text":" Get the first outbox event and persist it to the Outbox table ."},{"type":"codeinline","content":[{"type":"text","text":"OrderPlacementSaga"}]},{"type":"text","text":" Class to implement Saga All the specific components of the flow associated with the use case , Include :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Used to perform Saga An outbox event for a component of a stream "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" To compensate for Saga An outbox event for a component of a stream "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Event handler , It's used to deal with other Saga Response messages from participants "}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"OrderPlacementSaga"}]},{"type":"text","text":" Implementation is too long , It's not suitable to show it all here ( You can "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/blob\/master\/saga\/order-service\/src\/main\/java\/io\/debezium\/examples\/saga\/order\/saga\/OrderPlacementSaga.java","title":"","type":null},"content":[{"type":"text","text":"GitHub Check out its full code on the "}]},{"type":"text","text":"), But here we show some of the core components :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"@Saga(type=\"order-placement\", stepIds = {CREDIT_APPROVAL, PAYMENT}) 1️⃣\npublic class OrderPlacementSaga extends SagaBase {\n\n private static final String REQUEST = \"REQUEST\";\n private static final String CANCEL = \"CANCEL\";\n protected static final String PAYMENT = \"payment\";\n protected static final String CREDIT_APPROVAL = \"credit-approval\";\n\n \/\/ ...\n @Override\n public SagaStepMessage getStepMessage(String id) { 2️⃣\n if (id.equals(PAYMENT)) {\n return new SagaStepMessage(PAYMENT, REQUEST, getPayload());\n }\n else {\n return new SagaStepMessage(CREDIT_APPROVAL, REQUEST, getPayload());\n }\n }\n\n @Override\n public SagaStepMessage getCompensatingStepMessage(String id) { 3️⃣\n \/\/ ...\n }\n\n public void onPaymentEvent(PaymentEvent event) { 4️⃣\n if (alreadyProcessed(event.messageId)) {\n return;\n }\n\n onStepEvent(PAYMENT, event.status.toStepStatus());\n updateOrderStatus();\n\n processed(event.messageId);\n }\n\n public void onCreditApprovalEvent(CreditApprovalEvent event) { 5️⃣\n \/\/ ...\n }\n\n private void updateOrderStatus() { 6️⃣\n if (getStatus() == SagaStatus.COMPLETED) {\n PurchaseOrder order = PurchaseOrder.findById(getOrderId());\n order.status = PurchaseOrderStatus.ACCEPTED;\n }\n else if (getStatus() == SagaStatus.ABORTED) {\n PurchaseOrder order = PurchaseOrder.findById(getOrderId());\n order.status = PurchaseOrderStatus.CANCELLED;\n }\n }\n\n \/\/ ...\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1️⃣ Saga The steps of the id, Easy to carry out "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2️⃣ Returns the Outbox message to be published for the given step "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3️⃣ Return to the Outbox message to be published in the compensation step "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4️⃣ in the light of “payment” The event handler that replies to the message , It updates the status of the purchase order and Saga The state of ( adopt onStepEvent() Callback implementation ), According to different states , It could be done Saga, It may also start its rollback process by applying all the compensation messages ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"5️⃣ in the light of “credit approval” The event handler that replies to the message "}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"6️⃣ Based on the current Saga state , Update the status of the purchase order "}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"...\n\nthis.outboxEvent.fire(CreditEvent.of(sagaId, CreditStatus.CANCELLED));\n...\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" There's nothing new about consumer services and payment services , So for the sake of brevity , We're going to skip them here . You can "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga\/customer-service","title":"","type":null},"content":[{"type":"text","text":" here "}]},{"type":"text","text":" and "},{"type":"link","attrs":{"href":"https:\/\/github.com\/debezium\/debezium-examples\/tree\/master\/saga\/payment-service","title":"","type":null},"content":[{"type":"text","text":" here "}]},{"type":"text","text":" Check out their full code ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":" What happens if things go wrong "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In the realization of image Saga In such a distributed interaction mode , A key component is understanding how they behave when they fail , And make sure that in unforeseen circumstances , Can also be realized ( Final ) Uniformity ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Be careful ,Saga The negative output of the step ( such as , Payment service refused to pay because of invalid credit card ) It's not the fault scenario here , Because we clearly expect that participants may not be able to perform their part of the overall process , This will cause the corresponding compensation local transaction to be executed . It means , This common participant execution failure should not cause rollback of local database transactions , Because otherwise , There will be no reply messages sent to the choreographer through the Outbox ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" With that in mind , Let's discuss some possible fault scenarios :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Kafka The event handler of the message threw an exception "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The local database transaction was rolled back , And the news consumers didn't tell Kafka Agent confirms (acknowledge) It can process messages . Because the agent does not receive the confirmation that the message has been processed , So after a certain period of time , It repeats the message , Until it's confirmed . We should have surveillance to detect this scenario , Because before the message is processed ,Saga The stream will not continue processing ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Debezium connector Send a outbox message to Kafka And then it collapsed , At this point, the offset has not been committed in the source database transaction log (offset)."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" restart connector after , It will continue to read messages in the Outbox table from where the log offset was last committed , This may cause some outbox events to be sent twice , That's why all participants are required to be idempotent , As in the previous example, by using a unique message id To achieve that , Consumers can also pass journal Table tracks messages that have been successfully processed ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Kafka Not running or not accessible , For example, due to network segmentation ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Debezium connector In the Kafka Restore them to work when they are available again , But before that ,Saga Streams naturally cannot be processed ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":" The message has been processed , But to Kafka The confirmation failed ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" This message will be delivered again to the consumer service , And in consumer service journal The message will be found in the table id, So it's ignored as a duplicate message ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":" Parallel processing of multiple Saga Step by step , Yes Saga Concurrent update of state table "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Although we have discussed how choreographers can form a sequential process by successively triggering participating Services , But we should also imagine parallel processing of multiple steps Saga Realization . under these circumstances , Concurrent replies may compete for updates Saga The status table of . This scenario is detected by the optimistic lock on the table , This will cause the event handler to attempt to submit an update to an obsolete Saga Status version , And that leads to failure 、 Rollback and retry ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We can talk about more , But the semantics of the overall design is the final consistent system , Be able to guarantee at least one execution ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":" Extra benefits : Distributed tracking "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" When designing event flows between distributed systems , Insight in operation and maintenance is essential to ensure that everything runs correctly and efficiently . Distributed tracking provides this insight : It collects tracking information for each system , These systems contribute such interactive information , And allow the call flow to be checked , For example Web UI In the form of , This makes it a valuable tool for fault analysis and debugging ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Debezium The Outbox is connected with "},{"type":"link","attrs":{"href":"https:\/\/opentracing.io\/","title":"","type":null},"content":[{"type":"text","text":"OpenTracing"}]},{"type":"text","text":"( Yes "},{"type":"link","attrs":{"href":"https:\/\/opentelemetry.io\/","title":"","type":null},"content":[{"type":"text","text":"OpenTelemetry"}]},{"type":"text","text":" Our support is already on the road map ) The tight integration of norms solves this problem . adopt "},{"type":"link","attrs":{"href":"https:\/\/jaegertracing.io\/","title":"","type":null},"content":[{"type":"text","text":"Jaeger"}]},{"type":"text","text":" Such a tool , Just a few "},{"type":"link","attrs":{"href":"https:\/\/debezium.io\/documentation\/reference\/integrations\/tracing.html","title":"","type":null},"content":[{"type":"text","text":" To configure "}]},{"type":"text","text":", Can collect orders 、 Tracking information about consumers and payment services , And show them as end-to-end tracking results ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/44\/06\/4471819160de1a5f731c41e824ea2f06.jpg","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":" chart 7:Saga The flow of Jaeger UI"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Jaeger The visualization in shows us very well Saga How the flow is passed in through the order service REST request (1) The trigger , Outbox messages are sent to consumer services (2) And send it back to the order service (3), Then another message is sent to the payment service (4) And finally send it back to the order service (5)."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" With the help of tracking , We can easily identify unfinished flows ( for example , Because an event handler participating in the service failed to process a message ) And performance bottlenecks ( for example , An event handler takes an unreasonable amount of time to complete Saga The part of the stream that belongs to itself )."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":" Summary and prospect "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Saga The mode is to achieve long running ” Business affairs “ Provides a powerful and flexible solution , This requires multiple independent services to agree on whether to apply or abandon a set of data changes ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" With the help of CDC、Debezium and Apache Kafka The sender mode of implementation ,Saga Choreographers can decouple from the availability of all participating Services . Temporary interruption of a single participating service does not affect the overall Saga flow : After component recovery ,Saga It will continue from where it was interrupted ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Of course , We should expect services to be separated from each other , Minimize the need to interact with remote services . for example , Transfer the logic of credit line to order service itself , Avoid collaboration with consumer services , It may also be an option . however , According to the needs of the business , This kind of interaction across multiple services can be unavoidable , Especially when it comes to integrating legacy systems , Or the system is out of our control ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In the realization of image Saga In such a complex pattern , It is crucial to understand their constraints and semantics accurately . In the context of our proposed solution , There are two things to pay attention to , That's the inherent ultimate consistency and the limited isolation level of overall business transactions . for example , Because an order allocates part of the credit limit to the consumer, another order submitted at the same time may be rejected , But the first order may not be completed in the end ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The sample project discussed in this article is based on CDC And the Outbox pattern provides a proof of concept level Saga Choreography implementation , It's organized into two parts :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" A generic “ frame ” Components , It provides... In the form of a simple state machine Saga The logic of choreography , It also provides Saga Execution log ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We are talking about the concrete implementation of placing an order ( As shown above "},{"type":"codeinline","content":[{"type":"text","text":"OrderPlacementSaga"}]},{"type":"text","text":" Class and related REST Endpoint, etc )."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Further , We may extract the previous part into a reusable component , For example, through the existing Debezium Quarkus Extended implementation . If you are interested in it , Can pass Debezium Of "},{"type":"link","attrs":{"href":"https:\/\/groups.google.com\/g\/debezium","title":"","type":null},"content":[{"type":"text","text":" Mailing list, "}]},{"type":"text","text":" Contact us . One possible added feature is the concurrent execution of multiple Saga Step by step . Whether this is reasonable or not is a business decision , But from a technical point of view , It's not hard to support it . under these circumstances , to update Saga Competition in state can be a key issue ,"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/particular.net\/blog\/optimizations-to-scatter-gather-sagas","title":"","type":null},"content":[{"type":"text","text":" Dispersed - Gather Saga The optimization of the "}]},{"type":"text","text":" This article discusses possible solutions in this area . If there is a facility to monitor and identify those that have not been completed in a period of time Saga, It's also very interesting ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Our proposed implementation provides a reliable way to execute business , Can be implemented across multiple services ” All or nothing “ The semantics of the . For use cases with more complex requirements , For example, processes with conditional logic , Then you can learn about the existing workflow engines and business processing automation tools , such as "},{"type":"link","attrs":{"href":"https:\/\/kogito.kie.org\/","title":"","type":null},"content":[{"type":"text","text":"Kogito"}]},{"type":"text","text":". Another interesting technology is for "},{"type":"link","attrs":{"href":"https:\/\/github.com\/eclipse\/microprofile-lra","title":"","type":null},"content":[{"type":"text","text":" Long running activities (long-running activities,LRA) Of MicroProfile standard "}]},{"type":"text","text":", The specification is currently under development .MicroProfile The community is also talking about "},{"type":"link","attrs":{"href":"https:\/\/github.com\/eclipse\/microprofile-lra\/issues\/338","title":"","type":null},"content":[{"type":"text","text":" And Debezium This kind of transactional outbox realizes the integration of "}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Thank you very much "},{"type":"link","attrs":{"href":"https:\/\/twitter.com\/hpgrahsl","title":"","type":null},"content":[{"type":"text","text":"Hans-Peter Grahsl"}]},{"type":"text","text":"、"},{"type":"link","attrs":{"href":"https:\/\/github.com\/roldanbob","title":"","type":null},"content":[{"type":"text","text":"Bob Roldan"}]},{"type":"text","text":"、"},{"type":"link","attrs":{"href":"https:\/\/twitter.com\/nmcl","title":"","type":null},"content":[{"type":"text","text":"Mark Little"}]},{"type":"text","text":" and "},{"type":"link","attrs":{"href":"https:\/\/twitter.com\/ThomasBetts","title":"","type":null},"content":[{"type":"text","text":"Thomas Betts"}]},{"type":"text","text":" A lot of feedback in writing this article !"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Check the English text :"},{"type":"link","attrs":{"href":"https:\/\/www.infoq.com\/articles\/saga-orchestration-outbox\/","title":"","type":null},"content":[{"type":"text","text":"Saga Orchestration for Microservices Using the Outbox Pattern"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":" The authors introduce "},{"type":"text","text":":"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Gunnar Morling Is a software engineer , Passionate open source enthusiasts . He's leading "},{"type":"link","attrs":{"href":"https:\/\/debezium.io\/","title":"","type":null},"content":[{"type":"text","text":"Debezium"}]},{"type":"text","text":" project , This is a data capture for change (CDC) Tools for . He is a Java Champion, yes "},{"type":"link","attrs":{"href":"https:\/\/beanvalidation.org\/2.0\/","title":"","type":null},"content":[{"type":"text","text":"Bean Validation 2.0(JSR 380)"}]},{"type":"text","text":" The person in charge of the specification of , And created a number of open source projects , Such as "},{"type":"link","attrs":{"href":"https:\/\/github.com\/moditect\/layrry","title":"","type":null},"content":[{"type":"text","text":"Layrry"}]},{"type":"text","text":"、"},{"type":"link","attrs":{"href":"https:\/\/github.com\/moditect\/deptective","title":"","type":null},"content":[{"type":"text","text":"Deptective"}]},{"type":"text","text":" and "},{"type":"link","attrs":{"href":"https:\/\/mapstruct.org\/","title":"","type":null},"content":[{"type":"text","text":"MapStruct"}]},{"type":"text","text":". Prior to joining Red Hat Before ,Gunnar Widely engaged in logistics and retail industry Java EE Related items . He works in Hamburg, Germany . You can reach him on twitter :"},{"type":"link","attrs":{"href":"https:\/\/twitter.com\/gunnarmorling","title":"","type":null},"content":[{"type":"text","text":"@gunnarmorling"}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":" Link to the original text "},{"type":"text","text":":"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.zybuluo.com\/levinzhang\/note\/1790212","title":"","type":null},"content":[{"type":"text","text":" Using the Outbox mode to realize the micro service Saga layout "}]}]}]}

版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢
https://chowdera.com/2021/07/20210716164837903p.html

随机推荐