Microservice - how to limit and fuse service current

### One 、 Service avalanche ​ Microservice architecture is to divide a single application into a variety of small, wired Services , Each service completes a single business function . Compared with the traditional single service , Microservices are isolated 、 Technology heterogeneity 、 Extensible suite and simplified deployment . Usually, an application consists of multiple microservices , The data interaction between micro services needs to be completed by remote call . ​ The following figure shows a scenario in which microservices call each other ：  ​ Microservices A Call microservices B、C and D, Microservices C Call microservices again E. Suppose a moment , Microservices E Become unavailable . Microservices C Need to wait for microservices E Return the result , So requests will gradually pile up in microservices C, Form a blockage . With microservices C The pile of requests is increasing , Microservices A And it's going to get clogged up . Because the server can support a limited number of concurrent , So it will eventually run out of server resources , As a result, more micro services in the call chain are not available , Forming an avalanche effect . In this case, the whole service chain will collapse due to the collapse of one service , We call it the service avalanche . ​ In the microservice Architecture , There are usually two situations that can lead to service avalanches ： 1. The sudden increase in traffic , Beyond the service processing limit 2. A service in the call chain fails or responds slowly （ Delay ） In view of the above two situations , A corresponding solution has been created ： Service current limiting and service fusing . ### Two 、 Service flow restriction ​ Service flow limiting refers to limiting the number of service requests in a certain period of time to protect the system , It is mainly used to prevent service crash caused by sudden traffic , Like the second kill 、 Rush to buy 、 Double eleven, etc , It can also be used for safety purposes , For example, dealing with external violence . ​ The common current limiting algorithms are as follows ： - ** Counter algorithm ** ​ By maintaining an internal counter , Accumulate service requests over a period of time , Determine whether the counter reaches the preset threshold . If the threshold is not reached , Allow the request to pass through , And the counter adds 1; If the threshold is reached , Then refuse the service , Abandon the request . After entering the next timing cycle , The counter is cleared , Re count . ​ Counter algorithm is the simplest algorithm in current limiting algorithm , The disadvantage is that there is a spike phenomenon , Not only is the rate of request passing uneven , And the request output rate is not uniform , The concurrency requirement for subsequent processing is relatively high . such as ： Set the service cycle to 1 second , The upper threshold of the request is 1000. If the former 1ms I have received 1000 A request , Then the rest of the time can only refuse , And the back-end service needs concurrent processing 1000 A request . - ** Leaky bucket algorithm ** ​ The principle of leaky bucket algorithm can be understood in this way , Think of a service request as water flowing into a leaky bucket , The water in the leaky bucket flows out from the bottom of the bucket at a constant rate , When the water flowing into the leaking bucket is too fast , When the capacity of the leaky bucket is exceeded , Then it overflows directly . therefore , The leaky bucket algorithm can control the service requests to output uniformly at a fixed rate , Smooth burst traffic , Achieve traffic shaping , Provide a stable flow for subsequent processing . however , Leaky bucket algorithm can't control the request to input uniformly at a certain rate . - ** Token bucket algorithm ** ​ The token bucket algorithm is rate limited （Rate Limiting） And flow shaping （Traffic Shaping） One of the most commonly used algorithms in . Typically , The token bucket algorithm is used to control the number of data sent to the network , And allow the transmission of burst data . ​ First set a token bucket that can hold a fixed number of tokens , Use some mechanism to generate tokens at a constant rate . Every time a call is requested , Must go to the bucket to get the token , Only get the token , To release , Otherwise, you have to wait for the available token , Or refuse directly . If the token in the bucket consumes less than the speed it generates , The token will continue to increase , Until it's filled , The generated token will overflow from the bucket . ​ therefore , Token bucket algorithm can control the speed of uniform input request , It can also control the uniform output rate of the request . #### 3、 ... and 、 Service is broken ​ We are exposed to the word "fuse" in various scenarios . In high voltage circuits , If the voltage is too high somewhere , The fuse will blow , Protect the circuit . In the stock exchange , If stocks go up and down too much , It will also use a fusing mechanism , Stop trading , To control trading risk . ​ Again , In the microservice Architecture , The fusing mechanism also plays a similar role . When a microservice in the call link is unavailable for a long time or there is a delay , Slow response , The system will blow up the call to the microservice of the node , Quickly return error information . When the micro service is monitored to work normally , Resume the call link again . #### Four 、Hystrix ​ Service fusing and service current limiting are important means of service governance in microservice architecture , Many open source frameworks or products contain this kind of functionality . For example, Ali. dubbo,Netflix Hystrix,go-micro,go-kit etc. . ​ Hystrix yes Netflix The company's open source project , It's a delay and fault tolerance Library , Designed to isolate remote systems 、 Access points for services and third-party libraries , Prevent cascading failures , And in the complex decentralized system which can not avoid the occurrence of failure, the flexibility is realized . ​ Hystrix The project uses java Language development , The code hosting address is ：https://github.com/Netflix/Hystrix. in addition , Someone used go Language transplanted the project , The code hosting address is ：https://github.com/afex/hystrix-go/hystrix. ​ Hystrix You can do the following ： - Guarantee the reliability of third party service call by controlling delay and failure - Prevent cascading failures in complex decentralized systems , Prevent avalanche - Fail fast 、 Fast recovery - Step back and gracefully downgrade - Provide near real-time monitoring 、 Alarm and operation control ​ Hystrix It can make your system in case of dependent service failure , By isolating the services that the system depends on , Prevent service cascading failure , At the same time, it provides a failure fallback mechanism , Deal with failure more gracefully , And make your system recover more quickly from anomalies . Use Hystrix Very simple ： ``` package main import ( "github.com/afex/hystrix-go/hystrix" "log" ) func main() { // Every kind of microservice 、 Remote system 、 Calls to third party libraries are wrapped as a command , // You need to set the relevant configuration information before calling hystrix.ConfigureCommand( "ServiceA", // Command name hystrix.CommandConfig{ Timeout: 1000, // Timeout setting （ millisecond ） MaxConcurrentRequests: 1, // Maximum number of concurrent requests ErrorPercentThreshold: 50, // Error rate threshold , Exceeding the threshold will blow up the service SleepWindow: 5000, // After the service is out of service , How long will it take , The fuse is checked again to see if it is open （ millisecond ） RequestVolumeThreshold: 20, // Before the service goes out , Minimum number of requests required , Request threshold Whether the fuse is opened or not must satisfy this condition first ; The setting here means at least 5 It's a request ErrorPercentThreshold Error percentage calculation }) // Use hystrix Call remote service , And enable according to the command name ConfigureCommand The argument set in _ = hystrix.Do( "ServiceA", // Command name // Executive functions func() error { // Call remote service log.Println("invoke a remote service") return nil }, // Failure fallback function func(err error) error { log.Println("fall back") return err }, ) } ``` notes ： All source codes of this paper are located in ：https://github.com/wangshizebin/micro-service The development tool used in this paper is ：[goland](http://www.sousou88.com/software/2072802.html) From [ Whoosh download ](http://www.sousou88.com)