当前位置:网站首页>Interviewer: do you know the interface design principles of large factories? It's curd. I'm autistic on the spot

Interviewer: do you know the interface design principles of large factories? It's curd. I'm autistic on the spot

2020-12-06 19:29:43 Ao Bing

Have feelings , There are dry goods , WeChat search 【 Three Prince Ao Bing 】 Focus on this different programmer .

this paper GitHub https://github.com/JavaFamily Included , There is a complete interview site for a large factory 、 Materials and my series .

background

As the business grows , The system architecture changes from monomer architecture to service-oriented architecture , Horizontal layered architecture ; And then it becomes a microservice architecture ,

Service Grid , The interaction between services and services is becoming more and more complex , How to design an interface gracefully , What needs to be considered ? Especially for public service ( such as BFF) The interface of public domain name should be provided to the outside world , How to guarantee safety , I have sorted out some common measures since my work and how to realize it :

Data validation

Validation includes : Routine verification and business verification ;
Routine verification : Include required field verification , Length check , Type verification , Format check, etc ;
Business verification : It depends on the actual business , For example, the order amount cannot be less than 0 etc. ;

Idempotent design

So called idempotent , In short , It is that the result of multiple calls to the interface is consistent with the call once . Data change to do idempotent , Some interfaces are natural to ensure idempotency .

For example, query interface , Some changes to the data are a constant , And there are no other records and operations , That can also be said to be idempotent . In other cases , All changes to the data are involved 、 It is necessary to prevent the occurrence of repetitive operations for the change of state . The influence of repeated operation is prevented by indirectly realizing the idempotent of interface .

For example, we e-commerce is more common addition and subtraction GMV No matter how many times the same message comes, it should only add and subtract once , Otherwise, it will lead to wrong amount and even capital loss .

Request level : The results are consistent
Business level : The same user does not repeat orders , The goods are not oversold ,MQ No repeat consumption

The essence of idempotent is the problem of distributed locks , Distributed locks can normally be passed through redis or zookeeper Realization ;

In a distributed environment , Lock global unique resource , Serialize the request , Actually, it's a mutex , Prevent duplication , Solve idempotency

Security

1. Data encryption

We know that data is easy to be captured in the process of transmission , If it's transmitted directly, like http Protocol transfer , Then the data may be acquired by anyone in the process of transmission .

So you have to encrypt the data , The common practice is to conduct sensitive data such as the ID number md5 encryption . Now the mainstream practice is to use https agreement , stay http and tcp Add a layer of data security layer between (SSL layer ), This layer is responsible for data encryption and decryption .https How to configure and use , You can read my history article and study it by yourself .

Symmetric encryption : The key is constant during encryption and decryption , Common algorithms are DES,AES; The advantage is that the speed of encryption and decryption is fast ; The disadvantage is that before data transmission , Both sides of the service must agree on the key , If one party's key is compromised , Encrypting information is not safe .

Asymmetric encryption : Key pairs appear , After a key is encrypted , Decrypted by another key ; The private key is placed in the server file , The public key can be published to anyone ; The advantage is that it is more secure than symmetric encryption , But encryption and decryption is much slower than symmetric encryption , What is widely used is RSA Algorithm ;

https The implementation of is just a combination of two encryption methods , It integrates the advantages of both sides , Good in security and performance . Symmetric encryption and asymmetric encryption code implementation ,jdk Provides the relevant tool classes can be used directly , This article is just about .

2. Data signature

Introduce 3 Data signature security policy : Abstract [KEY] , Signature [ certificate ] , Signature + encryption [ certificate ]

The security policy describe Security level
Abstract [Key] Put the data and Key( Custom contract password ) After the combination, we will summarize Low security level , Contract key security is very low . In the case of contract key security, the data can be basically protected against tampering .
Signature [ certificate ] Use certificate and asymmetric signature algorithm to sign data In the security level , It can guarantee the unforgeability and non repudiation of data , But it doesn't guarantee the privacy of the data
Signature - encryption [ certificate ] Using certificates and asymmetric algorithms to sign data , Encrypts data using a one-time key and a symmetric algorithm High level of security , It can guarantee the unforgeability and non repudiation of data , And it can protect the privacy of the data .
  • Confidentiality (Confidentiality): Don't watch without permission
  • integrity (Integrity) : Don't tamper with
  • Usability (Availability) : Prevent unavailability
  • Non repudiation (Non-Repudiation): Users cannot deny their behavior

Abstract [KEY] The process : Combine the data to be submitted into a string in some way , And then through md5 Generate an encrypted string , This string is the signature of the packet , such as :

str: Parameters 1={ Parameters 1}& Parameters 2={ Parameters 2}&……& Parameters n={ Parameters n}$key={ User key };
MD5.encrypt(str);

Abstract [KEY] principle :Hash The algorithm is irreversible , And the results are unique , stay key When privacy is guaranteed , Can guarantee integrity
Abstract [KEY] defects :key It's hard to guarantee the privacy of , Plaintext transmission


Signature [ certificate ] The process : The client makes a plain text md5/SHA Calculation , The ciphertext is obtained by encrypting the calculated value with the private key , The client sends plaintext and ciphertext to the server , The server decrypts the ciphertext by public key to get the value A, At the same time, the server makes a md5/SHA Calculate the value B, It's worth A And value B, The same has to be verified , It can guarantee non usurpability and non repudiation , But it doesn't guarantee the privacy of the data ( Plaintext transmission )


Signature + encryption [ certificate ] The process : The client generates a random string , As password, And put this password adopt B Public key encryption generates ciphertext C, hold A In writing through password Encrypt to generate ciphertext B,
At the same time A It's written MD5/SHA The calculated value passes through A Private key encryption gets signature D, Put the ciphertext B And ciphertext C And signature D Send it to the server , The server decrypts the text through the private key C obtain password, And then through password Decipherment B You can get A Plaintext , At the same time, the signature can be used to verify whether the sender is A, as well as A Whether the data sent has been modified by a third party .

A malicious party can assume X, Pretending to be A, Sent a ciphertext B(password Generate ), Ciphertext C After the server receives the data , It can still be decrypted normally to get plaintext , But it is impossible to prove that the plaintext data is A It's still a malicious user B Sent . Signature D That means A Sign your own name , The server can verify .X Because there is no A The private key , This signature, it can't impersonate , It will be recognized by the server .

 encryption - Signature

3. Timestamp mechanism

The data is encrypted , The hotel grabs the data and can't see the real data ; But there are lawbreakers who don't care about real data , Get the data and make a malicious request directly , At this time, a simple way to do this is to consider the timestamp mechanism , Add the current time to each request , The server will compare the time in the message with the current time of the system , See if it's in a fixed time frame, like 5 minute , Malicious forged data cannot change the time in the message , exceed 5 Minutes can be considered an illegal request .

The pseudocode is as follows :

long interval=5*60*1000;// Timeout time 
long clientTime=request.getparameter("clientTime");
long serverTime=System.currentTimeMillis();
if(serverTime-clientTime>interval){
    return new Response(" Beyond the processing time ")
}

4. AppId Mechanism

Most websites need a user name and password to log in , This is actually a security mechanism ; The corresponding service can also use this mechanism , Not everyone can call , Before calling a service, you must apply for a unique appid, Provide the relevant key , When calling the interface, you need to provide appid+ Key information , The server will verify .

appid Using letters , Numbers , Special symbols are generated randomly , The only one generated appid See if the actual requirements of the system need to be globally unique ; Whether it's globally unique or not, it's best to have the following properties :

The trend is increasing : So when you save the database , Better performance of indexes

Information security : Random generation , Don't be continuous , It's easy to find rules

About the whole thing Id The common ways to generate are snowflake Way, etc

snowflake

Xnip2020-11-04_19-31-00

The diagram above describes the binary structure of a serial number .

No need to be the first , Constant for the 0, It means a positive integer ; Next 41 Bits indicate the timestamp , Accurate to milliseconds . To save space , This timestamp can be defined as the number of milliseconds experienced from a point in time (Java The default is 1970-01-01 00:00:00).

Later 10 Bit is used to identify the working machine , If there is a cross IDC The situation of , This can be 10 One is divided into two , Part of it is used to identify IDC, Part of it is used to identify the server ; Last 12 Bits are serial numbers , Self growth .

snowflake The core idea of 64bit Reasonable distribution of , But it is not necessary to strictly follow the division shown in the figure above . If there are fewer machines , You can shorten the machine properly id The length of , Save it for the serial number .

5. Blacklist mechanism

If so appid A lot of illegal operations , Or there is a medium black system , After analysis, directly put this appid Blacklist , All requests return error codes directly ;

We can give each appid Set a state such as : Initialization status , The normal state , Medium black state , Off state and so on ; Or we can use the distributed configuration center directly , Save blacklist directly , Every time you check if it's in the list ;

Current limiting mechanism

Common current limiting algorithms include : Token bucket current limit , Leaky bucket restrictor , Counter current limiting ;

  • Token bucket current limit
    The principle of token bucket algorithm is that the system puts tokens into the bucket at a certain rate , Discard the token when it's full ; When the request comes, it will take the token out of the bucket first , If you can get the token , Then you can continue to complete the request , Otherwise wait or refuse service ; Token bucket allows a certain amount of burst traffic , As long as there is a token, you can handle , Support multiple tokens at a time ;
  • Leaky bucket restrictor
    The principle of leaky bucket algorithm is to flow out requests at a constant rate , The incoming request rate is arbitrary , When the number of requests exceeds the capacity of the barrel , New requests waiting or denial of service ; We can see that the leaky bucket algorithm can restrict the data transmission speed ;
  • Counter current limiting
    Counter is a relatively simple and crude algorithm , Mainly used to limit the total number of concurrent , For example, database connection pool 、 Thread pool 、 The number of concurrent seconds ; As long as the total number of requests in a certain period of time exceeds the set threshold value, the current limiting will be carried out ;

Specifically based on the above algorithm how to achieve ,Guava Provides RateLimiter Tool class is based on token bucket algorithm :

 RateLimiter rateLimiter = RateLimiter.create(5);

The above code means that only five concurrent requests can be processed in one second , The above method can only be used for single application request throttling , Global current limiting is not allowed ; At this time, we need distributed current limiting , Can be based on redis+lua To achieve ;

summary

In fact, whether the interface is design or development , If it's not an urgent need, we can all think more about it , So your system will be more stable , During the launch and testing process bug less , And in terms of personal promotion , It's always a good thing to think more .

A lot of times, people are complaining : Oh, my company is small , I can't grow up in this kind of environment in my school . A fool , Most of the time, experts come here in the same way , But the same thing, everyone's attitude is different , Over time, the results will be different .

All right. , Now everyone should be at work , I stayed up late on duty and was still promoting the scene ( The article was written over the weekend , Now write a summary ), I'm aobing , The more you know , The more you don't know , See you next time .

Chatter

Ao Bing organized his interview articles into an e-book , common 1630 page !

Dry cargo is full. , The essence of words . Directory as follows , There are also interview questions and resume templates that I summarized when I reviewed , Now it's free .

link :https://pan.baidu.com/s/1ZQEKJBgtYle3v-1LimcSwg password :wjk6

I'm aobing , The more you know , The more you don't know , Thank you for your : give the thumbs-up Collection and Comment on , See you next time !


Articles are constantly updated , You can search through wechat 「 Three Prince Ao Bing 」 First time reading , reply 【 Information 】 I have the interview materials and resume template for the first-line large-scale factory , this paper GitHub https://github.com/JavaFamily Included , A complete interview site for a large factory , welcome Star.

版权声明
本文为[Ao Bing]所创,转载请带上原文链接,感谢
https://chowdera.com/2020/12/20201206192753552m.html