当前位置:网站首页>How many of the most common distributed ID solutions do you know

How many of the most common distributed ID solutions do you know

2020-12-07 02:43:30 itread01

# One 、 Decentralized ID Concept Speaking of ID, Characteristics are the only , In the human world ,ID It's the ID card , It's everyone's unique identity . In a complex decentralized system , A large number of data and messages need to be uniquely identified . For example , Database ID In the case of a single field, you can use auto increment as ID, But after the data is divided into databases and tables, a unique ID To identify a piece of data , This ID It's decentralized ID. For the decentralized type ID for , It also needs to have the characteristics of a decentralized system : High concurrency , High availability , High efficiency and other characteristics . # Two 、 Decentralized ID Implementation scheme The following table is a comparison of some common schemes : | | describe | Advantages | Disadvantages | | ---- | ---- | ---- | ---- | | UUID | UUID Is the abbreviation of universal unique identification code , The purpose is that all elements in an upper decentralized system have unique identification information , There is no need to specify a unique identifier through the central controller . | 1. Reduce the pressure on global nodes , Make primary key generation faster ;2. The generated primary key is globally unique ;3. Merging data across servers is convenient | 1. UUID Occupy 16 Characters , It takes up a lot of space ;2. It's not incremental numbers , Data write IO It's very random , And the efficiency of index decreases | | Database primary key auto increment | MySQL The database sets the primary key and the primary key grows automatically | 1. INT and BIGINT Types take up less space ;2. The primary key grows automatically ,IO Writing continuity is good ;3. Number type queries are faster than strings | 1. Concurrent performance is not high , Limited by database performance ;2. Sub warehouse and sub table , Need to be transformed , Complex ;3. Self increasing : The amount of data leaked | | Redis Self increasing | Redis Counter , Atomic self increasing | Using memory , Good concurrency performance | 1. Data loss ;2. Self increasing : The amount of data leaked | | Snowflake algorithm (snowflake) | The famous snowflake algorithm , Decentralized ID The classic solution for | 1. Independent of external components ;2. It works well | Clock back | At present, the popular decentralized ID There are two solutions :** Segment mode ** and ** Snowflake algorithm **. ** Segment mode ** Depending on the database , However, it is different from the database primary key auto increment mode . Suppose 100 For a segment 100,200,300, You can get 100 One ID, The efficiency has improved significantly . ** Snowflake algorithm ** It's by the sign bit + Time stamp + Working machine id+ A serial number , As shown in the figure : ![](https://img2020.cnblogs.com/blog/1538609/202012/1538609-20201206214701214-124661203.png) The sign bit is 0,0 A positive number ,ID It's a positive number . Time stamp bit needless to say , Used to store time stamps , The unit is ms. Working machine id Bits are used to store the machine id, It is usually divided into 5 Regions +5 Server ID bits . The sequence number is self increasing . * How much data can snowflake algorithm store ? Time frame :2^41 / (365*24*60*60*1000) = 69 year Scope of work procedure :2^10 = 1024 Serial number range :2^12 = 4096, Express 1ms Can generate 4096 One ID. According to the logic of this algorithm , Just use this algorithm Java Language is realized , Encapsulate as a tool method , Then each business application can directly use the tool method to obtain decentralized ID, Just make sure that each business application has its own working machine id that will do , And you don't need to build a separate acquisition decentralized ID Application of . Here's the twitter version Snowflake Algorithm : ``` public class SnowFlake { /** * Start timestamp */ private final static long START_STMP = 1480166465631L; /** * Number of bits occupied by each part */ private final static long SEQUENCE_BIT = 12; // Number of digits occupied by serial number private final static long MACHINE_BIT = 5; // Number of digits occupied by machine identification private final static long DATACENTER_BIT = 5;// Number of data center occupied /** * The maximum value of each part */ private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT); private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT); private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT); /** * The displacement of each part to the left */ private final static long MACHINE_LEFT = SEQUENCE_BIT; private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT; private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT; private long datacenterId; // Data Center private long machineId; // Machine identification private long sequence = 0L; // Serial number private long lastStmp = -1L;// Last timestamp public SnowFlake(long datacenterId, long machineId) { if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) { throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0"); } if (machineId > MAX_MACHINE_NUM || machineId < 0) { throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0"); } this.datacenterId = datacenterId; this.machineId = machineId; } /** * Generate next ID * * @return */ public synchronized long nextId() { long currStmp = getNewstmp(); if (currStmp < lastStmp) { throw new RuntimeException("Clock moved backwards. Refusing to generate id"); } if (currStmp == lastStmp) { // Within the same milliseconds , Serial number auto increment sequence = (sequence + 1) & MAX_SEQUENCE; // The number of sequences in the same millisecond has reached the maximum if (sequence == 0L) { currStmp = getNextMill(); } } else { // In different milliseconds , Serial number set to 0 sequence = 0L; } lastStmp = currStmp; return (currStmp - START_STMP) << TIMESTMP_LEFT // Time stamp section | datacenterId << DATACENTER_LEFT // Data center | machineId << MACHINE_LEFT // Machine identification part | sequence; // Serial number section } private long getNextMill() { long mill = getNewstmp(); while (mill <= lastStmp) { mill = getNewstmp(); } return mill; } private long getNewstmp() { return System.currentTimeMillis(); } public static void main(String[] args) { SnowFlake snowFlake = new SnowFlake(2, 3); for (int i = 0; i < (1 << 12); i++) { System.out.println(snowFlake.nextId()); } } } ``` # 3、 ... and 、 Decentralized ID Open source components ## 3.1 How to choose open source components The first thing to choose an open source component is to see whether the software features meet the requirements , It mainly includes compatibility and extension suite . Second, we need to look at the current technical capabilities , According to your own or team's technology stack and technical ability , Can it be used smoothly . Third , It depends on the community of open source components , The main concern is whether updates are frequent 、 Whether the project is maintained by someone 、 When encountering a pit, you can get in touch and ask for help 、 Whether it is widely used in the industry . ## 3.2 Meituan Leaf Leaf It is a decentralized model launched by meituan's basic R & D platform ID Build service , The name comes from the German philosopher 、 A sentence from mathematician Leibniz :“There are no two identical leaves in the world.”Leaf With high reliability 、 Low latency 、 Global uniqueness and so on . It has been widely used in meituan finance 、 Meituan takeout 、 Meituan liquor brigade and other departments . Specific technical details , Please refer to an article in meituan's technology blog :《Leaf Meituan decentralized ID Build service 》. at present ,Leaf The project is already in Github Open source :https://github.com/Meituan-Dianping/Leaf.Leaf The features are as follows : 1. Global uniqueness , There will be no repetition ID, And ID The overall trend is increasing . 2. High availability , Services are based entirely on a decentralized architecture , Even if MySQL Downtime , Can also tolerate a period of time when the database is unavailable . 3. High concurrency and low latency , stay CentOS 4C8G On the virtual machine , Remote call QPS Up to 5W+,TP99 stay 1ms Inside . 4. Easy access , Directly through the company RPC Service or HTTP Call to access . ## 3.3 Baidu UidGenerator UidGenerator Baidu open source based on Snowflake The decentralized efficiency of the algorithm is unique ID generator . Use a description from the official website :UidGenerator Work as a component in an application project , Support custom workerId Bits and initialization strategy , So it applies to docker In the virtualization environment, the instance will restart automatically 、 Drift, etc . In practice , UidGenerator Solve by borrowing future time sequence The inherent concurrency limitation ; Adopt RingBuffer To cache generated UID, Parallelization UID Production and consumption of , At the same time CacheLine Complement , Avoided by RingBuffer The hardware level 「 Pseudo sharing 」 The problem is . Final single machine QPS Up to 600 Ten thousand .UidGenerator Of GitHub Address :https://github.com/baidu/uid-generator ## 3.4 Open source component comparison Baidu UidGenerator yes Java Language ; The last record submitted was two years ago , Basically no maintenance ; Only snowflake algorithm is supported . Meituan Leaf It's also Java Language ; Recently maintained as 2020 year ; Support segment mode and snowflake algorithm . To sum up, the theory and the comparison of the two open source components , Or meituan Leaf A little better . What else do you know about the commonly used dispersions ID Solution

版权声明
本文为[itread01]所创,转载请带上原文链接,感谢
https://chowdera.com/2020/12/20201207024030517a.html