当前位置:网站首页>Tidb x Zhongtong technology | improves efficiency by 300%. Tidb cooperates with Zhongtong to make your package "visible in real time"

Tidb x Zhongtong technology | improves efficiency by 300%. Tidb cooperates with Zhongtong to make your package "visible in real time"

2020-11-10 12:04:44 PingCAP

「 We've already used it 」, It's our favorite word to hear , Behind the simple words is the heavy trust and trust . Starting today , We will pass 「 Believe in the power of openness 」 Series of in-depth case sharing , From a business point of view , Take a look at the business value that a database brings to users in various industries . This article will introduce TiDB Join hands with ZTE technology to build a full scene and full link digital platform service story .

Insight into every part of the package's journey
Live up to your every waiting

Order a second to kill , When the goods are received and unpacked , I believe most people are interested in “ A double tenth ” The experience of express logistics in this special period is quite familiar . After placing an order 『 expect to see someone who never comes 』 To the parcel 『 Whole process tracking 』, Recent years , Express delivery is not what it used to be “ A black box on your shoulders ”, Electronic face sheet 、 Automated sorting 、 Intelligent robot 、 Thanks to digital technology such as full link digital tracking , The express industry is brewing a new and predictable transformation .

Zhongtong express was founded in 2002 year , After more than ten years of development , At present, the overall business scale has reached the first in the world , It is also the first express enterprise with an adult business volume of 10 billion , Last year's double 11 even more completed orders than 2 A great achievement of 100 million . Zhongtong technology is the Internet logistics technology platform of Zhongtong express , We have a R & D team of more than 1000 people , Adhering to “ Internet + logistics ” Idea , And the company's strategy 、 Close business convergence , Create a full scene and full link digital platform service for the business of Zhongtong ecosystem .

Business challenges

The life cycle of express delivery can be divided into five words , Send and send to dispatch .

According to this process, the whole logistics link will be disassembled into several key nodes , A large amount of data will be generated at each key node , For each key node, each data express company will carry out relevant analysis , Including time limit monitoring ( For example, the process tracking of express delivery 、 Express delivery time at the delivery point and so on ). In the original architecture, a large amount of data statistical analysis relies on Oracle Build a lot of stored procedures on it , But as the amount of data increases , The problem of storage and computing is becoming more and more obvious , Just upgrading Oracle The hardware can't solve the problem fundamentally , And with the constant upgrading of hardware , The cost is getting higher and higher .

In recent years , The business volume of express delivery industry is advancing by leaps and bounds , With the development of business, the amount of data is increasing rapidly , Zhongtong encountered the following problems :

  • Store in Oracle Exadata The data cycle of all-in-one computers is getting shorter and shorter , The design of sub database and sub table can't meet the requirement of timeliness, and statistical analysis relies on stored procedures , The expansibility and maintainability of the system are not high .
  • Single machine encounters performance bottleneck during business peak period , High risk of failure , Data synchronization T+1 There is not enough time for analysis .
  • How to reduce the TCO.
  • Fast business development 、 The amount of data is surging , It can be stored in Exadata The data cycle of all-in-one computers is getting shorter and shorter , Business demand for data cycles is rising .
  • Business peak single machine performance bottleneck , High risk of single point failure , Data synchronization T+1, The analysis time limit is not enough .
  • test HBase、Kudu Building a real-time data warehouse , It is not compatible with the existing technology stack , And it can't support the multi-dimensional business side query.

In the face of these needs , Zhongtong Express's new construction IT In addition to being compatible with the past IT framework , More agility , To be able to respond to the needs of business development faster , And it can better promote the development of future business . In support of key business , The underlying database needs to satisfy strong consistent distributed transactions , Support high concurrency read and write , Provide flexible online scalability , And can be compared with Spark Technology ecology is closely integrated , Support the construction of large and wide tables , Support multi-dimensional query analysis .

Why TiDB

According to the actual business situation and technical pain points of China Telecom , To build the TiDB Database cluster , It realizes the real-time writing of production data of multiple application systems , With the help of TiSpark Real time data analysis is realized , Summary data , At the same time, the upper application provides standardized API Interface , It provides flexible query interface for business operators and express delivery personnel , Real time 、 convenient 、 Accurate query service request , choice TiDB The specific reasons are as follows :

  • TiDB Support online expansion , Data press Region Fragmentation , It has its own scheduling management component , Hot spot scheduling and data distribution .
  • Strongly consistent ACID Distributed transactions 、 Secondary indexes .
  • Can write and update simultaneously , And support rapid response to business needs 、 Query results .
  • Technology ecology and Spark Close combination , Support with Spark Do minute level statistical analysis quickly .
  • Support the construction of large and wide tables , Support multi-dimensional query analysis .

Solution

Order & Waybill Center

After the user orders through the platform client , Generate a unique delivery order number as a unique identification . Express delivery except order number , There will also be a lot of attribute information , Such as : The postman 、 The sender's cell phone 、 Mailing address 、 The recipient 、 Express type and other information . After the express order is generated , The user's mailing items will become “ Courier ”.

When the express is sent , The courier receives the goods from 、 Sweep code 、 Transfer and other express transfer events 、 place 、 Time information will be pushed to the system from time to time . Express circulation information can not only be simple quantitative data , It can also be descriptive text 、 Special information such as geographic location . The system needs to record the flow information into the monitoring data of express delivery , At the same time, modify the express status 、 Real time location, etc , So as to realize the package of 『 Whole process tracking 』.

In China express, traditional IT In the architecture , A lot of statistical analysis of data depends on Oracle , But as the amount of data increases , The problem of storage and computing is becoming more and more obvious , Just upgrading Oracle The hardware can't solve the problem fundamentally , And with the constant upgrading of hardware , The cost is getting higher and higher .

The above figure shows the architecture of the whole system of China Express after reconstruction :

  • On the left is the message access from various links , adopt Spark Real time computing takes these messages in , And Hive Dimension tables do something in distributed computing Merge and JOIN.
  • At the same time, it will follow offline T+1 The calculation and analysis of the data 、 There is HBase Data to do Merge The calculation of .
  • The final result of the calculation will be saved to TiDB Inside . Every day at regular intervals and TiDB Do a synchronization , hold TiDB Data synchronization to Hive, Make a data backup .
  • rely on TiSpark stay TiDB Do statistical analysis of the data , It's often called the aggregation layer , The summary layer includes public data and business level data , We will also put these data in Oracle One of them is , Including light summary and multidimensional summary .
  • be based on TiDB To provide detailed services , image API Service of interface 、 Details and some tags .

In terms of the new architecture , Each critical node supports scale out , Solved a single problem , At the same time, it reduces the basic IT cost .

Second delivery : Digitalization reshapes the whole process of distribution

For logistics enterprises , Reducing logistics costs has always been a challenge for the industry , Zhongtong express is also constantly innovating from the mode 、 From the whole link optimization 、 And even all aspects of logistics to improve efficiency 、 cost reduction .

In our daily delivery process , You can usually see information like this :” The express has arrived XXXX Transfer Center ”、” The express has arrived XXXX Distribution Center ”, This is between the factory warehouse and the delivery terminal , The mode of setting up transfer warehouse is called “ Second delivery ”.

The principle is through the big data platform , Warehouse address 、 Material demand 、 Vehicle capacity 、 Delivery times and other data , Analyze and calculate , Optimize distribution routes 、 Reasonable choice of logistics center address 、 Optimize warehouse location , So as to reduce the logistics cost , Improve logistics efficiency . In the logistics link process of the whole express delivery of China Express , There will be a lot of message access in each transport link , It is necessary to predict the full link routing and time effectiveness for each single express , Positioning to each ticket express transshipment link , The whole process is not only a huge amount of data , And it requires high timeliness .

As shown in the figure above , China Express is based on TiDB Carry on the construction of real-time data warehouse width table , Business OLTP Data is passed through TiDB Write in real time , follow-up OLAP Business through TiSpark Do minute level analysis . After business measurement ,TiSpark Sync 3 100 million data to Hive Probably need 10 minute , For the real-time data warehouse construction and offline T+1 The integration provides guarantee , Effectively support the timeliness analysis and monitoring of the whole link , It can locate the status of each ticket in every link in quasi real time . according to the understanding of , Through secondary delivery , In developed and dense cities , Save for the enterprise on average 25% Per package delivery cost of .

User benefits

The tide of informatization and intellectualization has become a trend , Technology and business go hand in hand , In order to let the express industry obtain the future competitiveness .

Efficiency :IT Support efficiency improvement 300%

China Express 2019 Business volume completed throughout the year 121.2 Million pieces , Year-on-year growth 42.2%, Exceed the average growth rate of the industry 16.9 percentage . In the past 2019 Promotion of the 11th National Congress of the Communist Party of China ,TiDB At the same time, support the line OLTP and OLAP The business of , QPS The peak is 12 ten thousand +, Support 10 billion level inserts and updates ,TiSpark Support business online minute level statistical analysis , Perfect guarantee of double 11 Zhongtong express IT Stable operation of services . besides , be based on TiDB The next generation database infrastructure built also brings the following benefits :

  • Solve a single problem , The overall structure is clear , Increased maintainability , System scalability enhancement .
  • Meet high performance OLTP Business needs , Support online scale out , On and off line storage and computing nodes at any time , Application insensitivity .
  • Data storage cycle from 15 Heaven supports to 45 God .
  • OLTP and OLAP Separate , Support analysis of more business dimensions .

Reduced Edition : Data driven refined operation , The cost is down year on year 17.1%

In the face of fierce competition , Zhongtong express through technological innovation , With TiDB A new architecture upgrade of the business system for the data base , At present, there are more than 100 Physical nodes ,200 For the rest of the examples TiDB Put into production , Main service bill 、 Settlement Center 、 Order center 、 Waybill Center 、 Message center 、 Transfer intelligent related product line , The benefits are as follows :

  • Data driven refined management measures continue to bring benefits to full play ,2020 In the second quarter , The cost of single ticket decreased year on year 17.1%.
  • TiDB Flexible and efficient 、 On demand expansion deployment scheme , Compared to before Oracle , Significantly reduce the TCO.

Walk with customers , Believe in the power of openness

Every time the database architecture is improved and implemented , Whether it's TB Grade or PB level , It takes effort , But it is also worth every enterprise to practice . In this day and age , No matter what the size of the business , Learn to use the power of open source , Avoid making wheels repeatedly .

Behind every seemingly relaxed, there is an unknown effort , Behind every seemingly bright and beautiful , There's an unknown payoff . The construction of distributed database is a long way to go ,TiDB We are willing to work with Zhongtong technology and every customer , Work hand in hand to do things well .

版权声明
本文为[PingCAP]所创,转载请带上原文链接,感谢