当前位置：网站首页>Tidb x Zhongtong technology | improves efficiency by 300%. Tidb cooperates with Zhongtong to make your package "visible in real time"
Tidb x Zhongtong technology | improves efficiency by 300%. Tidb cooperates with Zhongtong to make your package "visible in real time"
2020-11-10 13:58:04 【TiDB】
「 We've already used it 」, It's our favorite word to hear , Behind the simple words is the heavy trust and trust . Starting today , We will pass **「 Believe in the power of openness 」** Series of in-depth case sharing , From a business point of view , Take a look at the business value that a database brings to users in various industries . This article will introduce TiDB Join hands with ZTE technology to build a full scene and full link digital platform service story .
Insight into every part of the package's journey Live up to your every waiting
Order a second to kill , When the goods are received and unpacked , I believe most people are interested in “ A double tenth ” The experience of express logistics in this special period is quite familiar . After placing an order 『 expect to see someone who never comes 』 To the parcel 『 Whole process tracking 』, Recent years , Express delivery is not what it used to be “ A black box on your shoulders ”, Electronic face sheet 、 Automated sorting 、 Intelligent robot 、 Thanks to digital technology such as full link digital tracking , The express industry is brewing a new and predictable transformation .
Zhongtong express was founded in 2002 year , After more than ten years of development , At present, the overall business scale has reached the first in the world , It is also the first express enterprise with an adult business volume of 10 billion , Last year's double 11 even more completed orders than 2 A great achievement of 100 million . Zhongtong technology is the Internet logistics technology platform of Zhongtong express , We have a R & D team of more than 1000 people , Adhering to “ Internet + logistics ” Idea , And the company's strategy 、 Close business convergence , Create a full scene and full link digital platform service for the business of Zhongtong ecosystem .
The life cycle of express delivery can be divided into five words , Send and send to dispatch .
According to this process, the whole logistics link will be disassembled into several key nodes , A large amount of data will be generated at each key node , For each key node, each data express company will carry out relevant analysis , Including time limit monitoring （ For example, the process tracking of express delivery 、 Express delivery time at the delivery point and so on ）. In the original architecture, a large amount of data statistical analysis relies on Oracle Build a lot of stored procedures on it , But as the amount of data increases , The problem of storage and computing is becoming more and more obvious , Just upgrading Oracle The hardware can't solve the problem fundamentally , And with the constant upgrading of hardware , The cost is getting higher and higher .
In recent years , The business volume of express delivery industry is advancing by leaps and bounds , With the development of business, the amount of data is increasing rapidly , Zhongtong encountered the following problems ：
Store in Oracle Exadata The data cycle of all-in-one computers is getting shorter and shorter , The design of sub database and sub table can't meet the requirement of timeliness, and statistical analysis relies on stored procedures , The expansibility and maintainability of the system are not high .
Single machine encounters performance bottleneck during business peak period , High risk of failure , Data synchronization T+1 There is not enough time for analysis .
How to reduce the TCO.
Fast business development 、 The amount of data is surging , It can be stored in Exadata The data cycle of all-in-one computers is getting shorter and shorter , Business demand for data cycles is rising .
Business peak single machine performance bottleneck , High risk of single point failure , Data synchronization T+1, The analysis time limit is not enough .
test HBase、Kudu Building a real-time data warehouse , It is not compatible with the existing technology stack , And it can't support the multi-dimensional business side query.
In the face of these needs , Zhongtong Express's new construction IT In addition to being compatible with the past IT framework , More agility , To be able to respond to the needs of business development faster , And it can better promote the development of future business . In support of key business , The underlying database needs to satisfy strong consistent distributed transactions , Support high concurrency read and write , Provide flexible online scalability , And can be compared with Spark Technology ecology is closely integrated , Support the construction of large and wide tables , Support multi-dimensional query analysis .
According to the actual business situation and technical pain points of China Telecom , To build the TiDB Database cluster , It realizes the real-time writing of production data of multiple application systems , With the help of TiSpark Real time data analysis is realized , Summary data , At the same time, the upper application provides standardized API Interface , It provides flexible query interface for business operators and express delivery personnel , Real time 、 convenient 、 Accurate query service request , choice TiDB The specific reasons are as follows ：
TiDB Support online expansion , Data press Region Fragmentation , It has its own scheduling management component , Hot spot scheduling and data distribution .
Strongly consistent ACID Distributed transactions 、 Secondary indexes .
Can write and update simultaneously , And support rapid response to business needs 、 Query results .
Technology ecology and Spark Close combination , Support with Spark Do minute level statistical analysis quickly .
Support the construction of large and wide tables , Support multi-dimensional query analysis .
Order & Waybill Center
After the user orders through the platform client , Generate a unique delivery order number as a unique identification . Express delivery except order number , There will also be a lot of attribute information , Such as ： The postman 、 The sender's cell phone 、 Mailing address 、 The recipient 、 Express type and other information . After the express order is generated , The user's mailing items will become “ Courier ”.
When the express is sent , The courier receives the goods from 、 Sweep code 、 Transfer and other express transfer events 、 place 、 Time information will be pushed to the system from time to time . Express circulation information can not only be simple quantitative data , It can also be descriptive text 、 Special information such as geographic location . The system needs to record the flow information into the monitoring data of express delivery , At the same time, modify the express status 、 Real time location, etc , So as to realize the package of 『 Whole process tracking 』.
In China express, traditional IT In the architecture , A lot of statistical analysis of data depends on Oracle , But as the amount of data increases , The problem of storage and computing is becoming more and more obvious , Just upgrading Oracle The hardware can't solve the problem fundamentally , And with the constant upgrading of hardware , The cost is getting higher and higher .
The above figure shows the architecture of the whole system of China Express after reconstruction ：
On the left is the message access from various links , adopt Spark Real time computing takes these messages in , And Hive Dimension tables do something in distributed computing Merge and JOIN.
At the same time, it will follow offline T+1 The calculation and analysis of the data 、 There is HBase Data to do Merge The calculation of .
The final result of the calculation will be saved to TiDB Inside . Every day at regular intervals and TiDB Do a synchronization , hold TiDB Data synchronization to Hive, Make a data backup .
rely on TiSpark stay TiDB Do statistical analysis of the data , It's often called the aggregation layer , The summary layer includes public data and business level data , We will also put these data in Oracle One of them is , Including light summary and multidimensional summary .
be based on TiDB To provide detailed services , image API Service of interface 、 Details and some tags .
In terms of the new architecture , Each critical node supports scale out , Solved a single problem , At the same time, it reduces the basic IT cost .
Second delivery ： Digitalization reshapes the whole process of distribution
For logistics enterprises , Reducing logistics costs has always been a challenge for the industry , Zhongtong express is also constantly innovating from the mode 、 From the whole link optimization 、 And even all aspects of logistics to improve efficiency 、 cost reduction .
In our daily delivery process , You can usually see information like this ：” The express has arrived XXXX Transfer Center ”、” The express has arrived XXXX Distribution Center ”, This is between the factory warehouse and the delivery terminal , The mode of setting up transfer warehouse is called “ Second delivery ”.
The principle is through the big data platform , Warehouse address 、 Material demand 、 Vehicle capacity 、 Delivery times and other data , Analyze and calculate , Optimize distribution routes 、 Reasonable choice of logistics center address 、 Optimize warehouse location , So as to reduce the logistics cost , Improve logistics efficiency . In the logistics link process of the whole express delivery of China Express , There will be a lot of message access in each transport link , It is necessary to predict the full link routing and time effectiveness for each single express , Positioning to each ticket express transshipment link , The whole process is not only a huge amount of data , And it requires high timeliness .
As shown in the figure above , China Express is based on TiDB Carry on the construction of real-time data warehouse width table , Business OLTP Data is passed through TiDB Write in real time , follow-up OLAP Business through TiSpark Do minute level analysis . After business measurement ,TiSpark Sync 3 100 million data to Hive Probably need 10 minute , For the real-time data warehouse construction and offline T+1 The integration provides guarantee , Effectively support the timeliness analysis and monitoring of the whole link , It can locate the status of each ticket in every link in quasi real time . according to the understanding of , Through secondary delivery , In developed and dense cities , Save for the enterprise on average 25% Per package delivery cost of .
The tide of informatization and intellectualization has become a trend , Technology and business go hand in hand , In order to let the express industry obtain the future competitiveness .
Efficiency ：IT Support efficiency improvement 300%
China Express 2019 Business volume completed throughout the year 121.2 Million pieces , Year-on-year growth 42.2%, Exceed the average growth rate of the industry 16.9 percentage . In the past 2019 Promotion of the 11th National Congress of the Communist Party of China ,TiDB At the same time, support the line OLTP and OLAP The business of , QPS The peak is 12 ten thousand +, Support 10 billion level inserts and updates ,TiSpark Support business online minute level statistical analysis , Perfect guarantee of double 11 Zhongtong express IT Stable operation of services . besides , be based on TiDB The next generation database infrastructure built also brings the following benefits ：
Solve a single problem , The overall structure is clear , Increased maintainability , System scalability enhancement .
Meet high performance OLTP Business needs , Support online scale out , On and off line storage and computing nodes at any time , Application insensitivity .
Data storage cycle from 15 Heaven supports to 45 God .
OLTP and OLAP Separate , Support analysis of more business dimensions .
Reduced Edition ： Data driven refined operation , The cost is down year on year 17.1％
In the face of fierce competition , Zhongtong express through technological innovation , With TiDB A new architecture upgrade of the business system for the data base , At present, there are more than 100 Physical nodes ,200 For the rest of the examples TiDB Put into production , Main service bill 、 Settlement Center 、 Order center 、 Waybill Center 、 Message center 、 Transfer intelligent related product line , The benefits are as follows ：
Data driven refined management measures continue to bring benefits to full play ,2020 In the second quarter , The cost of single ticket decreased year on year 17.1％.
TiDB Flexible and efficient 、 On demand expansion deployment scheme , Compared to before Oracle , Significantly reduce the TCO.
Walk with customers , Believe in the power of openness
Every time the database architecture is improved and implemented , Whether it's TB Grade or PB level , It takes effort , But it is also worth every enterprise to practice . In this day and age , No matter what the size of the business , Learn to use the power of open source , Avoid making wheels repeatedly .
Behind every seemingly relaxed, there is an unknown effort , Behind every seemingly bright and beautiful , There's an unknown payoff . The construction of distributed database is a long way to go ,TiDB We are willing to work with Zhongtong technology and every customer , Work hand in hand to do things well .
- C++ 数字、string和char*的转换
- Won the CKA + CKS certificate with the highest gold content in kubernetes in 31 days!
- C + + number, string and char * conversion
- C + + Learning -- capacity() and resize() in C + +
- C + + Learning -- about code performance optimization
C + + programming experience (6): using C + + style type conversion
Latest party and government work report ppt - Park ppt
Online ID number extraction birthday tool
Field pointer? Dangling pointer? This article will help you understand!
GVRP of hcna Routing & Switching
- LeetCode 91. 解码方法
- Seq2seq implements chat robot
- [chat robot] principle of seq2seq model
- Leetcode 91. Decoding method
- HCNA Routing＆Switching之GVRP
- GVRP of hcna Routing & Switching
- HDU7016 Random Walk 2
- [Code+＃1]Yazid 的新生舞会
- CF1548C The Three Little Pigs
- HDU7033 Typing Contest
- HDU7016 Random Walk 2
- [code + 1] Yazid's freshman ball
- CF1548C The Three Little Pigs
- HDU7033 Typing Contest
- Qt Creator 自动补齐变慢的解决
- HALCON 20.11：如何处理标定助手品质问题
- HALCON 20.11：标定助手使用注意事项
- Solution of QT creator's automatic replenishment slowing down
- Halcon 20.11: how to deal with the quality problem of calibration assistant
- Halcon 20.11: precautions for use of calibration assistant
- "Top ten scientific and technological issues" announced| Young scientists 50 ² forum
- Reverse linked list
- JS data type
- Remember the bug encountered in reading and writing a file
- Singleton mode
- 在这个 N 多编程语言争霸的世界，C++ 究竟还有没有未来？
- In this world of N programming languages, is there a future for C + +?
- js Promise
- js 数组方法 回顾
- ES6 template characters
- js Promise
- JS array method review
- 【Golang】️走进 Go 语言️ 第一课 Hello World
- [golang] go into go language lesson 1 Hello World