Introduction TKEStack Tencent open source is a set of strong and easy-to-use enterprise class container choreography engine , It is one of the incubation projects of the open atomic open source foundation . This article is about TKEStack Ruyingzhe, head of the open source project 、TKEStack He Pengfei, senior product manager, is in cloud + Community salon online Share and organize , Introduce TKEStack Open source methodology , Hope to communicate with you .
One 、TKEStack brief introduction
TKEStack Is Tencent cloud open source container platform , A lot of friends asked ： With Kuberentes Why use TKEStack？ Actually Kuberentes As a capability platform, there are still many deficiencies . For example, it doesn't have UI、 Mirror warehouse 、 Rights management 、 journal 、 Monitoring and other basic operation and maintenance capabilities , Single handle Kuberentes As a business platform, it is still very thin , The container platform complements the above function points , hold K8S Packaged into a complete business platform .
TKEStack Open source history of
Tencent has been a container related computing platform for a long time .2009 Dr. Wu Jun came to Tencent from Google , Then learn from Google to make Tencent self-developed business platform T-Borg. here we are 2011~2013 Around the year, we began to use the community's mature and recognized technology as a business platform . There was an important technological shift , After turning to the mainstream technology of the industry ,Torca Turning to a platform dedicated to big data business , Hence the HOT（hadoop on torca）. here we are 2014 year ,Docker Technology also shows power , We are very optimistic about Docker、Kubernetes technology , So we quickly switched , And started to do a common business platform .
2014 In, there are also new products in GaiaStack To support Tencent's internal business ,2019 Tencent made business integration in , There are multiple business lines of consolidation ,Kuberentes Working with the public cloud business team to launch TKEStack The container platform , Integrated Kuberentes Technical ability .Gaiastack and TKE The combination of technology forms TKEStack This product , And in 2019 Officially open source in .
This year, , There is another very important time node , Just last month TKEStack Officially joined the open atomic open source foundation .
Two 、 Product orientation and direction
1. The container is the fact base of the cloud
Before introducing the positioning and orientation of the entire product , First of all, it is necessary to mention the hottest technology direction at present —— Cloud native .
Cloud native is the use of public cloud 、 Private and hybrid clouds are built and run to expand resilient applications . The container is basically the base role of the whole cloud native technology stack ,CNCF Officially, a large number of products and manufacturers have participated in the container family products , Basically, it's a situation of a hundred flowers blooming .
Among many products , There are two main products that are really familiar with , Namely openshift and rancher, They are the benchmark for open source container industry .
First of all, Red Riding Hood openshift, Everyone from K8S The code contribution of , Red hat K8S Code contribution rate is second to google The second , The technical ability is very strong , The product is also very perfect , And started to do open source products very early . The other is rancher, This year was suse Acquisition , The product experience is very good .
But in the use scenario of domestic enterprises , These two products also have some imperfections .
The first is the difference of user habits at home and abroad , Foreign products , It takes more care of the usage habits of foreign users . Foreign users like to use small and specialized products , Like command line tools . But most domestic users want to use one-stop platform , To interface and one-stop service .
Another problem is , Download some mirror images of foreign products at home , The Internet is a big problem , Often the network is blocked or very slow , It leads to a very poor use experience .
therefore TKEStack Making a container product needs to consider how to position the product ,TKEStack It's not about basic experience and global perfection openshift、rancher Go to benchmarking , They've been very mature , We just open source , It's hard to reach their maturity in a short time .
2. TKEStack Open source ideas
Tencent cloud's idea is to find the original blue ocean of cloud , We think in the future , Multi cloud and hardware and hardware platform are the same direction .
from 2009 Tencent has accumulated a lot in container technology since , Internal business also has a lot of accumulation in the cloud native domain . Based on Tencent's own technical advantages and accumulation, and we have a better understanding of the demands of domestic users , We plan to do some cloud native open source products based on these two points , Contribute Tencent's power to the original cloud Ecology .
（1） Heterogeneous hardware
First, heterogeneous hardware , Most of the usage scenarios are based on X86 The hardware , But there is a very important demand in China to support ARM. Tencent will do X86 and ARM Can be deployed within the same cluster .GPU In addition to having their own traditional comparative advantages GPU Beyond virtualization capabilities , And Intel GPU Ability and Cambrian and Huawei's GPU Related products stay TKEStack Fit it inside , So that the upper users can use .
（2） Heterogeneous infrastructure
Heterogeneous infrastructure is a hybrid cloud ,TKEStack The architecture design is to take one cluster to manage other clusters , Natural can do mixed cloud , Other clusters , Self built cluster registered to TKEStack Come inside , utilize TKEStack The image warehouse provided 、 authentication 、 journal 、 jurisdiction 、 monitor , In a TKEStack Interface to manage multiple K8S colony .
Tencent has accumulated a number of cloud native application related technical solutions , Two well known are stateful applications TAPP And NVIDIA GPU virtualization , There are also some big data packages that are integrating . Tencent hopes to take advantage of its internal open source components step by step TKEStack Integrated inside , Users can stay TKEStack Upper use .
（3） loose coupling
In addition, another important point is loose coupling ,TKEStack On the one hand, it supports self research business , On the other hand, it also supports commercial products , More than one product that is finally presented to the user state . Open source products that users use directly , There are also changes in internal use .
In the face of such demands , Just ask TKEStack To be pluggable 、 Building block assembly .A Products need to use logs ,B Products don't need to log , You can choose any combination you like ,TKEStack It has the ability of loose coupling .
3、 ... and 、 Open source methodology
TKEStack From last year 11 Since the open source in January , I have accumulated a lot of experience in almost a year , There are also many ways to do it , Thanks for the guidance of senior open source expert Ma Quanyi , Now, in addition to contributing code and technology to the outside world , It also embodies the accumulation of experience in the form of methodology , Here is a brief introduction to the experience and summary of open source governance model .
1. Why open source
Why open source ？ There are many answers to this question in many search engines , But it is more inclined to open source for personal development and the value of open source to the whole society . Tencent is more from as an enterprise or a department , What advantages can an open source project bring to companies and enterprises .
Only open source projects that can be commercially launched can get opportunities for development and prosperity . In recent years IT The development of the industry , There are a couple of big projects that you're familiar with ,Apache、Linux、CentOS wait , These all have a very deep open source background and commercial landing history .
First of all, open source can bring great Matthew effect to commercial version marketing , The stronger the stronger 、 The weaker the weaker .Apache First of all, there are open source projects , After the popularity of the open source project is opened, more and more users or developers join the project , Open source users are naturally transformed into commercial users or commercial product applications . Users want to improve the commercial version , It's natural to contribute your own code or experience optimization to the open source community , It's a virtuous circle .
Open source is an important support for commercial versions , Open source communities can contribute resources and power to commercial versions of products , It is also an open source commercial product to enrich its own features and environment adaptation 、 It is an important support for building a complete ecology .
Some customers are more sensitive to the problem that the business will be bound by a certain supplier , Providing users with open source commercial versions reduces their concerns , They will also be more willing to support open source projects .
The whole open source business and ecology support each other 、 Establishing standards for each other 、 Form ecology , Finally, it falls into the behavior of a commercial market , Turn into a virtuous circle , Open source projects 、 Business projects are getting bigger and bigger 、 Better and better .
2. Why open source projects need methodology
Take the big projects just mentioned as examples , In fact, the initial project was driven by some outstanding open source workers according to their personal preferences and enthusiasm .
But if you look at the various projects today , A huge project that can form a huge community or industry , It's hard to promote and perfect by personal enthusiasm alone . Such large industrial projects urgently need a set of methodology to support and guide the development of open source project strategy 、 Planning features 、 Building standards 、 Construction ecology , Step by step, step by step .
If it is to C Open source project of business model , Ecology is the most important point to be built . If it is to B Project , Cooperation is particularly important , Partner selection and governance are of great significance and value to open source .
3. Open source project architecture
（1） Pyramid model
The whole architecture of open source project is the pyramid model of open source strategy first , The position of the whole pyramid apex of enterprise open source should first make a good strategy , Only when you have a strategy , After pointing out the development direction of future products or projects , And then there's a follow-up standard 、 product 、 ecology .
Strategy is the premise that points out the general direction , Is the soul of the whole open source project . Before open source, we must think about what kind of effect we want to achieve by making products or projects 、 What is the future direction of development 、 Or what kind of position can open source projects in the more ideal future world stand on , What kind of contribution to you , Strategy is the soul and the core .
Projects should follow standards or industry de facto standards , We can pay more attention to whether open source projects can form industry standards or factual standards . Of course, it doesn't have to be specific 、 The interface specification , It can also be an implicit standard , Such as operating habits 、UI Interface design and so on , Guide and influence the user group of ideas and habits , Bring about a set of implicit standards .
Around the standard 、 Design open source projects around strategy , Design an open source project as a product , Realize that open source projects have their developers 、 There are corresponding users 、 There are real users , We don't do what we want , It's about taking an open source project as a real product 、 To be more and more perfect 、 More and more complete state .
Finally, with the product 、 Take the project as the core to build ecology , At the same time, we should transform ecology into the carrier of Commerce , This is the pyramid of enterprise open source strategy .
（2） The hourglass model
Next, the model of open source project governance , Or do you really want to run a whole set of open source projects . The most important part of the whole hourglass is the bottleneck part , Open source projects are products , Products are our top priority , The open source project finally presents to the customer is the whole product with the open source project as the core .
And the cornerstone of the product is the core Developer , The core developers can master the direction of the whole project , Maintaining the stability of the community .
Here are business partners who use open source projects , They are very motivated to perfect TKEStack The function of 、 Optimize and improve TKEStack Product quality and experience .
On this basis, there are more upstream developers from the community or other teams , They will do it for TKEStack Contribute to the adaptation of various environments 、 Various specific functions are abundant TKEStack Product line .
（3） Ecological partner model
about TKEStack Come on , The focus of enterprise open source projects is ecological partners , How to choose and cooperate with partners , The ecological partnership model is formed .
Partners are divided into several categories , First, the core developers and TOC Technical Committee , This is the core of the whole open source project , Only when the core team is stable can we keep TKEStack Stability of the whole open source project .
First TOC Technical committees can include founders 、 The mentor of the team 、 Architects 、 The product manager 、 Operations manager and so on , It's a committee to determine the direction of the entire open source project . The core developer community can come from within the company , Or from heavily used teams that contribute to the core code and architecture of an open source project .
In addition to the core workers, there are loyal followers , For example, the company after investment 、 There are very loyal fans of the project , They will contribute their energy to the whole project 、 Contribute to their adaptation to business scenarios or the real needs of customers can be achieved on open source projects .
An open source project should have an open and inclusive attitude to welcome all kinds of developers and even competitors to cooperate with each other , From a competitor's point of view 、 Some scenarios can also find points to work together . Everyone's starting point is to stand in the open source angle , It's all for the whole ecology and the community , From this point of view, it is easier to negotiate with customers, personal development or competitors .
The key to the success of the whole project is to choose the right core partners , Get a partner in the middle , Generally speaking, we should strive for more developers' power , Contribute to an open source community power toward a goal .
4. Other constraints on project development and success
For this methodology, there should be a governance model that can adapt to the methodology , Be able to implement methodology and open source ideal , There should be abundant market means to publicize open source projects 、 Promoting commercial products . meanwhile , This is also the whole open source community or products and projects, is a very good cardiotonic , Can promote the project better and faster development .
Four 、 Open source governance model
1. Open source project lifecycle governance
TKEStack Open source projects have lifecycle governance , Strategic planning from the beginning 、 To product demand 、 Development iterations 、 Release cycle 、 Mocha ITOM 、 operation management , Each stage has its own process and method .
For strategic plans and needs , There will be regular strategic plans TOC Technical Committee meeting for , The cycle is a half year or a year's annual plan or a half year plan , In order to formulate the whole TKEStack The annual milestone of .
After that, the milestones and roadmap will be posted on the code hosting website . The roadmap will be refined later , Really put it into everyday work , Track milestones and roadmaps in the form of weekly meetings .
Specific implementation stage , One is internal user feedback 、 Second, external user feedback . Internal customers will have a set of TAPD Requirements management system to track the whole state , External users use Github Issue track .
2. Demand process
Because of the open source , So there's a lot of feedback from users and external issues . If you follow the previous process , Requirements development tracking along two lines can cause a lot of conflicts and confusion , Product managers will be very tired of running on two lines , Coordinate internal and external needs .
For this question , We made improvements , It's automated . For example, external needs and problem lists , There are automated tasks that are synchronized into the internal system every day , The company's core developers can see the needs of external customers , And the first time to review , Give his development advice .
For a larger demand , We will also submit the development suggestions or outline design Proposal For the whole community to see .
3. Development process
The development process includes developing 、 Testing and final submission , Can be traced in the system as a whole , And it can also be synchronized to Github Open source website , Users and teams inside and outside the company are able to grasp the status of the whole requirement in the first time .
The development process follows Github Standard development process , Create a branch 、 establish PR apply 、 Discussion and final deployment 、 Testing, etc. , For details, please refer to Github Its official website .
4. Testing process
The other side of the development process is the testing process , Now? TKEStack Be able to do level 3 testing , Each test has different stages and goals .
UT It is to ensure the compilability of the whole code at each time the code is compiled 、 And there are no major mistakes in grammar .SmokeTest Every time PR Run when submitting , There are about a dozen test cases to ensure that the commit code does not affect TKEStack Core functions , At the end of the day release test , To ensure the reliability of the final release . This is the process of level 3 testing .
5. Release and maintenance
Publishing also has its own version release and branch management process . The main branch is marked in red below , It's a branch of long-term tracking , be-all fix,feat,docs And so on, all the code is provided to the main branch .
The main branch will release the version according to the plan , For example, the released version is v1.4.0, Is to find the appropriate release version on the main branch as planned , Pull from above release-1.4 The branch of , When it's released, it will v1.4.0 Put the label on the branch , This v1.4.0 The branch of is the branch we need to maintain for a long time . Follow up if there is bug Changes are also published synchronously on this branch , It will also be published on branches on a regular basis v1.4.1 Small version of . The overall release and maintenance plan can be seen .
6. Community governance
The last part of the whole governance model is community governance , Corresponding to the methodology mentioned earlier , Attract a variety of partners and teams to participate in the governance of the entire open source community .
The core developers within the team 、 The contributors within the company are responsible for TKEStack Core functions , To ensure the product development direction and core competence , It's the infrastructure work that's assigned .
The advantage of external partners is that they can adapt to the environment to meet the needs of real users , This is very important in terms of product function , There are also more involved in the construction of commercialization , And specific functions , Like logs 、 Monitoring and other optimization requirements for the user environment .
A large number of developers in the community can also have more optimization experiences , Some of the industry's better practices or methods can be involved in the community .
For each development team or for individual community developers , Tencent has also made a lot of efforts , For example, organize online and offline activities to promote and introduce relevant knowledge , for example TKEStack Recently, I have been doing open source reward task , We will regularly release some development tasks to all to participate in . Tencent will also reward the community with various rewards 、 Improve your enthusiasm for community activities .
5、 ... and 、 Case introduction
1. Inside the cloud
Tencent is now doing full volume business on the cloud , This is a process , Some services rely on peripheral systems that have not yet been put into the cloud , Therefore, it is difficult to switch to common cloud deployment in a short time .
On the other hand , In business, the cloud needs to have an intermediate process , And the machines in the cloud room and IDC The machines in the machine room are different , When all the business is switched to the cloud ,IDC The machine will be idle .
To solve these two problems ,TKEStack A set is deployed internally to support the business cloud .TKEStack The interface and the public cloud come down in one continuous line , There is no big difference in the experience of the two products . So we can't go to the cloud completely yet , But we have to make the products of cloud transformation , Now we can use TKEStack Complete the cloud , The condition is mature and further goes to Tencent public cloud ,TKEStack It has supported the internal business operation of millions of cores .
2. TKE Enterprise Edition
TKEStack In addition to open source, enterprise edition also supports internal business , It also supports commercial products . Compared with the internal resources on the cloud , The commercial enterprise version will be strengthened in many ways .
TKEStack Enterprise interface and open source TKEStack There are some different , Open source TKEStack More packaging on the interface based on commercial demands 、 There's more to show .
TKEStack The core capabilities of Enterprise Edition , For example, cluster scheduling 、 establish 、 The core underlying capability of business management is TKEStack contribution , But in terms of microservices, operation and maintenance and operation related stronger user experience functions are more built by the enterprise version itself . This is to combine some of the open source underlying capabilities with the capabilities required for commercialization , Will become TKEStack Enterprise Edition , They don't affect each other .
6、 ... and 、 Summary and Outlook
Open source is similar to entrepreneurship , To be good at the time and the earth .
days , It's right time to do the right thing , For example, at present, everyone is doing cloud native , At this time, the container platform can attract users , If you make one openstack platform , The relative attraction is not that strong , So do the right thing at the right time , It's time to follow the mainstream technology trend .
The right place , With a good direction, you have to be able to do . Tencent has accumulated more than ten years in this respect , The technical ability is relatively strong , Experience in the use of existing internal business , Have Tencent public cloud operation experience , There are also large-scale training support capacity , The combination of these can be said to be an open source container platform, with the technical foundation .
and , It is difficult to have such an opportunity in China to have a group of people who are fully open source , Long time focus, no business in the open source field KPI, It's an idealized process . The process is actually to support both core business , And open source products .
In a comprehensive coordination between the two , Have the right strategy to drive the continuous iteration of the product , With the products, more people are needed to participate in the ecology to identify with the ecology , Use 、 maintain , And make contributions to ecology . More people are involved in this product to be more alive , Is a qualified open source product .
It's the right time, the right place and the right people , It's the same thing to sell a product commercially , Good business opportunities 、 Have the ability to make products , There are right people making popular products .
The future cloud computing infrastructure must be in a multi-dimensional heterogeneous state . Multidimensional is three-dimensional , Hardware is heterogeneous 、 Infrastructure is heterogeneous （ Public cloud and private cloud ）、 Business is also heterogeneous , Not only some very simple stateless services, but also stateful tasks , Including online business 、 Offline business and so on .
We hope to provide users with a one-stop common infrastructure platform at the multi-dimensional and heterogeneous level , This is a TKEStack Vision .
TKEStack In this year 10 Month issued 1.4 edition , The main core points include the application market 、 Import cluster supports the installation of plug-ins , There are also cluster small version upgrade functions gradually up to TKEStack in , At the same time, it fixed a lot of user feedback problems related to experience .
This year, 12 month ,TKEStack There will be upgrades at several important points , For example, the integration of big data components , To keep Tencent cloud advantage products and TKEStack Sync . We will integrate excellent big data suite in the year ,TKEStack One click can deploy common big data Suite . High availability of a cluster is also a difficult issue in the industry , There are too many unknowns to a private cloud environment , It's troublesome in this scenario .
Next year, ,TKEStack Will continue to develop to the upper level , For example, copying applications , At least, we should have the ability to use all products of Tencent cloud public cloud .
at present , about TKEStack The call for upgrading the whole platform is IPV6, But it's up to the authorities IPV6 Official support for . Next year Tencent's public cloud and hybrid cloud will also have more capabilities , Like all of them DNS Services can be accessed in multiple clusters , Cluster migration can also provide more tools , It's easier to do the whole thing without a threshold K8S Cluster migration , Or container migration , These capabilities can really solve the common problems of many users .
Q：TKE How to realize service discovery ?
A： We don't have the ability to do more service discovery , Namely K8S Native service Service discovery mechanism , But the next version will soon integrate your own Service mesh product , Take the original cloud route .
Q： Where do external demands come from , Are they all customers ?
A： We are Github Open source , There are people in the community who ask for our products to come and see , It will be taken out to see if it is suitable to do , If it is suitable, it will do it , Of course, there are also priorities .
Q： What is the integration of big data components ？
A： There is no integration in the current version , We will try to put it in this year , It will contain common big data 、AI Components , Containerized deployment will be in this big bag , Because there are still intellectual property issues to deal with , We hope to integrate it into TKEStack One click deployment , Let's really make big data and AI Business can TKEStack Run better on .
Q：TKEStack How to build the operation system of ?
A： The operation system has just been introduced , Let's talk about it briefly , It's mainly about having internal core developers 、 Commercialization drives open source products, and people continue to do it , Just a few developers is certainly not enough , It will also attract more people into , On the one hand, it attracts the internal technical team , For example, other departments of Tencent cloud , They also have a lot of product capabilities in the containerization process, and we will attract in , There are also many business partners , Some aspects of business partners can be well nurtured TKEStack, Or to TKEStack Put forward some requirements , He did it on his own , You can also submit this part , Business partners are also continuing to work for TKEStack Contribute .
External users are now focusing on attracting more users to join us in the construction , For a project as big as a container platform , We hope that more people can participate in it and make the whole product more perfect , Next, some activities are brewing , For example, open source reward related activities , For example, joining the open atomic foundation also hopes that other users can participate in TKEStack In the to .
Q： Please ask the teacher TKEStack How to design their own monitoring ？
A： At present, Prometheus cloud monitoring is the de facto standard , All cloud and container products are monitored with Prometheus , We will install a Prometheus plug-in for each business cluster . At present, the industry is popular with thanos The internal business of the scheme has also been applied , The next step will also be integrated into TKEStack In the to , future TKEStack By default, Prometheus , If it's highly available, we'll offer optional thanos programme .
Q：TKE There are several versions ？
A： There are mainly container services of public cloud TKE, And Tencent's proprietary cloud TKE, in addition , It's deploying the open source version independently TKEStack; And commercial versions based on open source versions TKE Enterprise Edition 、TCNP. Proprietary cloud TKE Use the public cloud TKE Technology realization and product form of , Depending on... In public or private clouds IAAS resources , For example, virtual machines 、 VPC. Independently deployed version , Based on bare metal and open source solution deployment , Don't rely on other resources .
Q：TKEStack You have to use Kuberentes Do you , and Kuberentes What does it matter ？
A： Yes , Because it's based on Kuberentes For the upper packaging ,Kuberentes If you use it directly, it's the command line , You can do all kinds of scheduling , But how do users log in 、 How the console manages the application 、 Monitor where to watch and so on ,TKEStack Is based on Kuberentes Do the upper packaging , hold Kuberentes Really become a usable business platform , This is a Kuberentes and TKEStack The relationship between .
Q：TKE Of configMap When to implement automatic discovery of changes , Re deploy Services , It doesn't need to be artificial ？
A：TKEStack Support for native K8S configmap function , Now there is only one interface that hasn't been done for the time being , Soon the current version under development has done UI turn , The next version can be used, just like the public cloud UI Set the product capability on .
Q：SmokeTest and ReleaseTest Of test case What's the difference ？
A： Every time PR Level of submission , The main focus will be on testing that this code submission will not affect the whole TKEStack The basic core function of , Test case time is also required , Try to finish it in an hour or half an hour , It's a simple test set 、 Core test set , And every release is automated release test , The purpose is to evaluate the version , Guarantee of version quality , So the test set is a comprehensive guarantee of test quality .
Q： and Racher What's the difference? ？
A：Racher It's a product that's done very well with the technology experience , Very mature products .TKEStack The key direction is to make the basic experience available 、 Not too much of a pursuit of detail , Because it's hard to use the same level in a short time , More heterogeneous clusters 、 A hybrid cloud 、 In terms of plug-in provision, we should focus on , But the technology experience should also be guaranteed continuously , You are welcome to raise your questions on use .
Q：TKE Cross cluster vpc Is there any implementation of access adoption service discovery ？
A： Service discovery among multiple clusters is now supported ,TKEStack We haven't done it ourselves. We are also planning the capabilities of hybrid cloud related products , The hybrid cloud is a big project early next year , It's not just the service discovery problem that needs to be solved , There are also network problems 、 Cluster versions are different 、 Plug ins, different problems, etc. have to be solved , It will be completed in the first half of next year .