In the previous article, beige has summarized the core competence of programmers , as well as Java The basic knowledge that programmers should learn when they grow up . In a Java Programmers work 3、5 Years later, , Has been able to undertake most of the core development work , Growing into a senior developer on the team . Most of the problems in our work can be solved by ourselves . At this stage, many students will face a new growth puzzle , In the end, what aspects do you need to continue to improve ？ How can you become an architect in a team ？
There are many books on the market that analyze and dismantle the capabilities of architects , for example 《 Talk about architecture 》《 Core technology of billion level traffic website architecture 》《 Technical framework of large website ： Core principles and case analysis 》 etc. , There are some related practical and theoretical knowledge sharing in the book . If you have time, you are recommended to read these books .
Today, beige combined with my work experience , Also to talk about how to do the application system architecture design .
It should be noted that , What architects themselves require is comprehensive capabilities , Architecture is also divided into business architecture 、 Application system architecture 、 Technology Architecture 、 Data architecture and other multiple dimensions, multiple subdivision areas , But what I'm sharing today about architecture , More or more focus on the server-side application system architecture .
01 guiding ideology
As an architect , When doing application system architecture , It's better to gradually precipitate a set of guiding ideology , The guiding ideology is used as a guideline when encountering confusion or indecision in the process of architecture design . My personal experience is as follows
- Balance and trade-offs
Architecture is a complex job , Consider the needs of the moment , And pay attention to possible changes in the future ; It should be considered comprehensively enough , It's also easy to implement ; We need to measure the cost of implementation , Also pay attention to the efficiency of landing . All this means that we need to balance the architecture , Learn to choose .
- Iteration and evolution
A good architecture , It must have evolved over a long period of iteration . When doing architecture design , Limited by the current well-defined needs , It's often impossible to think so comprehensively about the future . Even when we do architecture design, we have taken all aspects into consideration , After the system goes online, it will encounter some new unknown problems , And as the requirements continue to iterate , New changes and challenges will be introduced , Therefore, we continue to optimize and iterate the architecture , We must pay attention to it .
- Focus on business needs
There is no flawless, perfect architecture , There's no one size fits all architecture , There is only an architecture suitable for the current business and product needs . We can learn from 、 Absorb other people's experience and practice , But the most appropriate architecture must be designed and evolved to meet the actual needs of the business . Architecture designed out of business requirements will encounter new problems in development .
02 Architecture design goals
In the design of application system architecture , Some universal architecture design goals should be followed . These goals include
- Feasible ： The architecture was designed , Must be able to be implemented and landing , If the technology used in the design architecture is not mature or has no production environment availability , This kind of architecture can only stay in the theoretical stage , There is no possibility of landing , This kind of architecture has no great value and significance .
- Extensible ： Although it is impossible to predict the future changes in demand , But in some cases , The architecture we design should also have the capacity of users 、 bandwidth 、 When the amount of data increases , It has good expansibility ; besides , When doing requirement analysis , Put the system 、 Domain services and even modules are well planned and designed , Be able to reasonably deal with the problems that may arise from the expansion of future demand , It is also an embodiment of architecture extensibility .
- Highly available ： The system architecture needs to consider the system availability under various abnormal conditions , Such as the sudden burst of flow peak elimination limit current , Degradation after a service or interface failure , Through various measures designed in the architecture to ensure that the entire application system is highly available .
- Measurable ： Finally, the architecture design of application system needs to be implemented at the code level , Provide online publishing system and services . When we design the system architecture , It needs to be estimated based on business data , Design the system data index which satisfies certain order of magnitude , Quantify the results of architecture design . When the system load is too high and needs to be expanded , You can refer to quantitative data indicators to estimate the required servers and resources .
Next, start with business requirements analysis , To the final system online operation , According to the knowledge and content that needs attention in different stages , To illustrate the whole application system architecture design process needs to consider the problem .
03 Demand analysis stage
As mentioned earlier , The system architecture should focus on business requirements , Business needs analysis , It is a necessary prerequisite for system architecture design . Here is a brief description of my work in the requirement analysis stage . Under normal circumstances , Requirements that go into the architect level , After at least one round of discussion and communication between business personnel and product personnel , Or have basic business and product requirements rudimentary , Or have already produced business MRD( Business requirements statement ) Or product PRD( Product requirements specification ). At this time, the architect needs to have a full discussion and communication with business and product students after entering , Identify and produce the following information ：
- Identify clear business objectives , Find out what the business needs to address at the core of the problem , What is the expected effect through products and systems .
- Business process carding and business modeling . The core of business modeling is to sort out user roles 、 Business scenario 、 Process and related rules policy . Find out what kinds of operations can be performed by different user roles in which scenarios , As a result, we have a clearer overall understanding of the business .
- According to the business model 、 Business output MRD Or product output PRD, Sort out a general list of subsystems and functions . The function list is sorted out from the first level , And refine it layer by layer , Generally, it should be no more than four floors . For example, the first level is divided into user management 、 Order management 、 After sales return and replacement management, etc . In the second level, user management is subdivided into registration and login , Master data maintenance 、 Disable, enable, etc , By analogy .
- According to the business model and function list , Estimate and refine non functional requirements , For example, for the total number of users 、 At the same time, the number of online users 、 System access and other non functional requirements , system performance 、 response time 、qps System performance requirements , System duration 、 Personnel input 、 Project organization structure 、 Other project constraints such as cost, etc . In the demand analysis stage , I will eventually produce a list of functional requirements and non functional requirements based on the business model . This will be the core reference and basis for subsequent application system architecture .
04 Architecture design stage
After the requirements analysis phase is completed , We have entered the system architecture design stage .
In the system architecture design stage , I usually do data modeling first . According to the business model and function list , You can sort out the general system 、 Modules and functions , Thus, the conceptual model of the database can be basically determined . Through the conceptual model design of database , Combined with the list of functional requirements produced in the demand analysis stage , The detailed requirements of the whole system can be printed in the brain . At the same time, through the design of conceptual model , The relationship between different data entities is relatively clear , The division of services or areas has also taken shape . After completing the conceptual model of the database , Start the detailed system architecture design and technology selection stage .
In the system architecture design stage , I will follow the idea of layered architecture design , Gradually refine and expand . In the current mainstream technical solutions , Front and back separation is basically the default standard , On the one hand, the core reason is the rapid development of front-end and back-end technologies , The division of labor among technical professionals is clearer ; On the other hand, current products are moving , The front-end and back-end architecture cannot support the current mobile end, especially App Class application development , Separation of the front and back ends has also become a necessary measure .
According to the idea of hierarchical architecture design , The whole architecture is divided into front-end access layer 、 Service layer 、 Data access layer 、 Data storage layer . In some highly concurrent systems , There will also be caching related technologies throughout the architecture hierarchy .
Front end access layer
First , In the front-end access layer , It mainly solves the problem of user traffic from the terminal to the application server .
In the front-end access layer , The core needs to solve the response speed and stability of terminal display .
In traditional PC In the Internet scenario , Most products and applications are based on browsers , Through the way of web page to present to users , Such as Jingdong 、 Taobao and other e-commerce websites . At the terminal level , Because of the huge amount of user requests , With the help of page statics 、CDN Methods such as , Speed up page loading . At the request level , In order to improve stability and cope with greater traffic challenges , Before traffic enters the application server , By using DNS、 Software and hardware load way for traffic diversion , Send requests to different application servers . The hardware loads that will be used here are F5 etc. , And the software load is mainly through nginx Realization , In the early days there was LVS、Haproxy And so on .
Finish with the access layer , Let's look at the service layer . Service is the core of application system , Responsible for the implementation of business and product core requirements and logic .
At the service level , The key problem to be solved is , Whether the service can be expanded flexibly and rapidly , In order to cope with the possible sharp increase of system visits in the future .
To solve this problem , There is a trade-off between distributed architecture design and not . The core of distributed architecture design is to decouple services , To solve the problem of horizontal expansion of applications , Reduce the pressure of single service and single machine . If the system needs to bear the traffic in the foreseeable future will have a greater growth , Distributed architecture design is necessary . If the system flow is relatively limited and stable for a long period of time , The distributed architecture is not so necessary . Here is the need to balance and trade-offs .
Although the distributed architecture can bring the convenience of service level expansion , In order to cope with the possible large increase of system traffic in the future , But it needs to pay a large system maintenance cost . If you choose not to use distributed architecture for the time being , The service layer can also be at the engineering level , According to the interface layer of the service 、 The logical layer of the service and the data layer of the service are divided into modules and sub modules , Then, when the code is built and packaged, it is integrated into a single application as a jar or war Package released together . Based on my previous experience , In the early stages of a business and product , The architecture of single application mode with modular division , Can quickly complete the product MVP Version of the online trial and error , When the business develops to a certain scale, the service-oriented transformation of the distributed architecture will be started , It's also a better choice .
Whether you choose a distributed architecture solution from the beginning , Or in the later stage of the service-oriented transformation of the distributed architecture , We will face another selection problem ： Whether to use distributed services or microservices to build applications .
Distributed services and microservices are different concepts in the evolution of distributed architecture , As I understand it , Distributed services focus on distributed , Focus on solving the pressure and load sharing of service , Microservices focus on the granularity of service units , Pay more attention to the atomicity of services 、 independence . Both are for understanding coupling , But it's a little different in terms of the granularity of decoupling . The specific choice also needs to depend on the scale of the application system 、 Comprehensive consideration such as domain division .
It will be applied to the selection of distributed service architecture technology , There are traditional RPC Framework oriented technology system , And with Spring Boot Microservice framework based Spring Cloud Family bucket .
Conventional RPC Framework to the early Ali open source Dubbo The frame represents , The core implementation is based on dynamic proxy + The interface communication between services is realized by reflection , The underlying calls between services and services are based on socket To achieve data transmission interaction . and Spring Cloud Family bucket , The core is based on Http Protocol implementation of a set of microservices framework , Services and calls between services are made through Http Interface to achieve interactive . comparison Spring Cloud,RPC Frameworks generally have customizable data structures 、 Network transmission speed is fast 、 High efficiency , and Spring Cloud Because it's based on Http Protocol for network transmission , The packet size of a message is affected by Http The protocol restrictions are relatively bloated after being encapsulated , In the network transmission efficiency is relatively low , but Spring Cloud Because of a whole set of components and ecological support , So in the case of not doing too much performance demanding , It is also one of the solutions that can be adopted quickly at present . Another way to combine the two , utilize Spring Cloud Design and implement the access gateway of the interface , Then forward the request to RPC In service , Is currently some large-scale applications commonly used solutions .
In the use of Spring Cloud or RPC Frame the process , Another concern is the coupling between services . Although distributed services or microservices themselves are designed to solve the coupling problem , But applications and Applications 、 The dependency between services is not due to the use of Spring Cloud or RPC The frame disappears , In some scenarios , Moderate dependence between services is allowed and necessary , But excessive service dependence , It has even developed into interdependence between services , It's a situation that needs to be avoided . Through message middleware , Will have a relationship between upstream and downstream dependence , Decouple through messages , So as to avoid strong dependence between services , It's one of the solutions . Commonly used message middleware is RabbitMQ、RocketMQ、Kafka etc. .
Data access layer
Finish the service layer , Let's look at the data access layer . All applications are inseparable from data , Data is the soul of a system . The data access layer is mainly used to complete the data interaction between the service layer and the data storage layer . Because of the diversity of data storage methods , An important function of data access layer is to encapsulate different data storage methods , Try to shield the service layer from specific data storage details . Another important role of the data access layer , It is to solve the problem of resource management in data storage scenarios , Including connection pool resources 、 Sub database and sub table 、 Separation of reading and writing, etc . Whether it is necessary to do sub database and sub table 、 Separation of reading and writing, etc , According to the actual data size of the application 、 The choice of data storage methods and other comprehensive consideration .
Data storage layer
Last , Then consider the construction and selection of data storage layer .
The data storage layer is used to persist the data generated by specific applications . Different data storage methods have different business scenarios . If you want to be higher in business , Generally, relational databases such as Mysql、Oracle etc. ; For data structure and table structure expansibility requirements more flexible can be used NoSQL Database such as MongoDB、CouchDB etc. ; If the amount of data to be stored is very large , You can choose to be based on hdfs The storage scheme is as follows Hbase、Hive etc. ; There will also be pairs of documents 、 Pictures, etc. that need storage , Distributed file storage or cloud storage can be used . There is no good or bad solution for data storage itself , Specifically, it is necessary to analyze the actual needs of the business to make a balance and selection .
In addition to the several levels mentioned above , In most Internet applications , When the number of users accessing the system reaches a certain level , or QPS In higher scenes , It also reduces the response time of the system and interface by increasing the cache , Improve the performance of the system . Common caching frameworks are Redis、Memcache etc. . In the architecture design phase , Let's take the whole system apart according to the idea of layering , According to the actual needs of the business , Balance and trade-off between technologies and solutions used in each layer . The completion of architecture design phase does not mean the completion of the whole application system architecture . Next, we need to implement the architecture design into the code .
05 Encoding phase
After the architecture design is completed , For development engineers , We can guide the detailed requirement design and coding according to the architecture design scheme . For architects , At this stage, we need to focus on the following points ：
Engineering structure ： Good engineering structure , It can greatly reduce the maintenance cost of subsequent code and the cooperation cost of large team . The engineering structure of each floor will also have different emphasis and different , We can precipitate the engineering structure scaffold suitable for our enterprise at ordinary times , Create when needed , It can improve efficiency , You can also unify the engineering template .
Design patterns ： For complex needs , Architects need to focus on , Combine different design patterns 、 Extend to code design , Improve code extensibility and maintainability .
The code template ： Code templates can unify code styles , Improve the readability and maintainability of the code .
Code checking ： By checking the code , Find unreasonable code organization and Design 、 Potential code pitfalls and so on . You can also use some third-party plug-ins or tools, such as PMD、Findbugs Wait for an auxiliary examination .
06 Online operation stage
A good architecture , It has to be tested online . After the tester completes the quality test , Need to do online release of the application system , Entering the online operation stage .
In the online operation stage , The core responsibility of the architect is to set various monitoring and alarm indicators .
At the level of monitoring , First of all, we need to monitor the performance indicators of the application system . Through some open source or enterprise self-developed monitoring tools , It is necessary to use the physical resources of the application system, such as disk 、 Memory 、CPU、 Network monitoring ; Slow query of data storage components such as database is needed 、I/O、 The size of the library is monitored ; For middleware such as RabbitMQ、Redis Wait for reading and writing 、 Capacity, etc ; Response time to application system performance, such as interface response time 、95 Line 、99 Line monitoring and so on .
Another important monitoring object is the monitoring of business indicators , For example, the number of login users over a period of time 、 The amount of SMS sent 、 Order quantity 、 Order payment, etc . The monitoring of business indicators can often reflect some system level anomalies , Guide to investigate the root cause of abnormal business indicators , So as to improve the reliability and stability of the system .
The specific monitoring indicators need to be set reasonably according to different monitoring objects .
After establishing monitoring indicators , We also need to establish an alarm system , The abnormal indicators will be alerted in time . General monitoring system will provide similar SMS 、 Nail and other alarm mode access .
Last , In the online operation phase, it is also necessary to collect and pay attention to the log of the system , Check and repair the abnormality in time , Guarantee the steady operation of the system .
The above is from the requirement stage of a business , To the whole business application system online operation , As an architect, you need to focus on all aspects .
Of course , Each of the architecture points and technical solutions , It's a big issue to start with . When you have a holistic view of Architecture , Then we can go deep into different fields to study , And practice and trial and error in the work , Finally complete the advancement of your own skills .
Since the birth of computer , The software and hardware technologies are constantly updated , The software architecture is constantly upgraded . And with big data cloud computing 5G The leadership of the times , New business scenarios such as audio and video 、 live broadcast 、 Business challenges such as the Internet of things will follow , Our pursuit of architecture is endless .
I hope you and I can continue to evolve Architecture , Keep on learning , Continuous improvement .