当前位置:网站首页>Platform upgrade of 300000 lines of code: changing tires for running cars

Platform upgrade of 300000 lines of code: changing tires for running cars

2021-03-29 18:11:28 InfoQ

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong"}],"text":" This article was originally published in "},{"type":"text","marks":[{"type":"italic"},{"type":"strong"}],"text":"Mahmoud Hashemi Of individuals "},{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong"}],"text":" Blog , Authorized by the original author InfoQ Translate and share ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"2020 Years can be capricious . Even though it's beyond people's control , But over time , I find myself devoting more and more time to something that feels accessible : For the large enterprise class I helped build Web Applications "},{"type":"link","attrs":{"href":"https:\/\/simplelegal.com\/","title":null,"type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"SimpleLegal"}]},{"type":"text","marks":[{"type":"italic"}],"text":" Design a solution for the future ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":" Now it's done , This platform upgrade can easily be at the top of my most complex projects , At the moment , The happiest ending . Happiness comes at a price , But with some proper methods , The price may not be as high as you think ."}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" summary "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We will "},{"type":"link","attrs":{"href":"https:\/\/simplelegal.com\/","title":null,"type":null},"content":[{"type":"text","text":"SimpleLegal"}]},{"type":"text","text":" The main products of , One 30 Wan Xing Django-1.11-Python 2.7-Redis-Postgres-10 The code base , Migration to Django 2.2-Python 3.8-Postgres-12 Technology stack , Finish as scheduled , And there was no major site incident . It feels great ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" As the technical director of this project , What does it look like ? For me, , That's true :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/a7\/a7936f3e65682bf40d198be21b54bc02.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" But as engineering director , What is its cost ?"},{"type":"text","marks":[{"type":"strong"}],"text":"3.5 Year development time , Each line of code only needs 2 dollar ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" I'm particularly proud of this result , Because in the process , We have also greatly improved the speed and reliability of the website and the development process itself . Now? , The product has a bright future , Ready to shine on the sales request for proposals and compliance questionnaires . most important of all , You don't have to worry about how to tactfully tell potential customers , They will be using unsupported technology ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In short , It's a huge 、 Sound investment , And it has paid off . If you're only here to see our own estimates of the work , That's all of the above . This article is about how to get your team to achieve the same results ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" background "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The story begins with 2013 year , Just from YC Hatched SimpleLegal For a new SaaS Legal technology has made all the right decisions :Python、Django、Postgres and Redis. In a typical start-up model , In the absence of technical barriers , Function comes first . The package is just a side-by-side upgrade ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" To 2019 year , The end of the technical runway is near . although Python 2 May have been extended support from different vendors , But in 2021 year ,Django 1 CVE There are very few volunteers for the patch .Web The framework has become a more risky attack , So it's time to pay off our technical debt ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" start "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" therefore , We are 2019 In the first 4 The season begins Tech Refresh Platform upgrade plan . The goal is : Upgrade the technology stack , And still offer new features , It's like changing a tire for a running car . We have to be careful , And that takes time . Here are some basic principles for long-term projects :"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":" Any weekly job 10 Projects with more than two hours should be spent every week 30 Minutes to synchronize ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":" Every regular meeting should be recorded . Put it in the invitation . Use the project log to record progress 、 Barriers and decisions ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":" This is a marathon , It's not a sprint . Avoid at night 、 Working on weekends and holidays ."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Let's start with a plan sketch , After an open discussion , In the end, it's only half right . Some of the early guesses worked out :"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":" go to "},{"type":"link","attrs":{"href":"https:\/\/github.com\/jazzband\/pip-tools","title":null,"type":null},"content":[{"type":"text","text":"pip-tools"}]},{"type":"text","text":", And based on the extensive change log analysis, release dependency . Identify incompatibilities py23 Version of the package .( Even though we have turned to "},{"type":"link","attrs":{"href":"https:\/\/github.com\/python-poetry\/poetry","title":null,"type":null},"content":[{"type":"text","text":"poetry"}],"marks":[{"type":"color","attrs":{"color":"#41accd","name":"user"}}]},{"type":"text","text":".)"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":" stay CI Add the row coverage report ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":" Improve the internal testing framework , Let developers write tests quickly ."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Here's more . Other plans are less realistic :"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":" stay 6 It will be... In six months CI Line coverage ranges from about 60% Upgrade to 95%."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":" Parallel conversion in three months app Package ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":" Take advantage of American holidays ( Thanksgiving 、 Christmas 、 new year ) Low flow time during , stay 2021 Gradually switch to new applications before 2000 ."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We are young ! Although we are naive , But at least we know there's a lot of work to do . To share the work , We are looking for 、 Hired and trained three dedicated overseas developers ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Leading questions "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Even with the addition of developers , To 2020 Mid term , More and more we realize that ,95% We're dreaming about our coverage , Say nothing of 100% 了 . Full coverage may be a best practice , but 3 One and a half developers can't do this kind of coverage . We did valuable tests , Even found the old Bug, But if we stick to this plan ,Django 2 It will eventually become a 2022 Years of the project .70%, We decided to revise the target ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We realize that , For most sites ,CI More sensitive than most users . So we focused on testing the most influential code . What's the big impact ?1) Failed the most noticeable code ;2) The hardest code to try again . By looking at traffic statistics 、 Batch job planning and asking support staff , You can build a high impact code list in a week ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" about 80% Our code base is not in this high traffic \/ In the high impact list . that 80% What to do ? Take advantage of error detection and quick fix ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" transformation Sentry Role "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f7\/f7c2752008eb562abd16aedce60c3018.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" One of the benefits of entrepreneurial life is , It's easy to try new tools . We are SimpleLegal One of the approaches adopted is , Put each 5 The last week of the week ( namely 20% Time for ) Leave it to the developers , Let them focus on the development process itself . Even the best chefs can't make five-star food in a messy kitchen . This is how we improve our work , Finally, it speeds up delivery ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" At a time like this , Someone came up with a brilliant idea , Use "},{"type":"link","attrs":{"href":"https:\/\/sentry.io\/","title":null,"type":null},"content":[{"type":"text","text":"Sentry"}]},{"type":"text","text":" Add specialized error reports to the system . In a day or two , We have a website , You can access and get the stack trace . It's amazing , But it was not until the Tech Refresh It was only at the beginning of the plan that we realized , Although integration takes only one day to develop , But it's going to take the team a few months to fully adopt ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" You see , Add... To a mature but fast-moving system Sentry It means one thing : The noise . Our website keeps making mistakes . Most errors are invisible , And it doesn't get in the way of users , Some users have quietly learned how to deal with long-standing website quirks . Soon , Our developers have learned to Sentry As a repository for debugging information .2019 year ,Sentry The event itself is not worth taking seriously .2020 year , Things have changed , The team responsible for the seamless upgrade of the platform needs to Sentry Become something else : Responsive website quality tool ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" How do we do it ? First step , adopt "},{"type":"link","attrs":{"href":"https:\/\/docs.sentry.io\/product\/sentry-basics\/guides\/getting-started\/#-how-many-projects-should-i-create","title":null,"type":null},"content":[{"type":"text","text":" The following best practices "}]},{"type":"text","text":" Enhance the inflow of Sentry The data of :"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":" Split the product into "},{"type":"link","attrs":{"href":"https:\/\/docs.sentry.io\/product\/sentry-basics\/guides\/getting-started\/#-how-many-projects-should-i-create","title":null,"type":null},"content":[{"type":"text","text":" A separate Sentry project "}]},{"type":"text","text":". This includes the front end and the back end ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":" Markup version . Don't label development environment deployment with branches , This can lead to Releases UI confusion . Add a separate branch tag for searching ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":" Separate the environment . This is very important for directional alarm .Sentry The client environment is through domain conventions and Django Of "},{"type":"link","attrs":{"href":"https:\/\/docs.djangoproject.com\/en\/3.1\/ref\/contrib\/sites\/","title":null,"type":null},"content":[{"type":"text","text":"sites frame "}]},{"type":"text","text":" To configure the . For the sake of understanding , Here's a baseline , We use these environments :"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":" Production environment : The current official version .DevOps monitor ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":" Sandbox environment : The current official version ( Some companies will make the next release ). For users to test changes .DevOps monitor ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","text":" demonstration \/ Sales environment : Last official version . It's mainly internal traffic , But it's also visible from the outside in the foreground .DevOps monitor ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":7,"align":null,"origin":null},"content":[{"type":"text","text":" Canary environment : The next official version . Also known as the transition environment . Internal flow .Dev monitor ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":8,"align":null,"origin":null},"content":[{"type":"text","text":"ProdQA Environmental Science : The current official version . Internally used to reproduce technical support issues .Dev monitor ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":9,"align":null,"origin":null},"content":[{"type":"text","text":"QA Environmental Science :Dev Branch 、dev Release 、 Internal flow . Debug data not monitored ."}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":10,"align":null,"origin":null},"content":[{"type":"text","text":" Local testing \/CI Environmental Science : By default, it is not published to Sentry."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" When the question is finally correctly marked and searchable , We use Sentry Newly added "},{"type":"link","attrs":{"href":"https:\/\/docs.sentry.io\/product\/discover-queries\/","title":null,"type":null},"content":[{"type":"text","text":"Discover Tools "}]},{"type":"text","text":" Export questions every week , And prioritizes the remaining errors . Our first concern is highly visible production errors for Non internal human users . The specific query is :"},{"type":"codeinline","content":[{"type":"text","text":"has:user !transaction:\/api\/* event.type:error !user.username:*@simplelegal.*"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We divide it into 4 class : Quick fix ( Small loophole )、 Quick error ( Will a vague 500 Errors turn into some form of operational 400 error )、"},{"type":"link","attrs":{"href":"http:\/\/agiledictionary.com\/209\/spike\/","title":null,"type":null},"content":[{"type":"text","text":"Spike"}]},{"type":"text","text":"( A big loophole , Need to study ) and Silence( Use Sentry The neglect function of ). stay 6 During the week , The number of events per week is more than per week 2500 It's down to less than 500 Time ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Through further efforts , The number of events per week is less than 100 Time , And spread out on a few issues , For a lean team , It's very easy to manage . although “Sentry Zero” Is the ideal , But we have achieved and maintained the real goal of responsive flow , This is largely due to "},{"type":"link","attrs":{"href":"https:\/\/sentry.io\/integrations\/slack\/","title":null,"type":null},"content":[{"type":"text","text":"Slack Integrate "}]},{"type":"text","text":". Our team no longer gets server error information from the support team . in fact , Now? , When a customer is in trouble , We'll tell them , And we already have a work order in process ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" It's very important to have a close relationship with the support team . In the strategy above , We've embedded more sensitive than real users CI. Although perfection is tempting , But it's OK to ask enterprise users to have a little patience , The premise is that the support team is ready . Sync with them every week , So there's less surprise . If they're motivated , You can also teach them some Sentry Basic knowledge of ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" New journey "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/05\/05783d1d0f878419369cea675b3ad316.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" With the elimination of noise , We are ready to move quickly . Here are some of the lessons we've learned from making these changes ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":" Appeal to business "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" If used properly , Rollback can make an error look like it never happened , It's a perfect complement to the quick fix strategy ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" The real atomic request "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Put operations into transactions as much as possible . open "},{"type":"link","attrs":{"href":"https:\/\/docs.djangoproject.com\/en\/3.1\/topics\/db\/transactions\/#tying-transactions-to-http-requests","title":null,"type":null},"content":[{"type":"text","text":"ATOMIC_REQUESTS"}]},{"type":"text","text":"( If it's not open ). however , Some requests do more than just change the database , For example, they send notifications , Join the backstage task team ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" stay SimpleLegal, We redesigned the architecture , All the side effects ( In addition to logging ) Delay until the response is returned successfully . Middleware can help , But we do it mainly through Redis The queue switches to be based on PostgreSQL Task queue for \/ Agent to achieve . This configuration ensures that , If an error occurs , The transaction will be rolled back , Tasks don't queue , Users will get a clean failure . We are Sentry Locate the fault in , Switch to the old site to eliminate , They will succeed the next time they try again ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Transactional test setup "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The fact proved that , Transactional is also critical to our testing strategy .SimpleLegal It has already surpassed Django The original fixture System . Most tests require complex Python Set up , This makes writing tests and running tests slow . To speed up writing and running , We encapsulate the entire test session into a single transaction , then , Before running any test cases , We set the basic state of the example . Test cases use these basic states as "},{"type":"link","attrs":{"href":"https:\/\/docs.pytest.org\/en\/stable\/fixture.html","title":null,"type":null},"content":[{"type":"text","text":"fixture"}]},{"type":"text","text":", And roll back to the basic state after each test case . Please refer to "},{"type":"link","attrs":{"href":"https:\/\/gist.github.com\/mahmoud\/10f6b6b0a9c5860030693357124131df","title":null,"type":null},"content":[{"type":"text","text":"contest.py excerpts "}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Some of the best practices are not for you "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Software scenarios are so different , It's an art to know which suggestions don't suit you . Here's what we've learned about the dead end ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" The use of namespace "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Considering that the code is divided into modules 、 package 、Django Application, etc , It can be tempting to use them as a unit of work . Don't start like this . Code partitioning can be very arbitrary , It's hard to know when you've got a risk ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" If there is automatic refactoring , As in the "},{"type":"link","attrs":{"href":"https:\/\/portingguide.readthedocs.io\/en\/latest\/","title":null,"type":null},"content":[{"type":"text","text":"2to3 transformation "}]},{"type":"text","text":" In the same , First of all, we need to migrate by conversion type . such , You just need to see a list of commands and affected paths . in addition , Automatic repair must follow a pattern , This means that more people can fix bugs caused by refactoring ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":" cover "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Coverage tools "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/57\/57cc3e421391def8f4e55d38a17bdba3.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Coverage is mixed for us . obviously , Coverage first strategy is untenable , But for prioritization and status checking , It still works . In the case of a single change , We found that coverage tools are a bit unreliable . We have never understood why the role of coverage is uncertain , We came to the conclusion that :“ image codecov Such ready-made tools may not be targeted at our scale monorepos.”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" In the process of hitting the coverage wall , We've looked at many other explanations for coverage . For us ,“ Routing coverage ”( each URL At least one integration test ) and “ Model representation covers ”( That is, each model object has a useful text representation , It can be used for Sentry debugging ) Much higher than row overlay priority . If there's more time , We'll want to build around these tools , Even around coverage statistics based on online analysis , Thus priority is given to the route with the highest traffic , It's not just the highest traffic lines . If you've heard of these methods , We'd like to discuss it with you ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Flat database migration "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" On the face of it , It seems reasonable to reduce the number of files that need to be upgraded . The fact proved that , flat "},{"type":"link","attrs":{"href":"https:\/\/docs.djangoproject.com\/en\/3.1\/topics\/migrations\/","title":null,"type":null},"content":[{"type":"text","text":" transfer "}]},{"type":"text","text":" It's a low-income strategy to eliminate files . Changing the structure of the historical migration file will complicate the go live process , Upgrading the migration file without flattening is very simple . Say nothing of , If you just want to accelerate CI, You can be like us in "},{"type":"link","attrs":{"href":"https:\/\/openedx.atlassian.net\/wiki\/spaces\/AC\/pages\/23003228\/Everything+About+Database+Migrations#EverythingAboutDatabaseMigrations-SquashingMigrations","title":null,"type":null},"content":[{"type":"text","text":"Open edX platform "}]},{"type":"text","text":" That's what I did on the Internet :"},{"type":"link","attrs":{"href":"https:\/\/github.com\/edx\/edx-platform\/blob\/66f0f9891f00994f77604a51dbb29736aa605fa8\/scripts\/reset-test-db.sh#L75","title":null,"type":null},"content":[{"type":"text","text":" Build a basic DB cache , Check every few months "}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The fact proved that ,"},{"type":"link","attrs":{"href":"https:\/\/sedimental.org\/awesome_python_applications.html#goal-1-a-better-development-cycle","title":null,"type":null},"content":[{"type":"text","text":" You can learn a lot from open source applications "}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Slowly adapt to the new technology stack "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" If you have multiple applications , Use a relatively small and simple application to experiment with the changes . Fortunately, , We have a separate application , Its tests run faster , This gives us a more compact understanding of the development cycle . similarly , If you have multiple production environments , Start with the environment with the least impact ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" hold CI Jobs are copied to the new technology stack , They all fail , But resist the urge to mark them as optional . contrary , Build a single file list of all tests and their current test status . We run programs for testing "},{"type":"link","attrs":{"href":"https:\/\/docs.pytest.org\/en\/stable\/","title":null,"type":null},"content":[{"type":"text","text":"pytest"}]},{"type":"text","text":" Built a little extension , It is based on the status list file to batch skip the test . then ,ratchet: Cancel and fix the test , Update file , Check whether the test passed , Then repeat . It's better than all over the code base "},{"type":"link","attrs":{"href":"https:\/\/docs.pytest.org\/en\/latest\/skipping.html#skipping-test-functions","title":null,"type":null},"content":[{"type":"text","text":"pytest Mark "}]},{"type":"text","text":" The decorator is more convenient and scannable . Please refer to "},{"type":"link","attrs":{"href":"https:\/\/gist.github.com\/mahmoud\/10f6b6b0a9c5860030693357124131df","title":null,"type":null},"content":[{"type":"text","text":"contest .py excerpts "}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" On line commissioning "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" stay 2020 Fourth quarter of 2007 , We've increased the infrastructure , To run the old and new sites in parallel with the same database support . We're in this cycle , Get traffic to the new technology stack , Build one that needs to be fixed Sentry Problem lines , Then close it , And track the time . Using the new technology stack is about 120 After one hour , After strategic expansion day and night , The organization has built up enough confidence , We can keep the site running at the most critical time : On Monday and Tuesday at the beginning of the month ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" The only problem is "},{"type":"link","attrs":{"href":"https:\/\/www.zdnet.com\/article\/aws-outage-impacts-thousands-of-online-services\/","title":null,"type":null},"content":[{"type":"text","text":"AWS The outage on Thanksgiving week "}]},{"type":"text","text":". At this point, we have completed the plan ahead of time , And built up enough confidence in the fast repair workflow , No longer need the original holiday test window . So , We thank a lot of people ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" We've been using a quick fix , Until we finish .“ complete ” It's not that there are no errors in the new system , It means that there are fewer events on the new system than on the old system . then , Continue to repair , And start scheduling time to remove scaffolding ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/7b\/7b1ef7b0ebac8d8b34b99a8bdb4b6ce7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":" Postscript "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" therefore , Once you use Django、Python、Linux and Postgres Current LTS edition , The task is done , Right ?"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Thank goodness , Technology debt never goes to 0. Although it is not a small matter to update and replace the core technology on schedule , But replacing rusty parts with shiny ones doesn't change the design . Architecture technology debt —— Mistakes in abstraction , Including the lack of abstraction —— It could be a bigger challenge . The solutions to these problems can't be fully generalized between projects , But they do benefit from this latest 、 The foundation of error free ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" For all projects that want to change tires , We hope that this review will help you confidently in the years to come 、 Improve the technology stack pragmatically ."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" Check the English text :"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/sedimental.org\/tech_refresh.html","title":null,"type":null},"content":[{"type":"text","text":"Changing the Tires on a Moving Codebase"}]}]}]}

版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢
https://chowdera.com/2021/03/20210322201740006D.html

随机推荐