自动化 Web 性能优化分析方案

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在构建 Web 站点的过程中，任何一个细节都有可能影响网站的访问速度。如果开发人员不了解前端性能相关知识，很多不利网站访问速度的因素会在线上形成累加，从而严重影响网站的性能，导致网站访问速度变慢、用户体验低下，最终导致用户流失。页面性能对网页而言，可谓举足轻重。因此，对页面的性能进行检测分析，是开发者不可忽视的课题。那么我们如何对页面进行监控分析及性能评判？对性能评判的规则又是什么样的呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从技术方面来讲，前端性能监控主要分为两种方式，一种叫做合成监控（Synthetic Monitoring，SYN），另一种是真实用户监控（Real User Monitoring，RUM）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合成监控，就是在一个模拟场景里，去提交一个需要做性能检测的页面，通过一系列的工具、规则去运行你的页面，提取一些性能指标，得出一个性能报告。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"真实用户监控，就是用户在我们的页面上浏览，浏览过程就会产生各种各样的性能数据，我们把这些性能数据上传到我们的日志服务器上，进行数据的提取清洗加工，最后在我们的监控平台上进行展示的一个过程。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前者注重“检测”，后者注重“监控”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下文将从多个方面，向大家介绍政采云前端 ZooTeam 的 Web 性能优化分析系统—— “百策”，是如何采集页面性能数据，并通过一系列计算加工，产出页面性能报表的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"百策，取名自历史人物魏征。魏征，字玄成。唐太宗李世民的谏臣，因敢于直谏，被称为诤臣。因多次指出唐太宗李世民的过错并能提出有效政策，因为被称为 “魏百策”。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"如何采集页面性能数据"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"百策系统采用了上文提到的 “合成监控” 的方案，得到了一组与宿主环境无关的性能数据。说到合成监控方案，当属 Google Chrome 团队出品的开源自动化分析工具 Lighthouse。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Lighthouse 的工作流程有几个主要的步骤。部分步骤发生在浏览器中，其余的步骤由 Lighthouse 运行器执行。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Lighthouse 工作原理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/98\/98c1c7fac13f7b5242e45a3577b02c7a.webp","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面是 Lighthouse 的组成部分："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"驱动"},{"type":"text","text":" 和 Chrome Debugging Protocol 进行交互。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"收集器"},{"type":"text","text":"  使用驱动程序收集网页信息。最小化后处理。收集器的输出结果被称为 "},{"type":"text","marks":[{"type":"strong"}],"text":"Artifact"},{"type":"text","text":"。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"审查器"},{"type":"text","text":" 将 "},{"type":"text","marks":[{"type":"strong"}],"text":"Artifact"},{"type":"text","text":" 作为输入，审查器会对其运行 1 个测试，然后分配通过\/失败\/得分的结果。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"类别"},{"type":"text","text":"   将审查的结果分组到面向用户的报告中（如最佳实践）。对该部分加权求和然后得出评分。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"简单来说流程就是：建立连接 -> 收集日志 -> 分析 -> 生成报告。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而我们的百策系统是在 Lighthouse 的基础上，进行了部分功能的定制和创新。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"区别于 Lighthouse 的功能"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1）使用 Puppeteer"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Puppeteer 是一个 Node 库，它提供了一个高级 API 来通过 DevTools 协议控制 Chromium 或 Chrome。相比较 "},{"type":"codeinline","content":[{"type":"text","text":"Selenium"}]},{"type":"text","text":" 或是 "},{"type":"codeinline","content":[{"type":"text","text":"PhantomJs"}]},{"type":"text","text":" ，它最大的特点就是它的 "},{"type":"codeinline","content":[{"type":"text","text":"DOM"}]},{"type":"text","text":" 操作可以完全在内存中进行模拟，即在 "},{"type":"codeinline","content":[{"type":"text","text":"V8"}]},{"type":"text","text":" 引擎中处理而不打开浏览器，而且 "},{"type":"codeinline","content":[{"type":"text","text":"V8"}]},{"type":"text","text":" 引擎 Chrome 团队在维护，会拥有更好的兼容性和前景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"运行下面这段代码就能获取到视窗可视区域的宽高："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"javascript"},"content":[{"type":"text","text":"const puppeteer = require('puppeteer');\n\n(async () => {\n  const browser = await puppeteer.launch();\n  const page = await browser.newPage();\n  await page.goto('https:\/\/example.com');\n\n  const dimensions = await page.evaluate(() => {\n    return {\n      width: document.documentElement.clientWidth,\n      height: document.documentElement.clientHeight\n    };\n  });\n\n  console.log('Dimensions:', dimensions);\n\n  await browser.close();\n})();\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Lighthouse 的方案是使用更底层的 DevTools 协议和浏览器交互，我们选择 Puppeteer 是因为其更简单，封装的一些常用的接口足够满足我们的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2）访问页面前模拟登录"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们不仅需要检测可匿名访问的前台页面，也需要检测登录后才能操作的后台页面。所以在检测前，我们需要增加一步登录操作。当然这个用户名密码也是可选的，当检测到这个 URL 需要登录时，才会提醒用户输入用户名密码。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/87\/879a471a1b6f0edecd7650528395ac59.webp","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3）收集到数据存入数据库"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们将每一次的检测数据保存入库，对数据进行整理，获取页面性能变化所呈现的趋势，为支持后续的统计分析提供数据保证。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"改造之后的流程是：建立连接 -> "},{"type":"text","marks":[{"type":"strong"}],"text":"登录检测"},{"type":"text","text":" -> 收集日志 -> "},{"type":"text","marks":[{"type":"strong"}],"text":"数据入库"},{"type":"text","text":" -> 分析 -> 生成报告"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"如何分析页面性能"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"定制各项指标"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"系统设计初期，我们拟定了一系列性能指标的权重和阈值："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/af\/afbad748a60aeafd2acb3a1c36f22259.webp","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Lighthouse 也制定了很多性能指标，譬如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"uses-webp-images：采用新一代格式提供图片"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"dom-size：DOM 节点总数"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"network-requests：页面加载完成时的请求数"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"……"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除此之外我们定制了更多的特色指标，譬如：OSS 图片是否使用了压缩后缀。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于公司的图片都是放在阿里云 OSS 上的，阿里云 OSS 提供了很方便的优化图片体积的方法，就是给图片的末尾加上类似参数 "},{"type":"codeinline","content":[{"type":"text","text":"?x-oss-process=image\/resize,w_187\/quality,Q_75\/format,webp"}]},{"type":"text","text":"，理论上放在 OSS 上的图片都需要添加压缩后缀。为什么说是理论上呢？因为我们发现如果图片上传 OSS 前，事先经过了其他压缩工具的压缩，再添加压缩后缀后，图片的体积变化不大，甚至会出现负增长。所以我们调整了评审算法，不再是简单地检查图片是否增加了 OSS 后缀，而是比较当前图片的体积与添加 OSS 压缩后缀后体积的差值是否超出比例。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"定制检测模型"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"检测系统试运行一段时间后，我们发现并不是所有的指标都适用于所有类型的页面。譬如我们的前台页面加载图片很多且对图片分辨率要求较高、而后台页面图片则比较少。我们既有传统的后端渲染 + jQuery 类型的页面、客户端渲染 React 页面、也有经过预渲染的 Vue 页面。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"将所有类型的页面按同一标准进行检测显然不太合适，所以我们引入了 "},{"type":"text","marks":[{"type":"strong"}],"text":"检测模型"},{"type":"text","text":" 的概念。我们根据页面所属的类型，按照对应检测模型进行性能检测。一个检测模型可以关联多个指标，并可以配置指标权重和级别。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/b1\/b198642af104b1fae93998959d1f939d.webp","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"权重越大扣分越多，级别分为 error 和 warning，其中 warning 级别的扣分项不算入总分中，譬如请求是否使用 Gzip 就是，因为这一项是前端无法优化的，需要推动服务端去改进。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"计算页面评分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/86\/86edc99caf178b42906912851e6770c0.jpeg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"提供优化建议和趋势"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"优化建议"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"针对每项扣分的审查项，我们都提供了详细的扣分原因，以及对应的解决方案："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/62\/62d35d9cf28889d8734991673455dc9f.webp","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"页面性能趋势变化图"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了开发新的功能，我们还积极地对老页面进行重构，为了分析重构前后的收益，我们还提供了查询历史数据和变化趋势的功能："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/1e\/1e5e68d2966ceed4333bad6e1dc795d9.webp","alt":"Image","title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/83\/837bc2d86f87e8cfe41bcc50737d1768.webp","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"总结"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"百策系统为前端页面提供性能优化分析服务，结合行业方案及业务场景的差异性，完善符合公司业务情况的最佳实践，百策系统制定了一套 Web 页面性能的检测标准，通过页面检测及模型分析，发现页面中请求耗时过长、请求资源过大、页面结构不合理等影响用户体验的问题，并针对问题提供相应的优化建议，帮助开发者产出渲染速度更快、资源占有更少、体验更好的页面。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"头图：Unsplash"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：政采云前端团队"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：https:\/\/mp.weixin.qq.com\/s\/2CHA5ewWz_SIlBrdEuVv7w"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：自动化 Web 性能优化分析方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"来源：政采云前端团队 - 微信公众号 [ID：Zoo-Team]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"转载：著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。"}]}]}