当前位置:网站首页>Flink on PaaSTA:Yelp运行在Kubernetes上的新流处理平台
Flink on PaaSTA:Yelp运行在Kubernetes上的新流处理平台
2020-11-06 01:15:27 【InfoQ】
{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"本文最初发布于yelp工程博客,由InfoQ中文站翻译并分享。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Yelp,我们每天使用"},{"type":"link","attrs":{"href":"https:\/\/flink.apache.org\/","title":"","type":null},"content":[{"type":"text","text":"Apache Flink"}]},{"type":"text","text":"处理TB级的流数据,为各种各样的应用提供支持:ETL管道、推送通知、机器人过滤、Session化等等。我们运行成百上千的Flink作业,因此,如果没有适当程度的自动化,像部署、重启和"},{"type":"link","attrs":{"href":"https:\/\/ci.apache.org\/projects\/flink\/flink-docs-release-1.11\/ops\/state\/savepoints.html","title":"","type":null},"content":[{"type":"text","text":"保存点"}]},{"type":"text","text":"这样的常规操作会花费开发人员数千小时的时间。最近,我们的工具室中增加了一个新的流处理平台,它基于Yelp的PaaS服务"},{"type":"link","attrs":{"href":"https:\/\/engineeringblog.yelp.com\/2015\/11\/introducing-paasta-an-open-platform-as-a-service.html","title":"","type":null},"content":[{"type":"text","text":"PaaSTA"}]},{"type":"text","text":"。其核心是一个"},{"type":"link","attrs":{"href":"https:\/\/kubernetes.io\/","title":"","type":null},"content":[{"type":"text","text":"Kubernetes"}]},{"type":"link","attrs":{"href":"https:\/\/kubernetes.io\/docs\/concepts\/extend-kubernetes\/operator\/","title":"","type":null},"content":[{"type":"text","text":"Operator"}]},{"type":"text","text":",它自动监视我们的Flink集群的fleet部署和生命周期。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/94\/1e\/94e5ab8a26535ffba733bf8fe61b441e.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"Flink on PaaSTA on Kubernetes"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"引入Kubernetes之前"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Yelp引入Kubernetes之前,Yelp的Flink工作负载运行在专用的AWS"},{"type":"link","attrs":{"href":"https:\/\/aws.amazon.com\/emr\/","title":"","type":null},"content":[{"type":"text","text":"ElasticMapReduce"}]},{"type":"text","text":"集群上,这些集群预装了Flink和"},{"type":"link","attrs":{"href":"https:\/\/hadoop.apache.org\/docs\/current\/hadoop-yarn\/hadoop-yarn-site\/YARN.html","title":"","type":null},"content":[{"type":"text","text":"YARN"}]},{"type":"text","text":"。为了实现EMR实例与Yelp生态系统其余部分的良好协同,我们之前的流处理平台Cascade在一个"},{"type":"link","attrs":{"href":"https:\/\/www.docker.com\/","title":"","type":null},"content":[{"type":"text","text":"Docker"}]},{"type":"text","text":"容器中运行大量的"},{"type":"link","attrs":{"href":"https:\/\/puppet.com\/docs\/pe\/2019.8\/peuser<\/i>guide.html","title":"","type":null},"content":[{"type":"text","text":"Puppet"}]},{"type":"text","text":"单体,以应用配置并启动一组常见的守护进程(在Yelp几乎所有的主机上运行)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/ec\/44\/ec553349f86e55448f70c38bd7553544.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"Cascade的架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢
https://www.infoq.cn/article/T47JE170VyVtywl19Z1H?utm_source=rss&utm_medium=article
边栏推荐
- C++ 数字、string和char*的转换
- C++学习——centos7上部署C++开发环境
- C++学习——一步步学会写Makefile
- C++学习——临时对象的产生与优化
- C++学习——对象的引用的用法
- C++编程经验(6):使用C++风格的类型转换
- Won the CKA + CKS certificate with the highest gold content in kubernetes in 31 days!
- C + + number, string and char * conversion
- C + + Learning -- capacity() and resize() in C + +
- C + + Learning -- about code performance optimization
猜你喜欢
-
C + + programming experience (6): using C + + style type conversion
-
Latest party and government work report ppt - Park ppt
-
在线身份证号码提取生日工具
-
Online ID number extraction birthday tool
-
️野指针?悬空指针?️ 一文带你搞懂!
-
Field pointer? Dangling pointer? This article will help you understand!
-
HCNA Routing&Switching之GVRP
-
GVRP of hcna Routing & Switching
-
Seq2Seq实现闲聊机器人
-
【闲聊机器人】seq2seq模型的原理
随机推荐
- LeetCode 91. 解码方法
- Seq2seq implements chat robot
- [chat robot] principle of seq2seq model
- Leetcode 91. Decoding method
- HCNA Routing&Switching之GVRP
- GVRP of hcna Routing & Switching
- HDU7016 Random Walk 2
- [Code+#1]Yazid 的新生舞会
- CF1548C The Three Little Pigs
- HDU7033 Typing Contest
- HDU7016 Random Walk 2
- [code + 1] Yazid's freshman ball
- CF1548C The Three Little Pigs
- HDU7033 Typing Contest
- Qt Creator 自动补齐变慢的解决
- HALCON 20.11:如何处理标定助手品质问题
- HALCON 20.11:标定助手使用注意事项
- Solution of QT creator's automatic replenishment slowing down
- Halcon 20.11: how to deal with the quality problem of calibration assistant
- Halcon 20.11: precautions for use of calibration assistant
- “十大科学技术问题”揭晓!|青年科学家50²论坛
- "Top ten scientific and technological issues" announced| Young scientists 50 ² forum
- 求反转链表
- Reverse linked list
- js的数据类型
- JS data type
- 记一次文件读写遇到的bug
- Remember the bug encountered in reading and writing a file
- 单例模式
- Singleton mode
- 在这个 N 多编程语言争霸的世界,C++ 究竟还有没有未来?
- In this world of N programming languages, is there a future for C + +?
- es6模板字符
- js Promise
- js 数组方法 回顾
- ES6 template characters
- js Promise
- JS array method review
- 【Golang】️走进 Go 语言️ 第一课 Hello World
- [golang] go into go language lesson 1 Hello World