当前位置:网站首页>Flink's sink: a preliminary study

Flink's sink: a preliminary study

2020-11-08 09:44:53 Programmer Xinchen

Welcome to visit mine GitHub

https://github.com/zq2599/blog_demos

Content : All original articles classified summary and supporting source code , involve Java、Docker、Kubernetes、DevOPS etc. ;

About sink

The following figure comes from Flink official , In the red box is sink, Visible real-time data from Source Start at , stay Transformation After the business logic is completed in sink end , therefore sink It can be used to process the calculation results , For example, console output or save database :
 Insert picture description here

This article is about 《Flink Of sink actual combat 》 The first chapter of , In order to have a preliminary understanding of sink, Through the basic API and addSink Method analysis and research , Lay a good foundation for the subsequent coding practice ;

  1. 《Flink Of sink One of the real battles : On 》
  2. 《Flink Of sink The second part of the actual battle :kafka》
  3. 《Flink Of sink The third part of the actual battle :cassandra3》
  4. 《Flink Of sink The fourth part of the actual battle : Customize 》

Start with an instance code

  1. Here is a simple flink Application code , The red box print The way is sink operation :
     Insert picture description here
  2. Here's the official sink The way , All are DataStream Class API, It can be directly called sink, Just in the code print It's one of them :
     Insert picture description here
  3. Let's look at the picture above API Source code , First look at print Method , stay DataStream.java in , as follows , It's actually called addSink Method , Participation is PrintSinkFunction:
     Insert picture description here
  4. The other is often used API yes writeAsText, Source code is as follows , Called writeUsingOutputFormat Method :
     Insert picture description here
  5. track writeUsingOutputFormat Discovery is also called addSink, Participation is OutputFormatSinkFunction
     Insert picture description here
  6. print and writeAsText Behind the call addSink, So another common writeAsCsv Methods? ? It's a call addSink? Open it and see , and writeAsText Same call writeUsingOutputFormat, In this method, you are calling addSink:
     Insert picture description here
  7. in summary ,data sink The key is addSink Input , namely SinkFunction Interface implementation , Use class diagrams to visualize common sink How is ability realized :

 Insert picture description here
8. From the figure above, we can see the abstract class RichSinkFunction With all kinds of sink Ability is closely related , We should focus on it , Show method signature on class diagram , Here's the picture :
 Insert picture description here
9. As shown in the figure above ,RichSinkFunction There is no content in itself , But it does SinkFunction, Inherit AbstractRichFunction, yes RichFunction and SinkFunction The combination of these two characteristics ;
10. RichFunction The characteristics of are in front of 《Flink Of DataSource Trilogy 》 We already know , It's about resources open and close;
11. SinkFunction The characteristics of ? It's obviously used to process the calculation results , The class diagram shows two invoke Method , Take a look at the official PrintSinkFunction.java
 Insert picture description here
12. writer.write(record) The source code is PrintSinkOutputWriter.java, As shown below :
 Insert picture description here

Summary

thus , We're right Flink Of sink With a basic understanding :

  1. Be responsible for the processing of real-time calculation results ( Like output or persistence );
  2. The main implementation is to call DataStream.addSink Method ;
  3. Various sink The realization of ability , The main way is to achieve addSink The interface defined by the input parameter of the method ;

Later chapters , Together with sink Let's practice the coding of the aspect , The direction of actual combat : Experience the official sink Ability , Customize sink Ability to achieve ;

Welcome to the official account : Xinchen, programmer

WeChat search 「 Xinchen, programmer 」, I'm Xinchen , Looking forward to traveling with you Java The world ...
https://github.com/zq2599/blog_demos

版权声明
本文为[Programmer Xinchen]所创,转载请带上原文链接,感谢