当前位置:网站首页>Flink's sink: a preliminary study

Flink's sink: a preliminary study

2020-11-08 16:12:35 Programmer Xinchen

Welcome to visit mine GitHub

https://github.com/zq2599/blog_demos

Content : All original articles classified summary and supporting source code , involve Java、Docker、Kubernetes、DevOPS etc. ;

About sink

The following figure comes from Flink official , In the red box is sink, Visible real-time data from Source Start at , stay Transformation After the business logic is completed in sink end , therefore sink It can be used to process the calculation results , For example, console output or save database :  Insert picture description here

About 《Flink Of sink actual combat 》 Series articles

This article is about 《Flink Of sink actual combat 》 The first chapter of , In order to have a preliminary understanding of sink, Through the basic API and addSink Method analysis and research , Lay a good foundation for the subsequent coding practice ;

Full series Links

  1. 《Flink Of sink One of the real battles : On 》
  2. 《Flink Of sink The second part of the actual battle :kafka》
  3. 《Flink Of sink The third part of the actual battle :cassandra3》
  4. 《Flink Of sink The fourth part of the actual battle : Customize 》

Start with an instance code

  1. Here is a simple flink Application code , The red box print The way is sink operation :  Insert picture description here
  2. Here's the official sink The way , All are DataStream Class API, It can be directly called sink, Just in the code print It's one of them :  Insert picture description here
  3. Let's look at the picture above API Source code , First look at print Method , stay DataStream.java in , as follows , It's actually called addSink Method , Participation is PrintSinkFunction:  Insert picture description here
  4. The other is often used API yes writeAsText, Source code is as follows , Called writeUsingOutputFormat Method :  Insert picture description here
  5. track <font color="blue">writeUsingOutputFormat</font> Discovery is also called addSink, Participation is <font color="blue">OutputFormatSinkFunction</font>:  Insert picture description here
  6. print and writeAsText Behind the call addSink, So another common <font color="blue">writeAsCsv</font> Methods? ? It's a call addSink? Open it and see , and writeAsText Same call <font color="blue">writeUsingOutputFormat</font>, In this method, you are calling addSink:  Insert picture description here
  7. in summary ,data sink The key is <font color="blue">addSink</font> Input , namely <font color="blue">SinkFunction</font> Interface implementation , Use class diagrams to visualize common sink How is ability realized :

 Insert picture description here 8. From the figure above, we can see the abstract class <font color="blue">RichSinkFunction</font> With all kinds of sink Ability is closely related , We should focus on it , Show method signature on class diagram , Here's the picture :  Insert picture description here 9. As shown in the figure above ,<font color="blue">RichSinkFunction</font> There is no content in itself , But it does <font color="blue">SinkFunction</font>, Inherit <font color="blue">AbstractRichFunction</font>, yes <font color="blue">RichFunction</font> and <font color="blue">SinkFunction</font> The combination of these two characteristics ; 10. <font color="blue">RichFunction</font> The characteristics of are in front of 《Flink Of DataSource Trilogy 》 We already know , It's about resources open and close; 11. <font color="blue">SinkFunction</font> The characteristics of ? It's obviously used to process the calculation results , The class diagram shows two invoke Method , Take a look at the official <font color="blue">PrintSinkFunction.java</font>:  Insert picture description here 12. <font color="blue">writer.write(record)</font> The source code is PrintSinkOutputWriter.java, As shown below :  Insert picture description here

Summary

thus , We're right Flink Of sink With a basic understanding :

  1. Be responsible for the processing of real-time calculation results ( Like output or persistence );
  2. The main implementation is to call DataStream.addSink Method ;
  3. Various sink The realization of ability , The main way is to achieve addSink The interface defined by the input parameter of the method ;

Later chapters , Together with sink Let's practice the coding of the aspect , The direction of actual combat : Experience the official sink Ability , Customize sink Ability to achieve ;

Welcome to the official account : Xinchen, programmer

WeChat search 「 Xinchen, programmer 」, I'm Xinchen , Looking forward to traveling with you Java The world ... https://github.com/zq2599/blog_demos

版权声明
本文为[Programmer Xinchen]所创,转载请带上原文链接,感谢