当前位置:网站首页>Installation and deployment of Flink

Installation and deployment of Flink

2020-11-09 17:46:45 Irving the procedural ape

One . Flink The download

Installation package download address , Select the corresponding Hadoop Of Flink Version download

 

 

 

 Flink  There are three deployment modes , Namely  Local、Standalone Cluster  and  Yarn Cluster.

Two . Local Pattern

about  Local  Mode ,JobManager  and  TaskManager  Will share one  JVM  To complete  Workload. If you want to validate a simple application ,Local  Patterns are the most convenient . Most of the practical applications use  Standalone  perhaps  Yarn Cluster, and local Mode just unzip the installation package and start it up (./bin/start-local.sh) that will do , There's no demonstration here .

3、 ... and . Standalone HA Pattern

Standalone Pattern as the name suggests , It is scheduled to execute on the local cluster , Independent of external scheduling mechanisms such as YARN,  Generally, it needs to be configured as HA, prevent Jobmanager All of a sudden , The whole cluster or task execution fails . So let's talk about that Standalone HA Mode building and installation

When Flink Program runtime , If jobmanager collapse , Then the whole program will fail . In order to prevent jobmanager A single point of failure , With the help of zookeeper The coordination mechanism of , Can achieve jobmanager Of HA To configure —-1 Lord (leader) More from (standby). there HA Configuration only involves standalone Pattern ,yarn The mode will not be considered for the moment . 

In this case, plan Jobmanager:hadoop01,hadoop02( One active, One standby);Taskmanager:hadoop02,hadoop03;zookeeper colony

1.   Cluster deployment planning

The name of the node master worker zookeeper
hadoop01 master  worker zookeeper
hadoop02 master worker zookeeper
hadoop03   woker zookeeper

 

 

 

 

 

 

 

2.  decompression

[hadoop@hadoop01 apps]$ tar -zxvf flink-1.7.2-bin-scala_2.11.tgz -C ./

  [hadoop@hadoop01 apps]$ ls
  azkaban flink-1.7.2 flink-1.7.2-bin-scala_2.11.tgz flume-1.8.0 hadoop-2.7.4 jq kafka_2.11-0.11 zkdata zookeeper-3.4.10 zookeeper.out

3.  Modify the configuration file

To configure masters file

   This file is used to specify the master node and its web Access port , For clusters Jobmanager,vi masters, add to master:8081

[hadoop@hadoop01 conf]$ vim masters hadoop01:8081
hadoop02:8081

To configure slaves file , This file is used to specify the slave node , For clusters taskManager. Add the following

[hadoop@hadoop01 conf]$ vim slaveshadoop01hadoop02hadoop03

The configuration file flink-conf.yaml

版权声明
本文为[Irving the procedural ape]所创,转载请带上原文链接,感谢