且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

Standalone+ZooKeeper 高可用部署_4

更新时间:2022-09-04 07:50:24

开发者学堂课程【大数据实时计算框架 Spark 快速入门Standalone+ZooKeeper高可用部署_4】学习笔记,与课程紧密联系,让用户快速学习知识。

课程地址https://developer.aliyun.com/learning/course/100/detail/1658


Standalone+ZooKeeper 高可用部署_4

配置结果如下:

Standalone+ZooKeeper 高可用部署_4

把配置文件拷贝过去,如下图:

Standalone+ZooKeeper 高可用部署_4

启动 Zookeeper,如下图:

Standalone+ZooKeeper 高可用部署_4

需要停掉再重新启动,如下:

Standalone+ZooKeeper 高可用部署_4

状态为 STANDBY,说明集群就搭建好了,如下图:

Standalone+ZooKeeper 高可用部署_4

Overview

Utilizing Zookeeper to provide leader election and some state storage, you can launch multiple Masters in your cluster connected to the same ZooKeeper instance. One will be elected "leader" and the others will remain in standby mode. If the current leader dies, another Master will be elected, recover the old Master's state, and then resume scheduling. The entire recovery process (from the time the the first leader goes down) should take between 1 and 2 minutes. Note that this delay only affects scheduling new applications - applications that were already running during Master fallover are unaffected.

刷新一下,如下:

Standalone+ZooKeeper 高可用部署_4

启动原来的 Master,结果如下:

Standalone+ZooKeeper 高可用部署_4

跑应用程序 REPL

Standalone+ZooKeeper 高可用部署_4

这里默认创建了 Spark Context,也创建了 sqlContext,如下:

Standalone+ZooKeeper 高可用部署_4

现在可以写 scala 代码:

Standalone+ZooKeeper 高可用部署_4

把 scala 应用程序跑起来,刷新一下,结果如下:

Standalone+ZooKeeper 高可用部署_4

把 Master 停掉,如下:

Standalone+ZooKeeper 高可用部署_4

会有信息报备,如下

Standalone+ZooKeeper 高可用部署_4

并没有真的去执行这个文件,只是标记了一下,如下:

Standalone+ZooKeeper 高可用部署_4