In cluster mode, the Spark driver runs inside an application master process which is managed by YARN on the cluster, and the client can go away after initiating the application. In client mode, the driver runs in the client process, and the application master is only used for requesting resources from YARN.
- Client mode, driver will be running in the machine where application got submitted and the machine has to be available in the network till the application completes.
- Cluster mode, driver will be running in application master(one per spark application) node and machine submitting the application need not to be in network after submission
Client mode
Cluster mode
If Spark application is submitted with cluster mode on its own resource manager(standalone) then the driver process will be in one of the worker nodes.
No comments:
Post a Comment