Problem Scenario 95 : You have to run your Spark application on yarn with each executor Maximum heap size to be 512MB and Number of processor cores to allocate on each executor will be 1 and Your main application required three values as input arguments V1 V2 V3.
Please replace XXX, YYY, ZZZ
./bin/spark-submit -class com.hadoopexam.MyTask --master yarn-cluster--num-executors 3
--driver-memory 512m XXX YYY lib/hadoopexam.jarZZZ
Answer: See the explanation for Step by Step Solution and configuration.
Explanation: Solution XXX: -executor-memory 512m YYY: -executor-cores 1
ZZZ : V1 V2 V3
Notes : spark-submit on yarn options Option Description archives Comma-separated list of archives to be extracted into the working directory of each executor. The path must be globally visible inside your cluster; see Advanced
Dependency Management. Executor-cores Number of processor cores to allocate on each executor. Alternatively, you can use the spark.executor.cores property, executor-memory Maximum heap size to allocate to each executor. Alternatively, you can use the spark.executor.memory-property. num-executors Total number of YARN containers to allocate for this application. Alternatively, you can use the spark.executor.instances property. queue YARN queue to submit to. For more information, see Assigning Applications and Queries to Resource Pools. Default: default.
It is always consider difficult to pass CCA Spark and Hadoop Developer Exam because of the complexity of the topics in the course. CCA175 Dumps study guide is really useful in understanding these concepts comprehensive for best performance in the final. My experience with these dumps has been very good and will specifically mention the support and directions that was provided by the experts on Dumps4download. Practice tests helped me a lot in boosting my performance in the actual exam settings which enhanced my confidence. I am thankful to everyone working for designing such a handy guide for preparation of all IT certifications. I recommend everyone to use CCA175 for preparation. My best wishes are for all my fellows.
ReplyDelete