Practice Test 2 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
A company decided to migrate their on-premise hadoop jobs to Google Cloud. As recommended by Google Cloud engineers, Dataproc is used to run Apache Hive jobs. Data residing in on-premise HDFS has been moved to Google Storage and connector was used for Dataproc to read the data. Upon monitoring the performance of Dataproc clusters running Hive jobs, you noticed the jobs are I/O intensive and use local disk to read/write data. This leads to performance issues. How can you solve this problem?
A. Increase persistent disk size for master node.
B. Increase persistent disk size for worker nodes.
C. Increase RAM capacity of Dataproc cluster’s worker nodes.
D. Use local HDFS storage of Dataproc cluster nodes instead of Google Storage.
When you want to move Hadoop & Spark workloads from an on-premises environment to Google Cloud Platform (GCP), It’s recommended to use Dataproc to run Apache Spark & Hadoop clusters. Local HDFS storage is a good option if you have workloads that involve heavy I/O. For example, you have a lot of partitioned writes. It is a good option if you also have I/O workloads that are especially sensitive to latency. For example, you require single-digit millisecond latency per storage operation.
Option A is incorrect: Increasing disk size for master node will not help with the performance issue.
Option B is incorrect: Increasing disk size for worker nodes alone is not enough. You should move data to local HDFS storage of Dataproc. Increasing size may help to increase HDFS storage.
Option C is incorrect: Increasing memory will not help fix the issue because the problem is because of intensive disk read/write.
Source(s):
Migrating Apache Spark Jobs to Cloud Dataproc:
https://cloud.google.com/solutions/migration/hadoop/migrating-apache-spark-jobs-to-cloud-dataproc
Comments are closed, but trackbacks and pingbacks are open.