Practice Test 3 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
The management team of your company decides to migrate their on-premise stack to Google Cloud. You have been assigned the task of migrating on-premise HDFS data and Apache Hive clusters. DevOps team requires that they should be able to still manage the components used in Google Cloud. What is the best approach for this scenario?
A. Move on-premise HDFS data to Google Storage. Use Dataproc to process the data.
B. Move on-premise HDFS data to Google Storage. Build a Dataflow pipeline. Use Compute Engine to launch instances and install the required dependencies for processing the data.
C. Move on-premise HDFS data to Dataproc’s persistent disks. Use Dataproc to process the data.
D. Move on-premise HDFS data to Dataproc’s persistent disks. Build a Dataflow pipeline. Execute the code in Spark framework provided by Dataflow.
Correct Answer: A
Dataproc is a cloud-native Apache Hadoop & Apache Spark service. Dataproc is a fully-managed service from Google to run Apache Hadoop & Spark clusters.
Dataflow is a simplified streaming/batching data processing service. With Apache Beam, it provides a rich set of windowing and session analysis primitives as well as an ecosystem of source & sink connectors.
Option B is incorrect: Dataflow is serverless which may not suit the DevOps requirement to fully manage the pipeline and it’s unnecessary to use Cloud Compute for installing dependencies.
Option C is incorrect: Dataproc’s HDFS is volatile, so it will be removed when the cluster is deleted. Dataproc clusters can be kept up indefinitely but this may lead to high costs which defeat the purpose of migration.
Option D: Due to reasons discussed in the options B & C.
Source(s):
Cloud Dataproc:
https://cloud.google.com/dataproc/
Cloud Dataflow:
Comments are closed, but trackbacks and pingbacks are open.