Practice Test 1 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
You use BigQuery as the main data warehouse. You decided to perform advanced data transformation of the data. You want to use Dataproc with Apache Spark to do the transformation. How can you enable Dataproc’s access to data in BigQuery?
A. Install Dataproc’s BigQuery connector on the cluster using initialization actions. Dataproc temporarily loads data from BigQuery to Google Storage. If failed, Dataproc deletes temp files before finishing the job.
B. Install Dataproc’s BigQuery connector on the cluster using initialization actions. Dataproc temporarily loads data from BigQuery to Google Storage. If failed, you need to manually delete temp files.
C. Dataproc cannot directly connect to BigQuery. You should export data from BigQuery to Google Storage first. Dataproc can then read data from Google Storage. You need to manually delete data files after Dataproc is done.
D. Dataproc can connect to BigQuery if you set the cluster as owner to the dataset.
Answer: B.
You can use a BigQuery connector to enable programmatic read/write access to BigQuery. This is an ideal way to process data that is stored in BigQuery. No command-line access is exposed. The BigQuery connector is a Java library that enables Hadoop to process data from BigQuery using abstracted versions of the Apache Hadoop InputFormat and OutputFormat classes.
You can access BigQuery from Dataproc by installing BigQuery connector to Dataproc cluster using initialization actions. When a Dataproc spark job reads from BigQuery, it writes the BigQuery table’s content temporarily to Google Storage using Dataproc cluster’s assigned bucket. If the job completes successfully, temporary files are automatically deleted from the cluster. If the job fails, you need to delete temp files manually.
Answer A is incorrect: If job fails, you need to delete temp files manually.
Answer C is incorrect: Dataproc can read from BigQuery by installing the connector. No need to export data from BigQuery to Google Storage manually.
Answer D: Dataproc cannot directly read from BigQuery without installing the connector first.
Source(s): https://cloud.google.com/dataproc/docs/concepts/connectors/bigquery https://cloud.google.com/dataproc/docs/concepts/connectors/bigquery
https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/init-actions
Comments are closed, but trackbacks and pingbacks are open.