Practice Test 4 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
Your company has TB’s of data stored on BigQuery. They have existing spark scripts for performing transformation and analysis on Cloud Data Proc. The output needs to be stored in BigQuery for future analysis. How can you set up BigQuery as an input and output source? Select 2 correct answers
A. Manually use a cloud storage bucket to import and export to and from both BigQuery and DataProc.
B. Install the BigQuery connector on your DataProc cluster.
C. Only Cloud Storage and HDFS can be used as DataProc input and output.
D. Specify the BigQuery connector in the jars parameter when submitting a job.
The correct answers are B & D.
BigQuery connector can be used as input and output source for DataProc cluster by using BigQuery connector. There are three ways BigQuery connectors can be used.
- By installing BigQuery connector using initialization action. This will install the Big Query connector while the cluster starts.
- Be specifying the BigQuery connector in the jars parameter when submitting a job. The jar could be placed on cloud storage and a path to the jar is provided.
- BigQuery connector classes can be included as dependencies in your code.
Option A is incorrect. Big Query connector can be used to set up input and output for BigQuery on your Dataproc cluster.
Option B is correct. Big Query connector can be installed on Dataproc cluster using initialization action.
Option C is incorrect. Using a BiqQuery connector, Dataproc can use BigQuery as input and output source.
Ref URL: https://cloud.google.com/dataproc/docs/tutorials/bigquery-connector-spark-example
Option D is correct. Users can specify the BiqQuery connector in the jars parameter when submitting a job.
For eg: –jars=gs://hadoop-lib/bigquery/bigquery-connector-hadoop2-latest.jar
Comments are closed, but trackbacks and pingbacks are open.