Practice Test 3 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
You have a Dataflow pipeline which streams data to be stored to BigQuery. It is required to deploy a new Dataflow pipeline. If the new pipeline is deployed, current pending data logs will fail and pushed to dead-letter Cloud Storage bucket. Instead, current data logs should be processed before deploying the new pipeline while pausing the streaming. How would you achieve this?
A. Create a new Dataflow pipeline with the new version, then switch the data stream to the new pipeline.
B. Deploy the new pipeline while keeping the old one. Let failed data logs be transferred to Cloud Storage “dead-letter” bucket. Change the current pipeline data source to be the dead-letter bucket.
C. Turn off Dataflow pipeline with ‘drain’ option.
D. Turn off Dataflow pipeline with ‘cancel’ option.
Correct Answer: C
Using the Drain option to stop your job tells the Cloud Dataflow service to finish your job in its current state. Your job will stop immediately ingesting new data from input sources. When all pending processing and write operations are complete, the Cloud Dataflow service will clean up the GCP resources associated with your job.
Because Cancel option immediately halts processing, you may lose any “in-flight” data. “In-flight” data refers to data that has been read but is still being processed by your pipeline.
Options A and B are incorrect: These are not the required options.
Option D is incorrect: Using the ‘Cancel’ option will lead to losing in-flight data.
Source(s):
Dataflow – Stopping a running pipeline:
https://cloud.google.com/dataflow/docs/guides/stopping-a-pipeline
Comments are closed, but trackbacks and pingbacks are open.