Practice Test 2 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
You have a Dataflow pipeline which streams data to be stored to BigTable after it has been transformed and enriched. Development team needs to modify the transformation code based on client’s needs. The pipeline is in production which keeps streaming and any interruption to the pipeline may lead to data loss or unexpected output.
How can you make sure the pipeline can be stopped without any consequences?
A. Turn off Dataflow pipeline with ‘cancel’ option.
B. Create a new Dataflow pipeline with the new transformation code, then switch data stream to the new pipeline.
C. Transfer Dataflow pipeline to write data to Google Storage. Perform the needed changes then transfer pipeline back to write to BigTable and re-process the data written in Google Storage.
D. Turn off Dataflow pipeline with ‘drain’ option.
Answer: D.
Using the Drain option to stop your job tells the Cloud Dataflow service to finish your job in its current state. Your job will immediately stop ingesting new data from input sources. When all pending processing and write operations are complete, the Cloud Dataflow service will clean up the GCP resources associated with your job.
Because Cancel option immediately halts processing, you may lose any “in-flight” data. “In-flight” data refers to data that has been read but is still being processed by your pipeline.
Answer A is incorrect: Using ‘Cancel’ option will lead to losing in-flight data.
Answer B & C are incorrect: Those options are not necessary.
Source(s):
Dataflow – Stopping a running pipeline: https://cloud.google.com/dataflow/docs/guides/stopping-a- pipeline
Comments are closed, but trackbacks and pingbacks are open.