Practice Test 2 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test
You have a system which writes its logs to a file. The system writes to a new log file every hour. You want to merge log files every 24 hours to a single file and upload it to a specific bucket in Google Storage. You created a Dataflow pipeline to process the logs files and you want to run this pipeline everyday at 3am.
Which of the following is the best approach to schedule this task?
A. Use Cloud Scheduler to create a cron job to run the Dataflow pipeline at 3am.
B. Create a Compute Engine VM and create a cron job to run Dataflow pipeline at 3am.
C. Configure Dataflow pipeline as a streaming job to process the data in real time.
D. Use Cloud Functions to run Dataflow pipeline at 3am.
Answer: A.
Cloud Scheduler is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations, and more. You can automate everything, including retries in case of failure to reduce manual toil and intervention. Cloud Scheduler even acts as a single pane of glass, allowing you to manage all your automation tasks from one place.
Answer B is incorrect: No need to create a VM when you can use Cloud Scheduler.
Answer C is incorrect: This process is a batch job. It is not required to make it a streaming pipeline.
Answer D is incorrect: You cannot schedule a Cloud Functions unless you use Cloud Scheduler.
Source(s):
Cloud Scheduler: https://cloud.google.com/scheduler/
Comments are closed, but trackbacks and pingbacks are open.