Practice Test 1 | Google Cloud Certified Professional Data Engineer | Dumps | Mock Test

Question 17

A company uses BigQuery as its main data warehouse. Data stored in Google Storage is being transformed and enriched using a Dataflow pipeline, to be later loaded into BigQuery. More than 80 different datasets exist in BigQuery with each dataset containing between 20-50 tables, all stored in a single project. Data analysts access BigQuery for their reporting tasks, while data scientists are using BigQuery ML (Machine Learning) by creating forecast models. Since BigQuery is used by a wide range of employees, the CTO wants to control the costs of running queries scanning GBs of data from users who frequently trigger such queries.
How can you achieve this?

A. Set project-level quotas on BigQuery by setting a fixed size limit to be used monthly.
B. Set monthly flat-rate pricing for BigQuery.
C. Set user-level custom quotas to all users with access to BigQuery
D. Separate datasets to different projects to benefit from monthly free tier.

Answer

Answer: C

Description:

If you have multiple BigQuery projects and users, you can manage costs by requesting a user-level custom quota that specifies a limit on the amount of query data processed per day.

Creating a custom quota on query data allows you to control costs at the project level or at the user- level.

Project-level custom quotas limit the aggregate usage of all users in that project.
User-level custom quotas are separately applied to each user or service account within a project.

Option A is incorrect: Setting a project-level quota is not the best approach for this scenario because this will not set user limit quotas and when the project reaches the limit set it will disallow all users from running queries. Note that, as stated, all datasets reside in a single cloud project.

Option B is incorrect: Flat-rate can be a possible approach. However, BigQuery does not provide flexible flat-rate pricing and the cheapest is (Monthly flat-rate: $2,000 for 100 slots, Annual flat-rate $1700 for 100 slots), which may not be a desirable option for small-medium businesses.

Ref.: https://cloud.google.com/bigquery/pricing#flat-rate_analysis_pricing

Option D is incorrect: Separating datasets to different projects will lead to more work from data engineers to maintain access among different projects in case users need to join tables from different datasets together. This solution is possible for testing and development projects, as well as small-scale dataset usage, but for this scenario, setting quotas is more efficient.

Source(s):

BigQuery – Creating custom cost controls: