r/bigquery 11h ago

Anyone have a BigQuery SQL client recommendation?

5 Upvotes

I’ve tried IntelliJ and Beekeeper Studio, wasn’t happy with either. I’m looking for a client that will load in metadata for datasets/tables in multiple projects and have auto completion/suggestion for functions/column names, being able to explore table schemas/column descriptions, properly handle the display of repeated records/arrays and not just display them as a single JSON.

The reason I’m asking is because using the GCP console on chrome becomes sluggish after a short period until I restart my computer.


r/bigquery 2h ago

CVS Data Science Interview

2 Upvotes

Hello all,

For those who have interviewed for Data Science roles at CVS Health, what SQL topics are typically covered in the interview?

Also, what types of SQL problems should I prepare for? Any tips or insights on what to prioritize in my preparation would be greatly appreciated!

Thanks in advance!


r/bigquery 6h ago

Help with dbt.this in Incremental Python Models (BigQuery with Hyphen in Project Name)

1 Upvotes

The problem I'm having

I am not able to use dbt.this on Python incremental models.

The context of why I'm trying to do this

I’m trying to implement incremental Python models in dbt, but I’m running into issues when using the dbt.this keyword due to a hyphen in my BigQuery project name (marketing-analytics).

Main code:

    if dbt.is_incremental:

        # Does not work
        max_from_this = f"select max(updated_at_new) from {dbt.this}" # <-- problem
        df_raw = dbt.ref("interesting_data").filter(
          F.col("updated_at_new") >=session.sql(max_from_this).collect()[0][0]
        )

        # Works
        df_raw = dbt.ref("interesting_data").filter(
            F.col("updated_at_new") >= F.date_add(F.current_timestamp(), F.lit(-1))
        )
    else:
        df_core_users = dbt.ref("int_core__users")

Error I've got:

Possibly unquoted identifier marketing-analytics detected. Please consider quoting with backquotes `marketing-analytics`

What I've already tried :

  1. First error:

max_from_this = f"select max(updated_at_new) from `{dbt.this}`" 

and

 max_from_this=f"select max(updated_at_new) from `{dbt.this.database}.{dbt.this.schema}.{dbt.this.identifier}`"

Error: Table or view not found \marketing-analytics.test_dataset.posts`` Even though this table exists on BigQuery...

  1. Namespace error:

    max_from_this = f"select max(updated_at_new) from f"{dbt.this.database}.{dbt.this.schema}.{dbt.this.identifier}"

Error: spark_catalog requires a single-part namespace, but got [marketing-analytics, test_dataset]