Local Data Stack — Setup Guide

Two environments,
one machine.

This guide sets up a complete local data analysis environment on your Mac — independent of any cloud service — and connects it to the shared team project on Google BigQuery. No prior experience with any of these tools is assumed.

World 1 — Runs on your Mac
DuckDB in VS Code Write and run SQL directly on local files
Metabase via Docker Visual dashboards, no cloud required
⟷
World 2 — Shared team project
BigQuery (Browser) Query shared data in Google Cloud
BigQuery (Terminal + Notebook) Programmatic access from your machine

World 1 is completely autonomous — it works without internet, without a Google account, and without any cost. World 2 connects you to the shared dataset your team already has running in Google Cloud.


Prerequisites

The Foundation

Before installing anything, you need two tools that everything else depends on: Homebrew and a basic understanding of the Terminal.

The Terminal — the black window that does everything

The Terminal is just a text-based way to talk to your Mac. Instead of clicking, you type a command and press Enter. That's it. Here are the only five commands you need to know:

Command What it does Example
pwd Shows which folder you are currently in /Users/iliya/projects
ls Lists all files in the current folder Shows your files like Finder would
cd Moves you into a different folder cd Developer/go_explore
brew Installs software on your Mac brew install python
python3 Runs a Python script or opens Python python3 my_script.py

Installing Homebrew

Homebrew is a package manager — think of it as an App Store for the Terminal. Most developer tools are installed through it. Open the Terminal and paste this:

Terminal
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Follow the prompts. It will ask for your Mac password. When it's done, verify it works:

Terminal
brew --version

You should see something like Homebrew 4.x.x. That means it's ready.


World 1 — Local
2.1 — DuckDB in VS Code

Running SQL directly on your files

DuckDB is a database engine that runs directly inside VS Code — no server, no setup, no running process in the background. You point it at a file, and you can query it with SQL immediately. It is extremely fast and handles files up to several gigabytes without issues.

⚠️ Critical — Install exactly this extension

Search for DuckDB in the VS Code Extensions panel. Install the one by Charlie Jonas only. Do not install "DuckDB SQL Tools", "SQLTools", or any other SQL extension — they conflict with each other and will cause errors that are very difficult to diagnose. If you have any of those already installed, uninstall them first.

Installing the extension

  1. Open the Extensions panel in VS Code

    Press Cmd + Shift + X or click the Extensions icon in the left sidebar.

  2. Search for "DuckDB" and install the Charlie Jonas version

    It shows 12,000+ installs and version 0.0.27. Click Install.

  3. Reload VS Code when prompted

    A small reload button will appear after installation. Click it.

Creating a database and loading data

A DuckDB database is just a single file on your Mac with the extension .db. You create it by connecting to a filename that doesn't exist yet — DuckDB creates it automatically.

  1. Open your project folder in VS Code

    Use File → Open Folder and navigate to your project directory.

  2. Select your database

    Press Cmd + Shift + P, type DuckDB: Select Database, then choose Attach Existing and navigate to your .db file. If you don't have one yet, choose Create New Database.

  3. Load a CSV file into the database

    Create a new file called queries.sql and paste this, adjusting the file path and table name:

queries.sql
-- Load a CSV file into a new table called "retailers"
CREATE TABLE retailers AS
SELECT * FROM read_csv_auto('/absolute/path/to/your/file.csv');

-- Check what was loaded
SELECT * FROM retailers LIMIT 5;

To run the query, place your cursor anywhere inside it and press Cmd + Shift + P → DuckDB: Run Statement at Cursor. A results panel will open with your data.

Optional — Shift + Enter shortcut

If you prefer to run queries with Shift + Enter (like MySQL Workbench), open Cmd + Shift + P → Open Keyboard Shortcuts (JSON) and add this entry:

keybindings.json
{
    "key": "shift+enter",
    "command": "duckdb.runStatementAtCursor",
    "when": "editorTextFocus && editorLangId == 'sql'"
}
💡 Column names with spaces or special characters

SQL is strict about column names. If a column name contains a space or special character — for example Unit sale price — you must wrap it in double quotes: "Unit sale price". Text values (the actual data) always go in single quotes: 'Germany'. Mixing these up is one of the most common sources of confusing errors.


World 1 — Local
2.2 — Metabase via Docker

Visual dashboards on your machine

Metabase is a dashboarding tool — it turns your data into charts and interactive reports without writing any code. It runs in a web browser, but the server runs locally on your Mac inside Docker.

What is Docker?

Think of Docker as a way to run a fully self-contained mini-computer inside your Mac. That mini-computer has its own operating system, its own programs, and its own file system. Metabase runs inside one of these containers, completely isolated from the rest of your Mac. You start it with one command and stop it with one command.

Make sure Docker Desktop is running before proceeding — you should see the Docker icon in your menu bar.

Starting Metabase

⚠️ Mount the folder, not the file

DuckDB creates temporary helper files (.db.wal) in the same folder as your database when it writes data. If Docker only knows about the single .db file, it cannot create these helpers and will crash. The solution is to mount the entire project folder into the container, not just the database file.

Replace /Users/yourname/your-project-folder with the actual path to your project folder on your Mac:

Terminal
docker run -d \
  -p 3000:3000 \
  -v /Users/yourname/your-project-folder:/data \
  --name metabase \
  metabase/metabase

This command tells Docker to start Metabase, make it available on port 3000, and map your local project folder to /data inside the container.

  1. Open Metabase in your browser

    Navigate to http://localhost:3000. The first start takes about 60 seconds.

  2. Create your admin account

    Fill in your name, email, and a password. This is just for local access — it doesn't connect to anything external.

  3. Connect your DuckDB database

    When asked to add a database, select DuckDB. For the database path, enter /data/go_explore.db — this is the path inside the container, not on your Mac.

  4. Start exploring

    Metabase will scan your tables. You can now build charts and dashboards by clicking through the interface — no SQL required, though you can use it if you prefer.

Stopping and restarting Metabase

To stop: docker stop metabase. To start again later: docker start metabase. You only need the long docker run command once — after that, start and stop are all you need.


World 2 — Team
3.1 — BigQuery in the Browser

Accessing the shared team project

BigQuery is Google's cloud database service. Your team's data lives there, and multiple people can query it simultaneously from anywhere. The simplest way to access it is directly in the browser.

⚠️ Always use the direct link

Do not navigate to BigQuery through the Google Cloud homepage — you will land in a generic console that may show a different or empty project. Always use the direct link with the project ID appended:

Browser URL
https://console.cloud.google.com/bigquery?project=mystical-proj-500509

Bookmark this URL. Once you open it, you will see the project's datasets and tables in the left panel. Click on any table to preview its contents, or click Query to open a SQL editor.


World 2 — Team
3.2 — BigQuery via Terminal

Connecting from your Mac directly

Instead of using the browser, you can query BigQuery directly from your Mac — through the Terminal, through a Python script, or through a Jupyter Notebook. This requires installing Google's command-line tools and authenticating once.

Step 1 — Install the Google Cloud SDK

The Google Cloud SDK is a collection of command-line tools for working with Google Cloud services. Install it via Homebrew:

Terminal
brew install google-cloud-sdk

Step 2 — Log in

This command opens a browser window where you log in with your Google account. After logging in, close the browser — your Terminal is now authenticated.

Terminal
gcloud auth application-default login

Step 3 — Set the project

Tell gcloud which project you want to work with:

Terminal
gcloud config set project mystical-proj-500509

Step 4 — Install the Python package

This package lets Python talk to BigQuery:

Terminal
pip3 install google-cloud-bigquery --break-system-packages

Step 5 — Run your first query

Verify everything works with this test query:

Terminal
python3 -c "
from google.cloud import bigquery
client = bigquery.Client(project='mystical-proj-500509')
result = client.query("""
    SELECT Country, COUNT(*) AS stores
    FROM go_explore_project.retailers
    GROUP BY Country
    ORDER BY stores DESC
    LIMIT 5
""")
for row in result:
    print(row)
"

You should see the top 5 countries by store count printed in your Terminal. If this works, your connection is fully set up.


World 2 — Team
3.3 — BigQuery via Jupyter Notebook

Interactive analysis in a Notebook

A Jupyter Notebook lets you write code in small blocks called cells and run each one individually. Results appear directly below the cell — tables, charts, numbers. It is the most comfortable way to explore data interactively.

Installing the required packages

Terminal
pip3 install jupyter pandas --break-system-packages

Starting a Notebook in VS Code

Create a new file with the extension .ipynb in VS Code. It will open as a Notebook automatically. Each cell can contain Python code. Press Shift + Enter to run a cell and move to the next one.

Querying BigQuery from a Notebook

Paste this into the first cell and run it. After that, every cell can contain a pure SQL query:

Jupyter Notebook — Cell 1 (run once)
from google.cloud import bigquery

client = bigquery.Client(project='mystical-proj-500509')

def bq(sql):
    """Run a BigQuery SQL query and return a table."""
    return client.query(sql).to_dataframe()
Jupyter Notebook — Any subsequent cell
bq("""
    SELECT
        r.Country,
        SUM(s.Quantity * s.Unit_sale_price) AS Total_Revenue
    FROM go_explore_project.daily_sales AS s
    JOIN go_explore_project.retailers AS r
        ON s.Retailer_code = r.Retailer_code
    GROUP BY r.Country
    ORDER BY Total_Revenue DESC
""")

The result appears as a formatted table directly below the cell. You are now running SQL against the shared team dataset from your own machine, with results rendered locally.

💡 What just happened

Your Mac sent the SQL query to Google's servers via the authenticated connection you set up with gcloud. Google ran the query on their infrastructure and sent the result back to your machine. The data processing happened in the cloud — the result lives locally in your Notebook.