Senior Data Engineer - City Data Platform (Hybrid)

City and County of San Francisco; San Francisco, Northern (CA, KY); Hybrid; USD 154,000 - 193,000; Full time

Job summary

DataSF in San Francisco is seeking a Senior Data Engineer to design, build, and maintain the City's data infrastructure. You will manage the Snowflake data warehouse and develop scalable pipelines to support analytics and operations, in a permanent exempt, full-time role with a hybrid schedule.

The ideal candidate has 3+ years of experience with Snowflake, BigQuery, or Databricks, strong SQL and Python skills, and experience with Terraform and data orchestration tools.

Qualifications

• Hands-on experience with modern cloud data platforms (Snowflake, BigQuery, Databricks).
• Strong SQL development and data modeling skills.
• Proficiency in Python for data processing (pandas, PySpark).
• Familiarity with Terraform and data orchestration tools (Airflow, Dagster).
• Knowledge of data governance, privacy, and security principles.

Responsibilities

• Design, build, and maintain scalable data pipelines and data warehouse solutions.
• Collaborate with data scientists, analysts, and product teams to meet data needs.
• Monitor data quality and platform health, guiding improvements.
• Provide leadership on best practices for data modeling and testing.

Skills

Cross-functional collaboration

Problem-solving

Public-service mindset

Education

Associate degree in CS/CE/IS

Tools

Snowflake

BigQuery

Databricks

dbt

Python

Pandas

PySpark

Terraform

Airflow

Dagster

Kafka

Kinesis

Snowpipe

Job description

DataSF in San Francisco is seeking a Senior Data Engineer to design, build, and maintain the City's data infrastructure. You will manage the Snowflake data warehouse and develop scalable pipelines to support analytics and operations, in a permanent exempt, full-time role with a hybrid schedule.

The ideal candidate has 3+ years of experience with Snowflake, BigQuery, or Databricks, strong SQL and Python skills, and experience with Terraform and data orchestration tools.