CalSTA (Data Hub) – Data Architect/Data Engineer

Delegata; United States; Hybrid; USD 108,000 - 154,000; Full time

Job summary

Delegata is seeking a Data Architect / Data Engineer to join the CalSTA Data Hub project. The role is hybrid from Sacramento, CA, with a $95/hour C2C rate and a focus on building end-to-end data pipelines, governance, and AI-assisted data exploration.

You will design and implement automated pipelines, define architecture standards, and work with governance on data sharing, ownership, and quality. Strong Python/SQL and Databricks experience are required.

Qualifications

• 7+ years in data engineering or closely related role.
• Experience on large-scale, enterprise-wide data engineering or data architecture efforts.
• Strong Python and SQL.
• Experience with Apache Spark or a cloud data platform (Databricks, Snowflake, BigQuery or similar). Databricks strongly preferred.
• Experience pulling data from REST APIs, databases or other source systems.
• Excellent communication skills, clear writing and good documentation habits.

Responsibilities

• Build and maintain automated pipelines that bring data from public portals, APIs and internal department systems into Delta tables.
• Assist the program manager in the platform’s architecture, naming standards, and documentation.
• Work with the data hub governance group to make decisions on architecture, data sharing, data contracts, data ownership and data quality standards.
• Advise the data hub governance group on how sensitive data can be protected, including row-level and column-level access rules.
• Make the data reliable through monitoring, alerts and data quality checks.
• Set up datasets and documentation so data experts and business users can explore the data in plain language with AI tools.
• Write and maintain skill files to guide AI coding assistants for pipelines.
• Help other CalSTA departments build their own pipelines using shared templates.

Skills

Python

SQL

Apache Spark

Databricks

REST APIs

Data governance

Documentation

Communication

Tools

Delta Lake

Unity Catalog

Databricks

Snowflake

BigQuery

Job description

Data Architect / Data Engineer

Location: Hybrid out of Sac, CA

Rate: $95 C2C

Active Client Project

The CalSTA Data Hub is a Databricks platform that brings transportation, safety, infrastructure and climate data from across CalSTA departments into one governed place. Analysts, dashboards, apps and business users rely on it. The first phase covers public state and federal data. The next phase adds internal, non-public data from the departments. We need a hands-on data architect and engineer to help build both phases.

What you will do

• Build and maintain automated pipelines that bring data from public portals, APIs and internal department systems into Delta tables.
• Assists the program manager in the platform’s architecture, naming standards, and documentation.
• Work with the data hub governance group to make decisions on architecture, data sharing, data contracts, data ownership and data quality standards.
• Advise the data hub governance group on how sensitive data can be protected, including row-level and column-level access rules, and keep it separate from public data.
• Make the data reliable through monitoring, alerts and data quality checks.
• Set up datasets and documentation so data experts and business users can explore the data in plain language with AI tools.
• Write and maintain skill files (instruction files that guide AI coding assistants) so pipelines are built consistently.
• Help other CalSTA departments build their own pipelines using shared templates.

Required

• 7+ years in data engineering or a closely related role.
• Experience on large-scale, enterprise-wide data engineering or data architecture efforts.
• Strong Python and SQL.
• Experience with Apache Spark or a cloud data platform (Databricks, Snowflake, BigQuery or similar. Databricks is strongly preferred).
• Experience pulling data from REST APIs, databases or other source systems.
• Excellent communication skills, clear writing and good documentation habits.

Preferred

• Hands-on Databricks experience (Delta Lake, Unity Catalog, Jobs).
• Experience governing sensitive or confidential data, such as access controls, data masking and privacy requirements.
• Experience working within a data governance program, such as data contracts, data stewardship and cross-agency data sharing agreements.
• Experience writing skill files, prompts or instructions for AI coding assistants.
• Experience with GIS or location-based data.