Data Architect and Engineer

eStaff LLC; Sacramento (CA); Hybrid; USD 140,000 - 180,000; Full time

Job summary

Estaff LLC in Sacramento, CA is seeking a hands-on Data Architect and Engineer to centralize data in a governed, single location for a hybrid project. Analysts, dashboards, apps, and business users will rely on this data as the first phase covers public state and federal data.

The role involves building pipelines from public portals, REST APIs, and internal systems, setting architecture standards, and enabling data discovery with AI tools while ensuring data quality and security.

Qualifications

• 7+ years in data engineering or related role.
• Experience in enterprise-scale data engineering or architecture.
• Strong Python and SQL.
• Experience with Spark or cloud data platforms (Databricks, Snowflake, BigQuery).
• Experience pulling data from REST APIs, databases, or other sources.
• Excellent writing and documentation habits.
• Hands-on Databricks (Delta Lake, Unity Catalog, Jobs).
• Experience governing sensitive data (access controls, masking, privacy).
• Data governance program experience (data contracts, data stewardship, cross-agency sharing).
• Experience writing skill files or prompts for AI coding assistants.
• Experience with GIS or location-based data.

Responsibilities

• Build and maintain automated pipelines from public portals, APIs, and internal systems into Delta tables.
• Assist the program manager with platform architecture, naming standards, and documentation.
• Collaborate with data hub governance on architecture, data sharing, data contracts, ownership and quality.
• Advise on protecting sensitive data with row/column access rules and separate from public data.
• Ensure data reliability with monitoring, alerts and quality checks.
• Set up datasets and documentation for data exploration by experts and business users using AI tools.
• Write and maintain skill files to guide AI coding assistants for consistent pipelines.
• Help other departments build pipelines using shared templates.

Skills

Python

SQL

Databricks

Data governance

REST APIs

Documentation

GIS data

Access controls

Data contracts

Spark

AI prompts

Tools

Databricks

Snowflake

BigQuery

Job description

We are seeking a hands-on Data Architect and Engineer in Sacramento, California, to centralize data in a single, governed location for a hybrid project. Analysts, dashboards, apps, and business users will rely on this data. The first phase covers public state and federal data. The next phase adds internal, non-public data from the departments.

What you will do:

• Build and maintain automated pipelines that bring data from public portals, APIs, and internal department systems into Delta tables.
• Assist the program manager in the platform's architecture, naming standards, and documentation.
• Work with the data hub governance group to make decisions on architecture, data sharing, data contracts, data ownership and data quality standards.
• Advise the data hub governance group on how sensitive data can be protected, including row-level and column-level access rules, and keep it separate from public data.
• Make the data reliable through monitoring, alerts and data quality checks.
• Set up datasets and documentation so data experts and business users can explore the data in plain language with AI tools.
• Write and maintain skill files (instruction files that guide AI coding assistants) so pipelines are built consistently.
• Help other departments build their own pipelines using shared templates.

Required

• 7+ years in data engineering or a closely related role.
• Experience with large-scale, enterprise-wide data engineering or data architecture efforts.
• Strong Python and SQL.
• Experience with Apache Spark or a cloud data platform (Databricks, Snowflake, BigQuery, or similar; Databricks is strongly preferred).
• Experience pulling data from REST APIs, databases or other source systems.
• Excellent communication skills, clear writing and good documentation habits.

Preferred

• Hands-on Databricks experience (Delta Lake, Unity Catalog, Jobs).
• Experience governing sensitive or confidential data, such as access controls, data masking and privacy requirements.
• Experience working within a data governance program, such as data contracts, data stewardship, and cross-agency data sharing agreements.
• Experience writing skill files, prompts, or instructions for AI coding assistants.
• Experience with GIS or location-based data.