Senior Databricks Data Engineer - Lakehouse & Governance

Siri InfoSolutions Inc; Irvine (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Siri InfoSolutions Inc in Irvine, CA is seeking a hands-on Senior Data Engineer to design, build, and operate enterprise-scale data solutions on the Databricks Lakehouse Platform. You will develop reliable batch and streaming pipelines, modernize legacy ETL workloads, and deliver governed data products for analytics and risk management.

The ideal candidate has deep experience with Databricks, Apache Spark, PySpark, Python, SQL, dbt, Apache Airflow, Delta Lake, Unity Catalog, and cloud data

Qualifications

• Typically 10 years of data engineering, warehousing, or distributed data-processing experience.
• Hands-on with Databricks, Spark, PySpark, Delta Lake, Python, and advanced SQL.
• Experience building production-grade ETL/ELT pipelines for large datasets.
• Experience with dbt Core/Cloud, including models, macros, tests, and incremental processing.
• Experience with Apache Airflow, Databricks Workflows, or similar orchestration tools.
• Experience with Unity Catalog, data lineage, RBAC, and data-governance controls.
• Experience with cloud data services on AWS, Azure, or Google Cloud.
• Working knowledge of Git, CI/CD, automated testing, monitoring, and production support.

Responsibilities

• Design, build, test, deploy, and maintain scalable ETL/ELT pipelines using Databricks and PySpark.
• Develop reusable ingestion and transformation frameworks for structured and streaming data.
• Implement batch, incremental, and streaming processing patterns with Delta Lake.
• Create and maintain dbt models, tests, and documentation with incremental processing.
• Develop Airflow DAGs and Databricks Workflows with dependency management and alerts.
• Integrate data from APIs, databases, files, event streams, and cloud services.
• Produce technical designs, lineage documentation, and deployment/runbook instructions.
• Collaborate with architects, product owners, analysts, and governance teams; mentor junior engineers.

Skills

Databricks

Apache Spark

PySpark

Delta Lake

Python

SQL

dbt Core/Cloud

Apache Airflow

Unity Catalog

Cloud services (AWS/Azure/GC)

Git / CI_CD

Data governance

Tools

Astronomer

Databricks Workflows

Terraform

Databricks Asset Bundles

Job description

Siri InfoSolutions Inc in Irvine, CA is seeking a hands-on Senior Data Engineer to design, build, and operate enterprise-scale data solutions on the Databricks Lakehouse Platform. You will develop reliable batch and streaming pipelines, modernize legacy ETL workloads, and deliver governed data products for analytics and risk management.

The ideal candidate has deep experience with Databricks, Apache Spark, PySpark, Python, SQL, dbt, Apache Airflow, Delta Lake, Unity Catalog, and cloud data