Senior Data Engineer — Databricks Lakehouse Migration

Sigma Software; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Sigma Software is seeking a Senior Data Engineer to lead the migration of a large-scale analytics platform from BigQuery to Databricks. You will design scalable Lakehouse architectures and develop Spark-based data processing pipelines for retail analytics.

Ideal candidates have 5+ years in data engineering, strong Python/SQL skills, and hands-on Databricks experience, with cloud platform knowledge (GCP/AWS/Azure) and data governance focus.

Qualifications

• 5+ years of experience as a Data Engineer.
• Strong Python and SQL skills.
• Hands-on Databricks experience.
• Spark/PySpark expertise and data platform design.
• Experience with cloud data platforms and ETL/ELT pipelines.

Responsibilities

• Migrate large-scale analytics platform from BigQuery to Databricks.
• Design scalable Lakehouse architectures with Delta Lake.
• Build and optimize data pipelines for retail datasets.
• Implement governance and security best practices.
• Collaborate with international teams on architecture decisions.
• Contribute to platform reliability and maintainability.

Skills

Python

SQL

Databricks

Apache Spark

Airflow

GCP

AWS

Azure

Data modeling

BigQuery migration

Tools

Delta Lake

CI/CD

Unity Catalog

Job description

Sigma Software is seeking a Senior Data Engineer to lead the migration of a large-scale analytics platform from BigQuery to Databricks. You will design scalable Lakehouse architectures and develop Spark-based data processing pipelines for retail analytics.

Ideal candidates have 5+ years in data engineering, strong Python/SQL skills, and hands-on Databricks experience, with cloud platform knowledge (GCP/AWS/Azure) and data governance focus.