Senior Databricks Data Engineer — Lakehouse & ETL Expert

DATAECONOMY Inc; Raleigh (NC); On-site; USD 120,000 - 190,000; Full time

Job summary

DATAECONOMY Inc. in Raleigh, NC is seeking a senior Data Engineer to build scalable ETL/ELT pipelines using Databricks (PySpark, Delta Lake) and ingest data across Bronze, Silver, and Gold layers.

You will develop data quality rules, optimize transformations, and orchestrate jobs with CI/CD in AWS/Azure/GCP. Candidates should have 7–10+ years in data engineering and deep knowledge of medallion architecture and Unity Catalog.

Qualifications

• Strong Python (PySpark) and SQL programming.
• Databricks experience including Spark, Delta Lake, and Workflows.
• Cloud experience with AWS, Azure, or GCP.
• 7–10+ years in data engineering.
• Knowledge of medallion architecture and Unity Catalog within Databricks.

Responsibilities

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables).
• Ingest structured, semi-structured, and streaming data across Bronze, Silver, and Gold layers.
• Develop optimized transformations, data quality rules, and reusable components.
• Implement best practices for job orchestration, monitoring, alerting, and automation.
• Apply strong SQL and performance-tuning skills; work with cloud environments (AWS/Azure/GCP).
• Leverage lakehouse concepts and tools (Lakeflow/Genie/Agent Bricks) to support use cases.

Skills

Python

SQL

Databricks

Cloud platforms

Medallion architecture

CI/CD

Tools

Airflow

dbt

Git

Unity Catalog

Job description

DATAECONOMY Inc. in Raleigh, NC is seeking a senior Data Engineer to build scalable ETL/ELT pipelines using Databricks (PySpark, Delta Lake) and ingest data across Bronze, Silver, and Gold layers.

You will develop data quality rules, optimize transformations, and orchestrate jobs with CI/CD in AWS/Azure/GCP. Candidates should have 7–10+ years in data engineering and deep knowledge of medallion architecture and Unity Catalog.