Senior Data Engineer - Databricks

DATAECONOMY Inc; Raleigh (NC); On-site; USD 120,000 - 190,000; Full time

Job summary

DATAECONOMY Inc. in Raleigh, NC is seeking a senior Data Engineer to build scalable ETL/ELT pipelines using Databricks (PySpark, Delta Lake) and ingest data across Bronze, Silver, and Gold layers.

You will develop data quality rules, optimize transformations, and orchestrate jobs with CI/CD in AWS/Azure/GCP. Candidates should have 7–10+ years in data engineering and deep knowledge of medallion architecture and Unity Catalog.

Qualifications

• Strong Python (PySpark) and SQL programming.
• Databricks experience including Spark, Delta Lake, and Workflows.
• Cloud experience with AWS, Azure, or GCP.
• 7–10+ years in data engineering.
• Knowledge of medallion architecture and Unity Catalog within Databricks.

Responsibilities

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables).
• Ingest structured, semi-structured, and streaming data across Bronze, Silver, and Gold layers.
• Develop optimized transformations, data quality rules, and reusable components.
• Implement best practices for job orchestration, monitoring, alerting, and automation.
• Apply strong SQL and performance-tuning skills; work with cloud environments (AWS/Azure/GCP).
• Leverage lakehouse concepts and tools (Lakeflow/Genie/Agent Bricks) to support use cases.

Skills

Python

SQL

Databricks

Cloud platforms

Medallion architecture

CI/CD

Tools

Airflow

dbt

Git

Unity Catalog

Job description

Raleigh, United States | Posted on 09/10/2026

DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.

We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.

Rutherford, NJ/ Jersey City, NJ

Full-time

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
• Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
• Develop optimized transformations, data quality rules, and reusable framework components.
• Implement best practices for job orchestration, monitoring, alerting, and automation.
• Strong SQL programming and performance tuning skills.
• Experience with cloud environments (AWS/Azure/GCP).
• Experience with modern data lakehouse concepts and distributed systems.
• Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
• Should be able to explain these concepts using real project examples and architecture decisions.
• Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Requirements

• Strong Python (PySpark) and SQL programming
• Databricks — Spark, Delta Lake, Workflows, Unity Catalog
• CI/CD, Git, job orchestration
• Cloud experience — AWS, Azure, or GCP
• 7–10+ years in data engineering
• Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Nice-to-Have Skills

• Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases
• Data governance,metadata management, Unity Catalog advanced features
• Airflow, dbt, or similar orchestration tools
• Data security, compliance, and access models
• Cost optimization and performance tuning in cloud environments
• Corporate/enterprise data warehousing background