Senior Data Engineer - Databricks

DATAECONOMY; Raleigh (NC); On-site; USD 120,000 - 160,000; Full time

Job summary

DATAECONOMY in Raleigh, United States, seeks a Senior Data Engineer to build scalable ETL/ELT pipelines using Databricks, PySpark, and modern lakehouse architectures. You will ingest data across Bronze/Silver/Gold layers, design transformations, and enforce data quality rules.

The role requires 7-10+ years in data engineering, strong Python/SQL skills, and cloud experience (AWS/Azure/GCP). You will collaborate on distributed systems and complex data projects within a fast-growing analytics

Qualifications

• Strong Python (PySpark) and SQL programming.
• Databricks — Spark, Delta Lake, Workflows, Unity Catalog.
• CI/CD, Git, job orchestration.
• Cloud experience — AWS, Azure, or GCP.
• 7-10+ years in data engineering.

Responsibilities

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
• Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
• Develop optimized transformations, data quality rules, and reusable framework components.
• Implement best practices for job orchestration, monitoring, alerting, and automation.
• Strong SQL programming and performance tuning skills.
• Experience with cloud environments (AWS/Azure/GCP).
• Experience with modern data lakehouse concepts and distributed systems.
• Explain concepts with real project examples and architecture decisions.

Skills

Python (PySpark)

SQL

CI/CD

Git

Job orchestration

Cloud experience

Data engineering

Tools

Databricks

Spark

Delta Lake

Workflows

Unity Catalog

Airflow

dbt

Job description

Raleigh, United States

DATAECONOMY is one of the fastest-growing Data & Analytics company with global presence. We are well-differentiated and are known for our Thought leadership, out-of-the-box products, cutting-edge solutions, accelerators, innovative use cases, and cost-effective service offerings.

We offer products and solutions in Cloud, Data Engineering, Data Governance, AI/ML, DevOps and Blockchain to large corporates across the globe. Strategic Partners with AWS, Collibra, cloudera, neo4j, DataRobot, Global IDs, tableau, MuleSoft and Talend.

Full-time

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
  


• Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
  


• Develop optimized transformations, data quality rules, and reusable framework components.
  


• Implement best practices for job orchestration, monitoring, alerting, and automation.
  


• Strong SQL programming and performance tuning skills.
  


• Experience with cloud environments (AWS/Azure/GCP).
  


• Experience with modern data lakehouse concepts and distributed systems.
  


• Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
  


• Should be able to explain these concepts using real project examples and architecture decisions.
  

Requirements

• Strong Python (PySpark) and SQL programming
  


• Databricks — Spark, Delta Lake, Workflows, Unity Catalog
  


• CI/CD, Git, job orchestration
  


• Cloud experience — AWS, Azure, or GCP
  


• 7-10+ years in data engineering
  

Nice-to-Have Skills

• Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases
  


• Data governance,metadata management, Unity Catalog advanced features
  


• Airflow, dbt, or similar orchestration tools
  


• Data security, compliance, and access models
  


• Cost optimization and performance tuning in cloud environments
  


• Corporate/enterprise data warehousing background