Senior Databricks Engineer

Concentrix; Iowa; Hybrid; USD 140,000 - 180,000; Full time

Job summary

Concentrix is seeking a Senior Databricks Engineer for the United States with remote/work at home flexibility. You will own end-to-end data lifecycle, from analysis and design through implementation and orchestration, using Databricks, Python, PySpark, and SQL to deliver scalable data solutions across cloud platforms.

You will lead design standards, guide other data engineers, and build production-ready pipelines that support analytics, reporting, and AI/ML use cases.

Qualifications

• Experience building data pipelines using Databricks Lakehouse architecture.
• Strong SQL and Python programming skills.
• Experience with batch and streaming data processing.
• Ability to design scalable data models and governance practices.

Responsibilities

• Design, develop, and maintain robust batch and streaming data pipelines for extracting, transforming, and loading data.
• Automate ingestion with Auto Loader, Lakeflow Connect, and Azure services like Data Factory and Event Hubs.
• Build pipelines with Lakeflow Spark Declarative Pipelines and ensure data quality expectations.

Skills

Databricks

Python

PySpark

SQL

Data pipelines

Cloud platforms

Job description

Senior Databricks Engineer - United States | Remote / Work at Home

Position Summary The Senior Data Engineer - Databricks will own the end-to-end data lifecycle, from analysis and solution design through implementation, orchestration, governance, and optimization. This role will build modern data solutions on the Databricks Data Intelligence Platform across Microsoft Azure, AWS, or GCP, using Databricks, Python, PySpark, and SQL to deliver scalable ingestion, transformation, modeling, and pipeline orchestration. As a senior engineer, you will lead solution design, establish engineering standards, and guide other data engineers in building reliable, governed, and production-ready data pipelines that support analytics, reporting, and AI/ML use cases.

Responsibilities

• Design, develop, and maintain robust batch and streaming data pipelines for extracting, transforming, and loading data from multiple sources into the Databricks lakehouse on Azure.
• Automate and streamline data ingestion using Auto Loader, Lakeflow Connect, and Azure services such as Azure Data Factory and Event Hubs.
• Build and manage pipelines with Lakeflow Spark Declarative Pipelines, formerly Delta Live Tables, including data quality expectations