Senior Databricks Engineer – Remote (USA)

Concentrix; United States; Remote; USD 106,000 - 140,000; Full time

Job summary

Concentrix is seeking a Senior Databricks Engineer to design and deliver end-to-end data solutions on the Databricks Lakehouse across Azure, AWS, or GCP. You will lead pipeline design, governance, and performance improvements while mentoring junior engineers.

The role requires strong Python, SQL, Databricks, and Azure data services experience, with EST remote work in the United States and collaboration with cross-functional teams to meet business goals.

Qualifications

• 10+ years of professional experience in data engineering, data & analytics, or data warehousing.
• 5+ years of hands-on experience working with Databricks in a data engineering capacity, including production workloads.
• 5+ years of experience with Python, including manipulating large datasets using PySpark DataFrames, Spark SQL, and/or the pandas API on Spark.
• 3+ years of data engineering experience on Microsoft Azure (e.g., ADLS Gen2, Azure Data Factory, Key Vault, Microsoft Entra ID).

Responsibilities

• Design, develop, and maintain robust batch and streaming data pipelines for ETL/ELT data ingestion into the Databricks lakehouse on Azure.
• Automate and streamline data ingestion using Auto Loader and Azure services.
• Build and manage pipelines with Lakeflow Declarative Pipelines and data quality checks.
• Perform data transformations with PySpark and Spark SQL for analytics and AI/ML use cases.
• Design and implement orchestration and scheduling of data workflows for reliability.
• Implement scalable data models using medallion architecture and dimensional modeling.

Skills

Databricks

Python

SQL

Azure

Education

Bachelor's degree

Tools

Azure Data Factory

Unity Catalog

Job description

Concentrix is seeking a Senior Databricks Engineer to design and deliver end-to-end data solutions on the Databricks Lakehouse across Azure, AWS, or GCP. You will lead pipeline design, governance, and performance improvements while mentoring junior engineers.

The role requires strong Python, SQL, Databricks, and Azure data services experience, with EST remote work in the United States and collaboration with cross-functional teams to meet business goals.