Remote Databricks Engineer - Lakehouse & PySpark

CNX; United States; Remote; USD 92,000 - 120,000; Full time

Job summary

Concentrix is seeking a Databricks Engineer to build, maintain, and optimize data pipelines on the Databricks Data Intelligence Platform running on Azure, AWS, or GCP. You will work with Python, PySpark, and SQL to ingest, transform, model, and orchestrate data pipelines, collaborating with cross-functional teams to meet business goals and performance requirements.

The role requires 6+ years in data engineering with 3+ years on Databricks, and a strong foundation in Python and SQL.

Qualifications

• 6+ years of professional experience in data engineering, data & analytics or data warehousing.
• 3+ years of hands-on experience working with Databricks in a data engineering capacity.
• 3+ years of experience with Python, including manipulating large datasets using PySpark DataFrames, Spark SQL, and/or the pandas API on Spark.
• 2+ years of data engineering experience on Microsoft Azure (e.g., ADLS Gen2, Azure Data Factory, Key Vault).

Responsibilities

• Build and maintain batch and streaming data pipelines for ETL/ELT data into the Databricks lakehouse on Azure.
• Implement data ingestion using Auto Loader, Lakeflow Connect, and Azure Data Factory.
• Develop pipelines with Lakeflow Spark Declarative Pipelines (formerly Delta Live Tables), including data quality checks.
• Perform data manipulation and transformation using PySpark and Spark SQL for analytics readiness.
• Configure, schedule, and monitor data workflows with Lakeflow Jobs.

Skills

Python

SQL

Databricks

Azure Data Factory

Git & CI/CD

Education

Bachelor's degree

Tools

Databricks

Azure Data Factory

Lakeflow

Unity Catalog

Job description

Concentrix is seeking a Databricks Engineer to build, maintain, and optimize data pipelines on the Databricks Data Intelligence Platform running on Azure, AWS, or GCP. You will work with Python, PySpark, and SQL to ingest, transform, model, and orchestrate data pipelines, collaborating with cross-functional teams to meet business goals and performance requirements.

The role requires 6+ years in data engineering with 3+ years on Databricks, and a strong foundation in Python and SQL.