Mid-Senior Databricks Engineer

Artingdigital; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

Arting Digital Private Limited is seeking a Mid-Senior Databricks Engineer to design and optimize scalable data pipelines. This remote role targets candidates with 3–6 years of data engineering experience and strong skills in SQL, Python, PySpark, Databricks, and Apache Spark.

The position focuses on building ETL workflows with Delta Live Tables, tuning Spark performance, and implementing Delta Lake optimizations.

Qualifications

• 3–6 years of experience in Data Engineering.
• Strong knowledge of SQL, Python, and PySpark.
• Hands-on experience with Databricks and Apache Spark.
• Understanding of Spark optimization, partitioning, and shuffling.

Responsibilities

• Develop and maintain scalable data pipelines using Databricks, Python, and PySpark.
• Build and automate ETL workflows using Delta Live Tables (DLT).
• Optimize Spark jobs through partitioning, shuffling, and performance tuning.
• Implement Delta Lake optimization techniques, including Z-Ordering and VACUUM.
• Manage data orchestration using Airflow or Azure Data Factory.
• Ensure data quality, governance, and pipeline performance.
• Collaborate with cross-functional teams to deliver efficient data engineering solutions.

Skills

SQL

Python

PySpark

Tools

Databricks

Apache Spark

Delta Live Tables

Delta Lake

Airflow

Azure Data Factory

Job description

Arting Digital Private Limited | Full time

Position: Mid-Senior Databricks Engineer

Location: Remote

Experience: 3–6 Years

Employment Type: Full-Time

Required Skills & Qualifications

3–6 years of experience in Data Engineering.

Strong knowledge of SQL, Python, and PySpark.

Hands-on experience with Databricks and Apache Spark.

Understanding of Spark optimization, partitioning, and shuffling.

Experience with Delta Lake, Z-Ordering, and VACUUM.

Knowledge of Apache Airflow or Azure Data Factory.

Understanding of data governance and data quality.

Key Responsibilities

Develop and maintain scalable data pipelines using Databricks, Python, and PySpark.

Build and automate ETL workflows using Delta Live Tables (DLT).

Optimize Spark jobs through partitioning, shuffling, and performance tuning.

Implement Delta Lake optimization techniques, including Z-Ordering and VACUUM.

Manage data orchestration using Airflow or Azure Data Factory.

Ensure data quality, governance, and pipeline performance.

Collaborate with cross-functional teams to deliver efficient data engineering solutions.