Remote Databricks Engineer — Scale Data Pipelines

Vidha It Solutions Llc; United States; Remote; USD 120,000 - 170,000; Full time

Job summary

Vidha IT Solutions LLC is seeking a Databricks Engineer to design and maintain scalable data pipelines using Databricks and Apache Spark. This remote role requires hands-on experience with Delta Lake, Lakehouse architectures, and cloud platforms.

You will collaborate with data scientists and analysts, optimize ETL/ELT workflows, manage Databricks clusters, and ensure data reliability. W2 employment only; visa is not restricted.

Qualifications

• Hands-on Databricks platform expertise and data engineering experience.
• Strong Spark, PySpark, and SQL skills.
• Experience with Delta Lake and Lakehouse architectures.
• Cloud experience on AWS/Azure/GCP and cloud-native data services.
• Experience building and managing ETL/ELT pipelines at scale.
• Proficient in Python or Scala.
• Strong problem-solving and collaborative skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark
• Build and optimize ETL/ELT workflows to support data ingestion, transformation, and delivery
• Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions
• Manage and monitor Databricks clusters, notebooks, and job scheduling for performance and cost efficiency
• Implement Delta Lake architecture and best practices for data reliability and governance
• Integrate Databricks with cloud platforms such as AWS, Azure, or GCP and enterprise data sources
• Troubleshoot and resolve data pipeline issues, ensuring high availability and data quality

Skills

Databricks

Apache Spark

PySpark

SQL

Data modeling

Problem solving

Team collaboration

Tools

Delta Lake

Lakehouse architecture

Cloud services (AWS/Azure/GCP)

Job description

Vidha IT Solutions LLC is seeking a Databricks Engineer to design and maintain scalable data pipelines using Databricks and Apache Spark. This remote role requires hands-on experience with Delta Lake, Lakehouse architectures, and cloud platforms.

You will collaborate with data scientists and analysts, optimize ETL/ELT workflows, manage Databricks clusters, and ensure data reliability. W2 employment only; visa is not restricted.