Remote Databricks Engineer: Scalable Data Pipelines

Vidha It Solutions Llc; Oregon; Híbrido; USD 120.000 - 160.000; Tempo integral

Resumo da vaga

Vidha IT Solutions LLC is seeking a Databricks Engineer to design and maintain scalable data pipelines using Databricks and Apache Spark. The role is remote and requires strong knowledge of ETL/ELT processes and data modeling.

You will collaborate with data scientists, analysts, and stakeholders to deliver data-driven solutions, manage Databricks clusters, and implement Delta Lake architecture for reliable data governance across cloud platforms such as AWS, Azure, or GCP.

Qualificações

• Proven experience as a Databricks Engineer or Data Engineer.
• Hands-on Databricks platform expertise.
• Strong proficiency in Apache Spark, PySpark, and SQL.
• Experience with Delta Lake, Lakehouse architecture, and data modeling.
• Experience with cloud platforms (AWS, Azure, or GCP) and cloud-native data services.
• Experience building and managing ETL/ELT pipelines at scale.
• Knowledge of Python, Scala, or other relevant programming languages.
• Strong problem-solving skills and ability to work in a collaborative, fast-paced environment.

Responsabilidades

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark
• Build and optimize ETL/ELT workflows to support data ingestion, transformation, and delivery
• Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions
• Manage and monitor Databricks clusters, notebooks, and job scheduling for performance and cost efficiency
• Implement Delta Lake architecture and best practices for data reliability and governance
• Integrate Databricks with cloud platforms such as AWS, Azure, or GCP and enterprise data sources
• Troubleshoot and resolve data pipeline issues, ensuring high availability and data quality

Conhecimentos

Problem-solving

Team collaboration

Fast-paced environment

Ferramentas

Apache Spark

PySpark

SQL

Delta Lake

Lakehouse architecture

Python

Scala

Descrição da vaga

Vidha IT Solutions LLC is seeking a Databricks Engineer to design and maintain scalable data pipelines using Databricks and Apache Spark. The role is remote and requires strong knowledge of ETL/ELT processes and data modeling.

You will collaborate with data scientists, analysts, and stakeholders to deliver data-driven solutions, manage Databricks clusters, and implement Delta Lake architecture for reliable data governance across cloud platforms such as AWS, Azure, or GCP.