Remote Databricks Architect: Spark & Delta Lake

Vidha It Solutions Llc; Polska; Hybrid; PLN 469,000 - 626,000; Full time

Job summary

Vidha IT Solutions LLC is seeking a talented Databricks Architect to join our growing team in a remote capacity. You will design, build, and optimize scalable data pipelines on Databricks, collaborating with data scientists, analysts, and stakeholders to deliver data-driven solutions.

This remote role involves managing Databricks clusters, implementing Delta Lake best practices, and integrating with cloud platforms like AWS, Azure, or GCP to ensure data reliability and governance.

Qualifications

• Proven experience as a Databricks Engineer or Data Engineer with hands-on Databricks platform expertise.
• Strong proficiency in Apache Spark, PySpark, and SQL.
• Experience with Delta Lake, Lakehouse architecture, and data modeling.
• Familiarity with cloud platforms (AWS, Azure, or GCP) and cloud-native data services.
• Experience building and managing ETL/ELT pipelines at scale.
• Knowledge of Python, Scala, or other relevant programming languages.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
• Build and optimize ETL/ELT workflows to support data ingestion, transformation, and delivery.
• Collaborate with data scientists, analysts, and business stakeholders to deliver data-driven solutions.
• Manage and monitor Databricks clusters, notebooks, and job scheduling for performance and cost efficiency.
• Implement Delta Lake architecture and best practices for data reliability and governance.
• Integrate Databricks with cloud platforms such as AWS, Azure, or GCP and enterprise data sources.
• Troubleshoot and resolve data pipeline issues, ensuring high availability and data quality.

Skills

Databricks

Data Engineer

Apache Spark

PySpark

SQL

Delta Lake

Lakehouse

Cloud platforms

ETL/ELT pipelines

Python

Scala

Tools

Databricks Platform

Job description

Vidha IT Solutions LLC is seeking a talented Databricks Architect to join our growing team in a remote capacity. You will design, build, and optimize scalable data pipelines on Databricks, collaborating with data scientists, analysts, and stakeholders to deliver data-driven solutions.

This remote role involves managing Databricks clusters, implementing Delta Lake best practices, and integrating with cloud platforms like AWS, Azure, or GCP to ensure data reliability and governance.