Databricks Architect

Resource Informatics Group; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

A data solutions company is seeking a Databricks Architect to design and develop scalable data solutions. The role involves optimizing ETL/ELT pipelines and collaborating with stakeholders. The ideal candidate should be experienced in Databricks and data governance, and possess coding skills in Python, SQL, or Scala. This is a remote position with a duration of 12+ months.

Qualifications

• Experience with designing scalable data solutions using Databricks.
• Proficient in developing ETL/ELT pipelines.
• Ability to collaborate with engineering and business teams.

Responsibilities

• Design and architect scalable data solutions using the Databricks Lakehouse Platform.
• Develop and optimize ETL/ELT pipelines using Apache Spark and Delta Lake.
• Implement data governance and quality practices.

Skills

Databricks Lakehouse Platform

Apache Spark

Delta Lake

Python

SQL

Scala

CI/CD Pipelines

Terraform

Job description

Hi,

The following requirement is open with our client.

Job Title : Databricks Architect

Location : Remote

Duration : 12+ Months

Description:

• Design and architect scalable data solutions using the Databricks Lakehouse Platform.
• Develop and optimize ETL/ELT pipelines using Apache Spark and Delta Lake.
• Collaborate with data engineers, analysts, and business stakeholders to gather requirements.
• Implement data governance, quality, and security best practices using Unity Catalog.
• Integrate Databricks with various data sources across cloud and on-premise environments.
• Optimize performance and cost of data pipelines and jobs running on Databricks.
• Use Python, SQL, and/or Scala to build robust data engineering solutions.
• Support deployment using CI/CD pipelines and infrastructure-as-code tools like Terraform.
• Stay updated on Databricks features and recommend improvements or new capabilities.
• Provide technical leadership, guidance, and mentoring to engineering teams.