Databricks Architect

Data Piper; Pleasanton (CA); On-site; USD 120,000 - 150,000; Full time

Job summary

Data Piper is seeking a Resident Solutions Architect with over 10 years of experience in data engineering and consulting. This role focuses on leveraging expertise in Databricks and cloud ecosystems (AWS, Azure, GCP) to deliver data solutions remotely.

Ideal candidates should have extensive experience with distributed computing, MLOps, and CI/CD practices. This is an excellent opportunity for those who thrive in a dynamic environment.

Qualifications

• 10+ years experience in data engineering, data platforms & analytics.
• 10+ years of consulting experience.
• Hands-on experience in development on Databricks.

Responsibilities

• Work on data engineering projects with a focus on analytics.
• Consult on data platform architectures across cloud ecosystems.

Skills

Data Engineering

Cloud platforms (AWS, Azure, GCP)

Databricks

Distributed computing (Spark)

MLOps

CI/CD

Machine Learning

Education

Data Engineering Professional certification

Job description

Job Description

Title: Resident Solutions Architect

Location: Remote

• 10+ years experience in data eng, data platforms & analytics, 10+ years of consulting experience
• Completed Data Engineering Professional certification & required classes
• Hands‑on experience in development on Databricks
• Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one
• Deep experience with distributed computing with Spark with knowledge of Spark runtime internals
• Familiarity with CI/CD for production deployments
• Working knowledge of MLOps
• Current knowledge across the breadth of Databricks product and platform features

Qualifications

• Databricks, Unity Catalog, DLT
• Familiarity with Machine Learning

Additional Information

All your information will be kept confidential according to EEO guidelines.