Databricks Architect

Quantum Integrators; New Jersey; On-site; USD 150,000 - 190,000; Full time

Job summary

Quantum Integrators in New Jersey is seeking an experienced Databricks Architect to design, implement, and optimize scalable data platforms using the Databricks Lakehouse architecture. The role emphasizes collaboration with business and technical stakeholders to deliver secure, compliant data solutions in the Life Sciences sector.

The candidate should have 10+ years in data engineering and architecture, 3–5+ years with Databricks, and strong SQL/Python skills.

Qualifications

• Bachelor’s or Master’s degree in CS, IS, Eng, or related field.
• 10+ years of data engineering and data architecture experience.
• 3–5+ years hands-on Databricks experience.
• Experience in Life Sciences industry (pharma, biotech, medical devices, or healthcare).
• Strong SQL and Python programming skills; Spark expertise (PySpark/Scala/Spark SQL).
• Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
• Cloud experience with Azure, AWS, or Google Cloud.
• Knowledge of data lakehouse, data warehousing, and data modeling.
• CI/CD and DevOps practices.
• Excellent communication, leadership, and stakeholder management.

Responsibilities

• Design and implement enterprise-scale data platforms using Databricks.
• Architect end-to-end data solutions including ingestion, transformation, storage, and analytics.
• Develop scalable ETL/ELT pipelines using Apache Spark and Databricks.
• Design and optimize Delta Lake architectures for performance and reliability.
• Collaborate with business stakeholders to translate data requirements into technical solutions.
• Define data governance, security, and best practices for the Databricks environment.
• Optimize workloads for cost, scalability, and performance.
• Lead architecture reviews, technical design sessions, and code reviews.
• Mentor data engineers and establish development standards.
• Integrate Databricks with enterprise data sources, APIs, and reporting platforms.
• Support production deployments, troubleshooting, and performance tuning.
• Ensure data solutions align with Life Sciences regulatory and compliance requirements, where applicable.

Skills

Data engineering

Data architecture

SQL

Python

Communication & leadership

Stakeholder management

CI/CD / DevOps

Education

Bachelor's or Master's degree in Computer Science or related field

Tools

Databricks

Delta Lake

Unity Catalog

Databricks Workflows

Azure

AWS

Google Cloud

Apache Spark

Job description

Job Title: Databricks Architect (Onsite)

We are seeking an experienced Databricks Architect to design, architect, and implement scalable data platforms using the Databricks Lakehouse architecture. The ideal candidate will have extensive experience in modern data engineering, cloud platforms, and enterprise data architecture within the Life Sciences industry. This role requires close collaboration with business and technical stakeholders to deliver secure, scalable, and compliant data solutions.

Job Summary

We are seeking an experienced Databricks Architect to design, architect, and implement scalable data platforms using the Databricks Lakehouse architecture. The ideal candidate will have extensive experience in modern data engineering, cloud platforms, and enterprise data architecture within the Life Sciences industry. This role requires close collaboration with business and technical stakeholders to deliver secure, scalable, and compliant data solutions.

Key Responsibilities

• Design and implement enterprise-scale data platforms using Databricks.
• Architect end-to-end data solutions, including data ingestion, transformation, storage, and analytics.
• Develop scalable ETL/ELT pipelines using Apache Spark and Databricks.
• Design and optimize Delta Lake architectures for performance and reliability.
• Collaborate with business stakeholders to understand data requirements and translate them into technical solutions.
• Define data governance, security, and best practices for the Databricks environment.
• Optimize workloads for cost, scalability, and performance.
• Lead architecture reviews, technical design sessions, and code reviews.
• Mentor data engineers and establish development standards.
• Integrate Databricks with enterprise data sources, APIs, and reporting platforms.
• Support production deployments, troubleshooting, and performance tuning.
• Ensure data solutions align with Life Sciences regulatory and compliance requirements, where applicable.

Required Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.
• 10+ years of experience in data engineering and data architecture.
• 3–5+ years of hands‑on experience with Databricks.
• Experience working in the Life Sciences industry (pharmaceutical, biotechnology, medical devices, or healthcare).
• Strong expertise in Apache Spark (PySpark/Scala/Spark SQL).
• Strong SQL and Python programming skills.
• Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
• Experience with Azure, AWS, or Google Cloud.
• Strong understanding of data lakehouse architecture, data warehousing, and data modeling.
• Experience implementing CI/CD pipelines and DevOps practices.
• Excellent communication, leadership, and stakeholder management skills.