Lead Data Architect & Engineer: Databricks Data Lakehouse

Quantum Technologies. LLC; Atlanta (GA); On-site; USD 89,000 - 126,000; Full time

Job summary

Quantum Technologies LLC in Atlanta is seeking a Data Architect and Lead Data Engineer to drive enterprise data strategy, design scalable architectures, and lead data engineering initiatives across platforms such as Databricks and Azure.

You will partner with business stakeholders to translate requirements into technical specs, build robust data pipelines, enforce governance, and mentor engineers through complex projects.

Qualifications

• 10+ years of experience in enterprise data architecture and engineering.
• Advanced proficiency in data processing frameworks and tools such as Parquet, Iceberg, PySpark, Lambda, Databricks, and Azure data services.
• Proficiency in programming languages like Python, Java, or Scala for data processing.
• Extensive hands-on experience with Databricks, data lakehouse architectures, and governance controls.
• Automation and CI/CD in data projects using Git/Bitbucket and Jenkins.

Responsibilities

• Interface with business partners and IT to define data requirements and ensure alignment with IT strategy.
• Translate high-level business needs into functional specifications for data systems and manage requirements, approvals, and changes.
• Develop data models, data flows, and governance processes to support reliable analytics.
• Build, test, and maintain large-scale data pipelines and reports using SQL-driven tools.
• Participate in buy-vs-build decisions and project planning with business units.
• Mentor junior team members and ensure adherence to standards and best practices.

Skills

Data engineering

Databricks

PySpark

Python

Java/Scala

CI/CD

Education

Bachelor's degree

Tools

Git/Bitbucket

Jenkins

Job description

Quantum Technologies LLC in Atlanta is seeking a Data Architect and Lead Data Engineer to drive enterprise data strategy, design scalable architectures, and lead data engineering initiatives across platforms such as Databricks and Azure.

You will partner with business stakeholders to translate requirements into technical specs, build robust data pipelines, enforce governance, and mentor engineers through complex projects.