Azure Data Architect: Databricks, Data Pipelines & Governance

Cloud Analytics Technologies, LLC; Atlanta (GA); On-site; USD 120,000 - 190,000; Full time

Job summary

Cloud Analytics Technologies, LLC seeks a senior data engineer to partner with business units and IT, translating complex requirements into scalable data solutions. You will design and implement large-scale data pipelines, optimize Spark workloads in Databricks, and drive data governance and security measures across cloud platforms.

The role requires 10+ years in data engineering, proficiency with Parquet, Iceberg, PySpark, and Azure data services, and strong SQL/ER data modeling skills.

Qualifications

• 10+ years of experience in data engineering, including design, development, testing and operations.
• Proficiency with data processing frameworks and tools for large-scale pipelines.
• Strong programming skills in Python, Java, or Scala for data processing.

Responsibilities

• Interface between IT and business units to gather requirements and translate to specifications.
• Build, maintain and optimize large-scale data pipelines in Databricks and related cloud services.
• Develop reports and governance artifacts using SQL and BI tools.
• Lead continuous delivery and automation efforts for data platforms.

Skills

Data engineering

Parquet

Iceberg

PySpark

Databricks

Azure data services

Python

Java

Scala

CI/CD

Git/Bitbucket

Jenkins

DAB deployments

Security & governance

AWS integration

SQL/Relational DBs

Education

Bachelor's degree in Computer Science / Information Systems

Tools

Git/Bitbucket

Jenkins

SQL (Oracle/SQL Server)

Data modeling (ER)

Job description

Cloud Analytics Technologies, LLC seeks a senior data engineer to partner with business units and IT, translating complex requirements into scalable data solutions. You will design and implement large-scale data pipelines, optimize Spark workloads in Databricks, and drive data governance and security measures across cloud platforms.

The role requires 10+ years in data engineering, proficiency with Parquet, Iceberg, PySpark, and Azure data services, and strong SQL/ER data modeling skills.