Lead Azure Data Engineer

Machine Intelligence Technologies, LLC; Alpharetta, Northern (GA, KY); Hybrid; USD 150,000 - 190,000; Full time

Job summary

Machine Intelligence Technologies LLC seeks an Architect to design and implement scalable data platforms on Azure and Databricks. You will lead development of ETL/ELT pipelines across batch and streaming workloads, while ensuring data quality, governance, and performance.

Collaborating with data scientists, analysts, and business teams, you will shape data models and support operations, security, and architecture best practices across the platform.

Qualifications

• Experience architecting scalable data platforms and pipelines.
• Strong knowledge of Azure and Databricks.
• Proficiency with Python for Spark data engineering.

Responsibilities

• Architect, design, and implement scalable data platforms on Azure/Databricks.
• Lead ETL/ELT pipelines across batch and streaming workloads.
• Mentor engineers and enforce data governance and quality standards.

Job description

Architect, design, and implement scalable data platforms and pipelines on Azure and Databricks.

• Build and optimize data ingestion, transformation, and processing workflows across batch and real-time data streams.
• Work extensively with ADLS, Delta Lake, and Spark (Python) for large-scale data engineering.
• Lead the development of complex ETL/ELT pipelines, ensuring high quality, reliability, and performance.
• Design and implement data models, including conceptual, logical, and physical models for analytics and operational workloads.
• Work with relational and lakehouse systems including PostgreSQL and Delta Lake.
• Define and enforce best practices in data governance, data quality, security, and architecture.
• Collaborate with architects, data scientists, analysts, and business teams to translate requirements into technical solutions.
• Troubleshoot production issues, optimize performance, and support continuous improvement of the data platform.
• Mentor junior engineers and contribute to building engineering standards and reusable components.

MACHINE INTELLIGENCE TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. MACHINE INTELLIGENCE TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will MACHINE INTELLIGENCE TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract