Data Platform Engineer - AI

Programmers.io; Redmond; Sur place; AUD 180 000 - 240 000; Plein temps

Résumé du poste

Programmers.io is seeking a hands-on Technical Lead for Data Platform & Services Engineering with strong Azure data engineering, backend services, APIs, analytics, and cloud architecture expertise. You will design, build, modernize, and operate enterprise-scale data platforms on Microsoft Azure.

You will lead design and implementation of scalable data platforms, lakehouse architectures, and secure API backends, while guiding modernization, reliability, and DevOps practices across the team.

Qualifications

• Experience with ETL/ELT, data warehousing, and data modeling.
• Experience with Lakehouse/Medallion architecture.
• Hands-on Azure Data Engineering with cloud architecture and data platforms.

Responsabilités

• Design and build scalable Azure data platforms, ETL/ELT pipelines, and analytical solutions.
• Design Lakehouse/Medallion architectures using ADLS Gen2, Delta Lake, Bronze/Silver/Gold layers.
• Design and develop REST APIs and backend services using C#, .NET, and ASP.NET Core.
• Build data and integration services using Azure SQL, Cosmos DB, Kusto, Azure Functions, Event Hubs, and Service Bus.
• Implement microservices and event-driven processing patterns with secure authentication and authorization.
• Modernize legacy data pipelines, services, APIs, storage platforms, and application components.
• Support platform and data migrations while ensuring functional and data parity.
• Implement logging, telemetry, dashboards, and proactive alerting using Azure Monitor and Application Insights.

Connaissances

ETL/ELT

Data Warehousing

Data Modeling

Lakehouse

Medallion Architecture

Outils

Azure DevOps

Git

Azure

Docker

Kubernetes (AKS)

Azure Key Vault

Azure Monitor

Cosmos DB

ADLS Gen2

Delta Lake

Description du poste

We are looking for a hands-on Technical Lead – Data Platform & Services Engineering with strong expertise in Azure Data Engineering, backend services, APIs, analytics, and cloud architecture. The candidate will design, build, modernize, and operate enterprise-scale data platforms and services on Microsoft Azure.

Key Responsibilities:

• Design and build scalable Azure data platforms, ETL/ELT pipelines, and analytical solutions.
• Design Lakehouse/Medallion architectures using ADLS Gen2, Delta Lake, and Bronze/Silver/Gold layers
• Design and develop REST APIs and backend services using C#, .NET, and ASP.NET Core.
• Build data and integration services using Azure SQL, Cosmos DB, Kusto, Azure Functions, Event Hubs, and Service Bus.
• Implement microservices and event-driven processing patterns with secure authentication and authorization.

Modernization & Reliability

• Modernize legacy data pipelines, services, APIs, storage platforms, and application components.
• Support platform and data migrations while ensuring functional and data parity.
• Implement logging, telemetry, dashboards, and proactive alerting using Azure Monitor and Application Insights

Cloud, DevOps & Technical Leadership

• Build and maintain CI/CD pipelines using Azure DevOps and Git.
• Implement secure cloud practices using Azure Key Vault, identity, authentication, and access controls.
• Support Docker and Kubernetes/AKS-based workloads where applicable.

Mandatory Skills:

• ETL/ELT, Data Warehousing & Data Modeling
• Lakehouse / Medallion Architecture