Azure Data Engineer: Pipelines, APIs & AI

Agility Partners; Carmel (IN); On-site; USD 120,000 - 160,000; Full time

Job summary

Agility Partners is seeking an Engineer to design and build cloud-based data platforms and backend services for a client in the energy sector. You will create scalable data pipelines using Azure Data Factory, Synapse, and Data Lake, while developing REST APIs with FastAPI and Python.

The role emphasizes data quality, automated testing, and CI/CD, plus integrating AI capabilities and model-serving APIs into the platform.

Qualifications

• Hands-on experience with Azure data services and data pipelines.
• Strong Python development and API integration skills.
• Familiarity with CI/CD, testing, and deployment best practices.
• Knowledge of asynchronous architectures and event-driven systems.

Responsibilities

• Design, build, and maintain scalable data pipelines and orchestration workflows using Azure tools.
• Develop backend services and REST APIs with FastAPI and Python.
• Implement data quality checks and CI/CD processes to improve reliability.
• Integrate ML/AI capabilities into data platforms and applications.

Skills

Python programming

Data engineering

API development

CI/CD practices

Asynchronous/event-driven architecture

Tools

Azure Data Factory

Azure Synapse

Azure Data Lake Storage Gen2

SQL

FastAPI

Azure Functions

Containerization

GitHub

Job description

Agility Partners is seeking an Engineer to design and build cloud-based data platforms and backend services for a client in the energy sector. You will create scalable data pipelines using Azure Data Factory, Synapse, and Data Lake, while developing REST APIs with FastAPI and Python.

The role emphasizes data quality, automated testing, and CI/CD, plus integrating AI capabilities and model-serving APIs into the platform.