Role overview
Take ownership of end-to-end data modernization on Azure Synapse or Fabric, translating client requirements into scalable pipelines and delivering governed, high-quality data products. The work supports downstream AI/ML applications and natural-language search experiences.
Responsibilities
- Design, build, and optimize scalable ETL/ELT pipelines using advanced T-SQL and Python within Azure Synapse Dedicated SQL Pools and Azure Data Hub.
- Develop and manage medallion architecture schemas (Bronze, Silver, Gold) tuned for high-performance SQL analytics and AI-ready workloads.
- Integrate AI capabilities into data operations, including automated incident triage, cost-reduction recommendations, and proactive pipeline monitoring.
- Implement data quality checks and monitoring frameworks that keep datasets trustworthy.
- Administer data warehouses, data lakes, and relational plus NoSQL databases.
- Operate workflow orchestration tools such as Airflow, Prefect, or Dagster for scheduling and monitoring pipelines.
- Collaborate with AI/ML engineers to ground large language model applications through vector embeddings and semantic search metadata.
- Maintain data security and compliance standards, and tune storage plus processing costs using FinOps automation.
- Troubleshoot and resolve data issues independently.
Requirements
- Expertise in T-SQL with deep experience using Azure Synapse Dedicated SQL Pools, Azure Data Hub, and relational plus NoSQL databases.
- Strong Python proficiency for data manipulation, AI orchestration, and pipeline development, including PySpark and Pandas.
- Hands-on experience across the Azure Stack, including Synapse Pipelines, ADLS Gen2, and Azure Data Hub for enterprise data warehouse operations.
- Familiarity with AI integration patterns such as prompt engineering, vector databases, and semantic layer management for natural-language query tools.
- Experience building and managing data pipelines and ETL/ELT processes at scale.
- Solid grasp of data warehousing concepts, data modeling, and data quality principles.
- Ability to work independently and take ownership of data infrastructure components.