AI & Data -Senior

EY; Bengaluru; Híbrido; INR 1.800.000 - 3.000.000; Jornada completa

Descripción de la vacante

EY is seeking a data engineer specializing in Azure data services to design and implement production-grade data pipelines. You will work with Azure Data Factory, Azure Databricks, and Azure SQL to build scalable data architectures and ensure data quality across pipelines.

Responsibilities include creating analytics-ready datasets for BI tools, applying best practices in data modelling and secure data handling, and enabling CI/CD for data workflows in a regulated environment.

Formación

• 4–8 years of experience in data engineering or closely related field.
• Proficient with Azure data services and modern data architectures.
• Strong problem solving, analytical and data quality focus.

Responsabilidades

• Design and build production-grade data pipelines using Azure Data Factory and Azure Databricks.
• Develop scalable data architectures leveraging Azure SQL, Data Lake and data modelling techniques.
• Implement data quality validation, reconciliation processes and resolve pipeline issues.
• Deliver analytics-ready datasets for reporting platforms such as Power BI.
• Implement CI/CD pipelines for data engineering workflows and ensure secure data handling.

Conocimientos

Azure Data Factory

Azure Databricks

Azure SQL

Data Lake

Data modelling (star/snowflake)

Data pipeline design

Descripción del empleo

Experience: 04- 08 years.

Location: Bangalore

Key Responsibilities

Data Engineering

• Design and build production-grade data pipelines using Azure Data Factory and Azure Databricks.
• Develop scalable data architectures leveraging Azure SQL, Data Lake, and modern data modelling techniques (star/snowflake schemas).
• Implement data quality validation, reconciliation processes, and resolve operational issues.

Analytics & AI Enablement

• Use Python for data processing, transformation, and analytics.
• Develop and maintain Jupyter Notebooks for exploration and experimentation.
• Utilize MLflow for experiment tracking and model lifecycle management.

BI Enablement

• Deliver curated, analytics-ready datasets for Power BI and other reporting platforms.
• Optimize data models and structures for efficient reporting, dashboarding, and analytics consumption.

Engineering Practices

• Implement CI/CD pipelines for data engineering workflows.
• Ensure secure data handling practices, including sensitive data protection.
• Monitor, troubleshoot, and resolve pipeline failures and performance issues.

Required Skills & Qualifications

Strong expertise in:

• Azure Data Factory
• Azure Databricks
• Azure SQL and Data Lake
• Solid understanding of data modelling (star and snowflake schemas).
• Experience building scalable and reliable data pipelines.

Core Competencies

• Strong problem-solving and analytical skills.
• Ability to manage production-grade data environments.
• Focus on data quality, scalability, and performance optimization.

Nice to Have

• Experience working in regulated data environments.
• Exposure to Azure OpenAI or AI/ML use cases.
• Knowledge of prompt engineering, evaluation frameworks, and experimentation techniques.