Gen AI Data Engineer - Azure AI Foundry

PwC; India; Hybride; INR 900 000 - 1 300 000; Plein temps

Résumé du poste

PwC India seeks a data engineer to build and maintain data pipelines using Apache Airflow and Azure Data Factory, while optimizing data architecture, databases, warehouses, and data lakes for scalable storage and retrieval.

You will validate data quality, develop dimensional models with DAGs, and collaborate with teams to drive data strategy, security, and governance, applying data anonymization and performance-tuning practices.

Qualifications

• Bachelor's degree in a relevant field.
• Minimum 4 years of data engineering experience.
• Proficiency in English (spoken and written).

Responsabilités

• Develop and manage data pipelines with Airflow and Azure Data Factory.
• Improve data architecture, databases, and performance.
• Leverage AWS and Azure Databricks for scalable data solutions.
• Design and maintain data warehouses and data lakes.
• Perform data validation and quality checks.
• Build dimensional models and DAGs to optimize data flow.
• Collaborate on data strategy and anonymization techniques.
• Apply security best practices for data assets.
• Drive data engineering innovations through analytics.

Connaissances

Airflow

Azure

Data Modeling

SQL

Data Warehousing

Analytical Thinking

Formation

Bachelor's degree

Outils

Azure Data Factory

AWS

Databricks

Hadoop

SQL

Description du poste

Responsibilities

• Developing and managing data pipelines using Apache Airflow and Azure Data Factory to support data integration and transformation processes
• Implementing data architecture development and database optimization strategies to enhance data infrastructure and performance
• Utilizing Amazon Web Services (AWS) and Microsoft Azure Databricks for scalable data engineering solutions
• Designing and maintaining data warehouses and data lakes to facilitate efficient data storage and retrieval
• Conducting data validation and quality checks to confirm data accuracy and reliability
• Building dimensional models and directed acrylic graphs (DAGs) to optimize data flow and analysis
• Collaborating with teams to develop data strategy and data anonymization techniques for secure data handling
• Performing database administration and security best practices to safeguard data assets
• Leveraging analytical thinking and creativity to address complex data challenges and drive innovation
• Engaging in query optimization and performance tuning to improve database efficiency and responsiveness

What You Must Have

• At least a Bachelor's degree
• At least 4years of experience
• Oral and written proficiency in English required

What Sets You Apart

• Preference for at least one of the following fields of study: Management Information Systems, Computer and Information Science, Systems Engineering, Electrical Engineering, Chemical Engineering, Industrial Engineering, Mathematics, Statistics, Mathematical Statistics
• Demonstrating proficiency in Amazon Web Services (AWS) and Microsoft Azure Databricks
• Utilizing Apache Hadoop and Apache Airflow for data processing
• Developing data architecture and data modeling strategies
• Implementing data anonymization and data security best practices
• Excelling in data pipeline and data integration techniques
• Navigating complex data engineering platforms and tools