Data Engineer Senior Consultant

NTT DATA BUSINESS SOLUTIONS; Bengaluru; On-site; INR 2,600,000 - 4,200,000; Full time

Job summary

NTT DATA BUSINESS SOLUTIONS is seeking a Senior Data Migration Engineer in Bengaluru to design and implement Python-based extract/load pipelines, dbt transformations, and Airflow orchestration for a large Informatica PowerCenter migration to a modern data stack.

The role requires deep expertise in translating Informatica workflows into Python, dbt, and Airflow, delivering production-quality outputs at enterprise scale and contributing to CI/CD pipelines and handover materials.

Qualifications

• 5+ years of Informatica PowerCenter experience including mappings, sessions, workflows and parameterization.
• 3+ years of hands-on dbt Core experience with model layering, macros, tests and incremental materializations.
• Any number of years of Airflow experience including DAG authoring and task dependencies.
• 5+ years of Python experience including production use of parallelism and batch data processing.
• 5+ years of SQL experience with complex joins, window functions, and aggregations on large datasets.
• Experience with Oracle Data Pump (expdp/impdp) or SQL Server BCP / BULK INSERT for high-volume data movement.

Responsibilities

• Assess Informatica workflows and translate into Python EL scripts and dbt models.
• Develop and maintain Python EL pipelines to land high-volume data into the target warehouse using SQLAlchemy, cx_Oracle, pyodbc and bulk export tools.
• Design and develop dbt transformation models based on Informatica logic with best practices like incremental materializations and snapshots.
• Develop dbt tests and Python unit tests to ensure data quality and transformation correctness.
• Create Airflow DAGs to orchestrate Python EL and dbt scripts end-to-end, matching source workflow outputs.
• Contribute to GitLab CI/CD pipelines for dbt and Airflow, including lint gates, tests and deployment to shared NAS.
• Perform peer code reviews and provide constructive technical feedback.
• Troubleshoot performance issues and data discrepancies during SIT/UAT, reconciling row counts between source and target.

Skills

Python

SQL

Tools

Informatica PowerCenter

dbt Core

Airflow

SQLAlchemy

cx_Oracle

pyodbc

Oracle Data Pump

SQL Server BCP

Job description

Job Summary

The Senior Data Migration Engineer serves as a hands-on technical contributor and delivery anchor, responsible for designing and implementing Python-based extract/load pipelines, dbt-based transformations, and Airflow-based orchestration for a large-scale migration from Informatica PowerCenter to a modern data stack. This role requires deep expertise in converting complex Informatica workflows into Python, dbt, and Airflow, and the ability to deliver production-quality, functionally equivalent output at enterprise scale.

Responsibilities

• Assess complex Informatica workflows including sessions, mappings, and parameterizations and translate them into equivalent Python EL scripts and dbt transformation models.
• Develop and maintain Python EL pipelines to land high-volume data, including billion-row tables, into the target warehouse using SQLAlchemy, cx_Oracle, pyodbc, and bulk export tools such as Oracle Data Pump and SQL Server BCP.
• Design and develop dbt transformation models based on Informatica mapping logic, incorporating dbt best practices including model layering, macros, incremental strategies, and snapshot patterns.
• Develop dbt-native tests as well as custom Python unit tests to validate transformation correctness and data quality.
• Develop Airflow DAGs to orchestrate Python EL and dbt scripts end-to-end, producing output that is functionally equivalent to the source Informatica workflows.
• Contribute to GitLab CI/CD pipeline for dbt and Airflow code, including lint gates, automated testing, and deployment to shared NAS.
• Perform peer code reviews and provide constructive technical feedback to fellow engineers.
• Troubleshoot performance issues and data discrepancies during SIT and UAT, including row-count reconciliation between source Oracle/SQL Server systems and the target warehouse.
• Contribute to technical documentation, runbooks, and handover materials.

Required Skills Experience

• 5+ years of Informatica PowerCenter experience - mappings, sessions, workflows, and parameterization
• 3+ years of hands-on dbt Core experience - model layering, macros, tests, incremental materializations
• Any number of years of Airflow experience - DAG authoring, task dependencies, and incorporation with scheduling tool
• 5+ years of Python experience, including production use of parallelism and batch data processing
• 5+ years of SQL experience - complex joins, window functions, aggregations across large datasets
• Experience with Oracle Data Pump (expdp/impdp) or SQL Server BCP / BULK INSERT for high-volume data movement

Preferred

• Experience with SQLAlchemy, cx_Oracle, or pyodbc for Python-based database connectivity
• Experience mentoring junior and mid-level engineers in offshore delivery models.