Engineering Manager – Data Engineering

Impelsys; Bengaluru; On-site; INR 1,200,000 - 2,000,000; Full time

Job summary

Impelsys in Bengaluru seeks a senior data delivery lead to oversee end-to-end implementation and delivery of data projects, bringing new capabilities to market and driving large-scale product changes across the lifecycle.

You will supervise internal and external teams, translate business objectives into technical roadmaps, and ensure timely, high-quality data extraction and delivery using Python, Airflow, AWS services, and modern ETL/ELT practices.

Qualifications

• Excellent understanding of enterprise data warehousing principles and ETL/ELT solutions.

Responsibilities

• Oversee end-to-end implementation and delivery of data projects and programmes, bringing new capabilities to the market and implementing large scale changes across the lifecycle.
• Supervise and coordinate internal and external development teams during implementation.
• Translate business objectives and roadmaps into prioritised technology deliverables.
• Review requirements with business/data analysts to identify best technical solutions for data ingestion and distribution tasks.

Skills

Python

Apache Airflow

AWS Glue

AWS Lambda

AWS Redshift

ETL/ELT

Data Lakes/Warehouses

REST APIs

GraphQL

GitHub CI/CD

Agile/Scrum

Leadership

Tools

Python

Apache Airflow

AWS Glue

AWS Lambda

AWS Redshift

ETL/ELT

Data Lakes/Warehouses

Job description

This is an opportunity in which you will be responsible for overall implementation and delivery of projects and programmes, bringing new capabilities to the market, or implementing large scale change to existing products, covering the full lifecycle. You will oversee/manage the technical

delivery management for a business area. You will supervise and coordinate the development teams (both internal employees and external partners) during their implementation. You will work closely with the businesses to understand their business objectives, product strategies and roadmaps and requirements so they can be continually translated into priorThis is an opportunity in which you will be responsible for overall implementation and delivery of projects and programmes, bringing new capabilities to the market, or implementing large scale change to existing products, covering the full lifecycle. You will oversee/manage the technical

delivery management for a business area. You will supervise and coordinate the development teams (both internal employees and external partners) during their implementation. You will work closely with the businesses to understand their business objectives, product strategies and roadmaps and requirements so they can be continually translated into prioritised technology deliverables.

Cloud Integrations, Security

ECS/EKS, API Gateways, Route53/F5, VPC/ELB, REST / SOAP frameworks

Airflow, Glue, Lambdas, Redshift - with Node/Python.

Liaise with business/data analysts and business stakeholders to review new requirements and enhancements and help identify the best technical solution for the data ingestion/distribution tasks

Design data extraction and transformation logic based on prevailing ETL/ELT technologies, languages and open-source frameworks (e.g., Python, Airflow, AWS Data Wrangler, AWS Sage Maker, AWS EMR, AWS Glue etc.)

Supervise, coach and monitor the development teams and vendors in the implementation of designed data pipelines and transformations

Ensure all the data extraction and transformation jobs are successfully scheduled and executed, to guarantee data timeliness and quality

Design, build and run data quality metrics for early identification of anomalies and erroneous data pipeline executions

Report progress, risks and issues to the line manager and other relevant stakeholders (e.g., business users, senior management team, leadership team etc)

Evaluate new trends and technologies in the market and prepare suggestions and

recommendations to be presented to the line manager and other members of the senior management team

Drive a culture of collaborative reviews of designs, code and test plans.

Ability to arrive at actionable insights leveraging the pro-active or pre-emptive measures and algorithms around the data driven decision.

What we're looking for:

• Excellent understanding of enterprise data warehousing principles and ETL/ELT solutions
• Flexible and able to adjust to changing priorities in a fast-paced environment.
• Experience working on or with API's for data delivery/ingestion, e.g., REST APIs, GraphQL, etc.
• Experience working on or with batch-based data delivery/ingestion paradigms, e.g., csv files, ODBC connections, etc.
• Experience in Code management tools (GitHub) Build and Deployment tool (TravisCI).
• Excellent knowledge and working experience on data management technologies and frameworks, preferably on Python, Airflow, AWS API Gateway, SNS, SQS, Lambda
• Good understanding of Lambda and Kappa Architecture patterns
• Ability to work with large and complex data sets, data lakes and data warehouses
• SQL query skills. Hands-on experience with AWS Redshift, PostgreSQL and other database technologies like MySQL, Microsoft SQL Server, etc. and ability to write complex queries to extract/inject data into those databases
• Proven ability to manage a team of developers and to work in an objective oriented environment
• Experience with Agile methodology including WIP management, backlog grooming/prioritization, reporting and dashboards.

Key Skills:-

• Strong hands-on experience with Python
• Apache Airflow or similar workflow orchestration tools
• AWS Data Services - Glue, Lambda, Redshift, EMR
• ETL/ELT, Data Lakes & Data Warehousing
• REST/SOAP APIs and API integrations
• AWS API Gateway, SNS, SQS
• ECS/EKS, VPC, ELB, Route53/F5
• GitHub, CI/CD and Agile methodologies
• Engineering team leadership and stakeholder management