Lead / Manager - Data Engineering

Blend360; Hyderabad; On-site; INR 4,200,000 - 5,800,000; Full time

Job summary

Blend360 in Hyderabad seeks an accomplished Manager – Data Engineering to lead a team and own on-premise data infrastructure. You will drive scalable pipelines using Python, Spark, and Airflow, delivering robust ETL/ELT workflows and optimized SQL queries.

You will collaborate with architects and stakeholders to align data solutions with business goals, oversee design and delivery, and champion best practices across the data engineering practice.

Qualifications

• 8+ years of overall experience in Data Engineering.
• Strong hands-on experience with Python.
• Strong experience with Apache Spark / PySpark.
• Advanced SQL skills, including complex joins, CTEs, window functions, subqueries, and query optimization.
• Strong experience with Apache Airflow for workflow orchestration and scheduling.
• Experience working with on-premise data environments.
• Strong understanding of ETL/ELT processes and data pipeline architecture.
• Experience handling large volumes of data and optimizing data processing workloads.
• Good understanding of data quality, monitoring, troubleshooting, and performance optimization.

Responsibilities

• Lead and manage a team of Data Engineers and provide technical guidance and mentorship.
• Design, develop, and optimize scalable data pipelines in an on-premise environment.
• Build robust ETL/ELT workflows using Python, Spark, SQL, and Airflow.
• Design and manage complex Apache Airflow DAGs for data pipeline orchestration.
• Develop and optimize Spark-based data processing solutions for large datasets.
• Write complex and optimized SQL queries for data extraction, transformation, and analysis.
• Troubleshoot pipeline failures, performance issues, and data quality challenges.
• Work closely with architects, business stakeholders, and cross-functional teams to understand requirements and deliver solutions.
• Conduct code reviews and ensure adherence to engineering and development best practices.
• Drive technical design, estimation, planning, and end-to-end project delivery.
• Monitor team performance, project timelines, risks, and dependencies.
• Mentor engineers and contribute to building a strong data engineering practice.

Skills

Python

Apache Spark

SQL

Leadership

Data pipelines

Tools

Apache Airflow

On-premise data environments

ETL/ELT

Job description

Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly aligning human expertise with artificial intelligence. The company is dedicated to unlocking value and fostering innovation for its clients by harnessing world-class people and data-driven strategy. We believe that the power of people and AI can have a meaningful impact on your world, creating more fulfilling work and projects for our people and clients. For more information, visitwww.blend360.com

Job Description

We are looking for an experiencedManager – Data Engineeringwith 8+ years of experience in building and managing data engineering solutions in anon-premise environment. The ideal candidate should have strong hands-on expertise inPython, Apache Spark, SQL, and Apache Airflow, along with proven experience in leading data engineering teams and delivering scalable data pipelines.

Key Responsibilities

• Lead and manage a team of Data Engineers and provide technical guidance and mentorship.
• Design, develop, and optimize scalabledata pipelinesin an on-premise environment.
• Build robust ETL/ELT workflows usingPython, Spark, SQL, and Airflow.
• Design and manage complexApache Airflow DAGsfor data pipeline orchestration.
• Develop and optimize Spark-based data processing solutions for large datasets.
• Write complex and optimized SQL queries for data extraction, transformation, and analysis.
• Troubleshoot pipeline failures, performance issues, and data quality challenges.
• Work closely with architects, business stakeholders, and cross-functional teams to understand requirements and deliver solutions.
• Conduct code reviews and ensure adherence to engineering and development best practices.
• Drive technical design, estimation, planning, and end-to-end project delivery.
• Monitor team performance, project timelines, risks, and dependencies.
• Mentor engineers and contribute to building a strong data engineering practice.

Qualifications

• 8+ years of overall experience in Data Engineering.
• Strong hands-on experience withPython.
• Strong experience withApache Spark / PySpark.
• AdvancedSQLskills, including complex joins, CTEs, window functions, subqueries, and query optimization.
• Strong experience withApache Airflowfor workflow orchestration and scheduling.
• Experience working withon-premise data environments.
• Strong understanding of ETL/ELT processes and data pipeline architecture.
• Experience handling large volumes of data and optimizing data processing workloads.
• Good understanding of data quality, monitoring, troubleshooting, and performance optimization.

Leadership Requirements

• Proven experience managing or leadingData Engineering teams.
• Strong stakeholder and client management skills.
• Ability to provide technical direction while managing project delivery.
• Experience with resource planning, task allocation, mentoring, and performance management.
• Strong communication and problem-solving skills.
• Ability to work in a fast-paced, Agile environment.

Good to Have

• Experience in large-scale enterprise data platforms.
• Experience with data warehouse concepts and data modelling.
• Experience with on-premise Hadoop/data ecosystems.
• Experience in migrating or modernizing legacy/on-premise data platforms.

By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply