Data Engineering Team Lead

Mogi I/O : OTT/Podcast/Short Video Apps for you; Hyderabad; On-site; INR 2,200,000 - 2,800,000; Full time

Job summary

A tech company specializing in data solutions is seeking a Senior Data Integration and AI Analytics Specialist in Hyderabad. The role requires 5–8 years of experience in data integration and strong skills in Dataiku DSS, SQL, and Python. This position involves designing scalable data pipelines, automating workflows, and enabling AI/ML model development. Candidates should possess a solid understanding of ETL principles and cloud-based data operations. Competitive compensation package is offered.

Qualifications

• 5–8 years of experience in data integration, data engineering, or AI/analytics development.
• Strong hands-on experience with Dataiku DSS.
• Expertise in SQL and Python for data transformation.
• Experience with Snowflake, AWS, and cloud-based data pipelines.

Responsibilities

• Develop and maintain end-to-end ETL workflows integrating various data sources.
• Design scalable data pipelines with strong error handling.
• Collaborate with teams to automate workflows and ensure data quality.

Skills

Dataiku DSS

SQL

Python (Pandas, PySpark)

AWS

ETL design principles

API automation

data validation

dashboards and KPI visualizations

Job description

Overview

The Senior Data Integration and AI Analytics Specialist will design, develop, and optimize scalable data pipelines and AI-driven analytics workflows across enterprise cloud environments. This role requires deep hands-on experience with Dataiku DSS, strong command of SQL and Python, and solid working knowledge of Snowflake, AWS, and ETL design principles. The specialist will partner closely with data science, business, and engineering teams to automate complex workflows, ensure data quality, build reusable analytical assets, and drive operational efficiency across the organization.

Work Type

Permanent

Location

Hyderabad

Compensation

INR 2,200,000 – 2,800,000

Key Responsibilities

• Data Integration & Pipeline Development: Develop and maintain end-to-end ETL/ELT workflows integrating data from SQL Server, SAP HANA, Oracle, AWS Redshift, cloud storage, and APIs.
• Design scalable batch and real-time data pipelines with strong error handling and performance optimization.
• Implement workflow orchestration patterns supporting enterprise-grade data operations.

• Dataiku DSS Administration & Automation: Manage Dataiku environments including licenses, credentials, projects, and automation nodes.
• Automate pipelines, API integrations, and recurring tasks using Python, SQL, and Dataiku recipes/plugins.
• Build reusable components for auditing, user activity tracking, and operational reporting.

• AI & Analytics Enablement: Prepare, cleanse, and transform datasets for AI/ML model development in collaboration with data scientists.
• Implement data validation, monitoring, and reconciliation frameworks to ensure accuracy and reduce manual effort.
• Build interactive dashboards and KPI visualizations to support business decision-making.

Must-Have Qualifications

• 5–8 years of experience in data integration, data engineering, or AI/analytics development.
• Strong hands-on experience with Dataiku DSS (development + administration).
• Expertise in SQL and Python (Pandas, PySpark) for data transformation and automation.
• Experience with Snowflake, AWS (S3, Redshift), and cloud-based data pipelines.
• Solid understanding of ETL design patterns, batch scheduling, and workflow orchestration.
• Working knowledge of API automation, data validation, and reconciliation reporting.
• Experience collaborating with data science and business teams.
• Strong analytical, problem-solving, and documentation skills.

Nice-to-Have Qualifications

• Certifications: Dataiku Core Designer / ML Practitioner, AWS Solutions Architect, Azure Fundamentals.
• Exposure to DevOps tooling for data operations.
• Experience in dashboarding/BI tools (Tableau, Power BI, etc.).

Referrals and job notices are provided for informational purposes and do not form part of this job description.