Lead Data Engineer

Experion Technologies Pvt. Ltd.; Ernakulam; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Experion Technologies Pvt. Ltd. is seeking an experienced Lead Data Engineer with 7+ years of hands-on SQL Server, Azure Data Factory, ETL/data integration experience. You will design and maintain scalable data pipelines, optimize SQL, and drive cloud data solutions.

You will collaborate across architects, developers, QA, and analysts in an Agile setting, ensuring data quality, performance, and reliable production deployments while maintaining comprehensive technical docs.

Qualifications

• 7+ years in Data Engineering, ETL or related roles.
• Strong hands-on SQL Server development and administration.
• Expertise with T‑SQL, stored procedures, views and functions.
• Experience with Azure Data Factory for ETL/ELT and data integration.
• Familiarity with data warehousing concepts and data modeling.
• Experience with CI/CD and source control tools (Azure DevOps, Git, Bitbucket).
• Ability to work in Agile/Scrum environments and with distributed teams.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory.
• Develop complex T-SQL queries, stored procedures, and database objects in SQL Server.
• Performance tuning, troubleshooting, and data integrity management for SQL Server.
• Build data integration pipelines connecting multiple systems and data sources.
• Work with Azure data platform services to support cloud-based solutions.
• Analyze changing source data structures and adapt pipelines accordingly.
• Create and maintain technical documentation covering data flows and mappings.
• Participate in code reviews and follow development and deployment standards.
• Support production deployments and ongoing operational activities.

Skills

SQL Server development

ETL/ELT

Azure data platforms

Data modeling

Agile/Scrum

CI/CD

Data integration

Tools

Azure Data Factory

Microsoft SQL Server

Azure DevOps

Git

Bitbucket

Job description

We are looking for an experienced Lead Data Engineer with 7+ years of hands-on experience in SQL Server, Azure Data Factory, ETL/data integration, and Azure data platforms.

Job Description

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and related Azure services.
• Develop complex and optimized T-SQL queries, stored procedures, views, functions, and database objects in SQL Server.
• Perform SQL Server database development and administration activities, including performance tuning, troubleshooting, and data integrity management.
• Build and maintain data integration pipelines connecting multiple source and target systems.
• Work with Azure data platform services to develop and support cloud-based data solutions.
• Analyze source systems and changing data structures and adapt data pipelines accordingly.
• Apply light-to-medium data modeling techniques to support analytical and operational data requirements.
• Work with large volumes of structured and semi-structured data and ensure efficient data processing.
• Troubleshoot data pipeline failures, data quality issues, performance problems, and production incidents.
• Monitor data pipelines and proactively identify and resolve issues.
• Collaborate with developers, architects, QA teams, business analysts, and other stakeholders to deliver data solutions.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, backlog refinement, reviews, and retrospectives.
• Create and maintain technical documentation covering data flows, mappings, processes, dependencies, and troubleshooting procedures.
• Participate in code reviews and follow established development, security, and deployment standards.
• Support production deployments, releases, and operational activities as required.

Job Specification

• 7+ years of experience in Data Engineering, ETL, data integration, or related roles.
  • Strong hands-on experience with Microsoft SQL Server development and administration.
• Advanced knowledge of T-SQL, including:
  • Complex queries
  • Stored procedures
  • Views and functions
  • Query optimization
  • Indexing and performance tuning
  • Error handling and troubleshooting
• Strong experience with Azure Data Factory (ADF) for ETL/ELT and data integration.
• Practical experience working with Azure data platform services.
• Good understanding of data warehousing and data integration concepts.
• Experience with data modelling, including dimensional modelling and logical/physical data models.
• Ability to understand and work with evolving or changing source data structures.
• Strong troubleshooting and problem-solving skills.
• Strong technical documentation and communication skills.
• Experience working in an Agile/Scrum development environment.
• Ability to work independently while collaborating effectively with distributed/onshore-offshore teams.
• Experience with CI/CD and source control tools such as Azure DevOps, Git, or Bitbucket.
• To adhere to the Information Security Management policies and procedures.