Lead Data Engineer

Experion Technologies; Kerala; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Experion Technologies in India (Kerala) is seeking an experienced Data Engineer to design, build, and optimize large-scale ETL/ELT pipelines using Azure Data Factory and SQL Server. You will develop complex T-SQL queries, manage database objects, and tune performance for reliability at scale.

You will also integrate diverse data sources, apply dimensional modelling, support cloud-based solutions on the Azure data platform, and collaborate with distributed teams in an Agile environment to deliver

Qualifications

• 7+ years of experience in Data Engineering, ETL, data integration, or related roles.
• Strong hands-on experience with Microsoft SQL Server development and administration.
• Advanced knowledge of T-SQL, including: Complex queries, Stored procedures, Views and functions, Query optimization, Indexing and performance tuning, Error handling and troubleshooting.
• Strong experience with Azure Data Factory (ADF) for ETL/ELT and data integration.
• Practical experience working with Azure data platform services.
• Good understanding of data warehousing and data integration concepts.
• Experience with data modelling, including dimensional modelling and logical/physical data models.
• Ability to understand and work with evolving or changing source data structures.
• Strong troubleshooting and problem-solving skills.
• Strong technical documentation and communication skills.
• Experience working in an Agile/Scrum development environment.
• Ability to work independently while collaborating effectively with distributed/onshore-offshore teams.
• Experience with CI/CD and source control tools such as Azure DevOps, Git, or Bitbucket.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and related Azure services.
• Develop complex and optimized T-SQL queries, stored procedures, views, functions, and database objects in SQL Server.
• Perform SQL Server database development and administration activities, including performance tuning, troubleshooting, and data integrity management.
• Build and maintain data integration pipelines connecting multiple source and target systems.
• Work with Azure data platform services to develop and support cloud-based data solutions.
• Analyze source systems and changing data structures and adapt data pipelines accordingly.
• Apply light-to-medium data modeling techniques to support analytical and operational data requirements.
• Work with large volumes of structured and semi-structured data and ensure efficient data processing.
• Troubleshoot data pipeline failures, data quality issues, performance problems, and production incidents.
• Monitor data pipelines and proactively identify and resolve issues.
• Collaborate with developers, architects, QA teams, business analysts, and other stakeholders to deliver data solutions.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, backlog refinement, reviews, and retrospectives.
• Create and maintain technical documentation covering data flows, mappings, processes, dependencies, and troubleshooting procedures.
• Participate in code reviews and follow established development, security, and deployment standards.
• Support production deployments, releases, and operational activities as required.

Skills

ETL/ELT

SQL Server

T-SQL

Azure Data Factory

Azure data platform

Data modelling

Agile/Scrum

CI/CD

Azure DevOps

Git/Bitbucket

Tools

Azure Data Factory

Azure DevOps

Git

Bitbucket

SQL Server Tools

Job description

Job Description / Duties and Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory and related Azure services.
• Develop complex and optimized T-SQL queries, stored procedures, views, functions, and database objects in SQL Server.
• Perform SQL Server database development and administration activities, including performance tuning, troubleshooting, and data integrity management.
• Build and maintain data integration pipelines connecting multiple source and target systems.
• Work with Azure data platform services to develop and support cloud-based data solutions.
• Analyze source systems and changing data structures and adapt data pipelines accordingly.
• Apply light-to-medium data modeling techniques to support analytical and operational data requirements.
• Work with large volumes of structured and semi-structured data and ensure efficient data processing.
• Troubleshoot data pipeline failures, data quality issues, performance problems, and production incidents.
• Monitor data pipelines and proactively identify and resolve issues.
• Collaborate with developers, architects, QA teams, business analysts, and other stakeholders to deliver data solutions.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, backlog refinement, reviews, and retrospectives.
• Create and maintain technical documentation covering data flows, mappings, processes, dependencies, and troubleshooting procedures.
• Participate in code reviews and follow established development, security, and deployment standards.
• Support production deployments, releases, and operational activities as required.

Job Specification / Skills and Competencies

• 7+ years of experience in Data Engineering, ETL, data integration, or related roles.
• Strong hands-on experience with Microsoft SQL Server development and administration.
• Advanced knowledge of T-SQL, including:
  • Complex queries
  • Stored procedures
  • Views and functions
  • Query optimization
  • Indexing and performance tuning
  • Error handling and troubleshooting
• Strong experience with Azure Data Factory (ADF) for ETL/ELT and data integration.
• Practical experience working with Azure data platform services.
• Good understanding of data warehousing and data integration concepts.
• Experience with data modelling, including dimensional modelling and logical/physical data models.
• Ability to understand and work with evolving or changing source data structures.
• Strong troubleshooting and problem-solving skills.
• Strong technical documentation and communication skills.
• Experience working in an Agile/Scrum development environment.
• Ability to work independently while collaborating effectively with distributed/onshore-offshore teams.
• Experience with CI/CD and source control tools such as Azure DevOps, Git, or Bitbucket.
• To adhere to the Information Security Management policies and procedures.