Senior Data Engineer

UST; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

UST in Bengaluru, India is seeking a Senior Data Engineer to design, develop, and optimize scalable data pipelines and cloud data warehouses. You will own end-to-end delivery on Snowflake, SQL, Python, PySpark, dbt, and CI/CD.

You will implement data quality, observability, and modernization of legacy ETL while mentoring teammates and coordinating across cross-functional teams. The role demands strong technical ownership and collaboration with data governance and security teams.

Qualifications

• 6-8 years of professional Data Engineering experience.
• Experience designing enterprise data pipelines and data warehouses in production.
• Strong SQL, Python, PySpark with hands-on Snowflake.
• Experience with dbt, CI/CD, and data quality processes.
• Excellent collaboration and communication skills.

Responsibilities

• Design, develop, and maintain scalable batch and near-real-time ETL/ELT data pipelines using SQL, Python, and PySpark, with Snowflake as the primary cloud data platform.
• Build efficient and reusable dimensional, normalized, and analytical data models for enterprise reporting and analytics.
• Develop and optimize Snowflake data warehouse solutions, including query performance, workload management, security, scalability, and cost optimization.
• Implement automated data quality and validation frameworks covering profiling, schema validation, completeness, freshness, uniqueness, reconciliation, and business-rule checks.
• Develop and maintain dbt-based transformation workflows, including models, tests, documentation, and reusable transformation logic.
• Build and maintain CI/CD pipelines for data engineering workloads using Git-based version control, peer reviews, automated testing, deployment, environment promotion, and rollback mechanisms.
• Monitor pipeline health, data lineage, performance, costs, and service-level objectives; troubleshoot production issues and perform root-cause analysis.
• Integrate data from diverse sources, including databases, REST APIs, files, and event/streaming sources, while following secure data-access practices.
• Modernize and migrate legacy Informatica PowerCenter and other ETL workflows to cloud-native data engineering solutions using tools such as dbt and modern orchestration platforms.
• Perform data profiling, cleansing, standardization, reconciliation, and production data validation.
• Translate ambiguous business and technical requirements into scalable data engineering solutions and communicate design decisions, risks, and trade-offs effectively.
• Review technical designs, code, and pull requests while establishing engineering standards for coding, testing, and deployment.
• Mentor junior and mid-level engineers and coordinate delivery across cross-functional teams.

Skills

Snowflake

SQL

Python

PySpark

dbt

ETL/ELT

Data Modeling

Git-based CI/CD

Data Quality

Education

Bachelor’s or Master’s in Computer Science/Engineering/Mathematics

Tools

Informatica PowerCenter

Snowflake

Job description

Mandatory Skills:

Python, PySpark, ETL, Snowflake, SQL

Role Overview

We are looking for a Senior Data Engineer with strong hands-on expertise in Snowflake, SQL, Python, PySpark, dbt, ETL/ELT, Data Modeling, and Git-based CI/CD. The candidate will be responsible for designing, developing, optimizing, and supporting scalable enterprise data pipelines and cloud data warehouse solutions.

The role requires strong technical ownership across data engineering, data quality, performance optimization, automation, observability, and modernizing legacy ETL workflows.

Key Responsibilities

• Design, develop, and maintain scalable batch and near-real-time ETL/ELT data pipelines using SQL, Python, and PySpark, with Snowflake as the primary cloud data platform.
• Build efficient and reusable dimensional, normalized, and analytical data models for enterprise reporting and analytics.
• Develop and optimize Snowflake data warehouse solutions, including query performance, workload management, security, scalability, and cost optimization.
• Implement automated data quality and validation frameworks covering profiling, schema validation, completeness, freshness, uniqueness, reconciliation, and business-rule checks.
• Develop and maintain dbt-based transformation workflows, including models, tests, documentation, and reusable transformation logic.
• Build and maintain CI/CD pipelines for data engineering workloads using Git-based version control, peer reviews, automated testing, deployment, environment promotion, and rollback mechanisms.
• Monitor pipeline health, data lineage, performance, costs, and service-level objectives; troubleshoot production issues and perform root-cause analysis.
• Integrate data from diverse sources, including databases, REST APIs, files, and event/streaming sources, while following secure data-access practices.
• Modernize and migrate legacy Informatica PowerCenter and other ETL workflows to cloud-native data engineering solutions using tools such as dbt and modern orchestration platforms.
• Perform data profiling, cleansing, standardization, reconciliation, and production data validation.
• Translate ambiguous business and technical requirements into scalable data engineering solutions and communicate design decisions, risks, and trade-offs effectively.
• Review technical designs, code, and pull requests while establishing engineering standards for coding, testing, and deployment.
• Mentor junior and mid-level engineers and coordinate delivery across cross-functional teams.

Preferred Skills

• Snowflake SnowPro Core or equivalent cloud data certification.
• Experience with REST APIs, event-driven or streaming data.
• Experience working with high-volume security, operational, or enterprise datasets.
• Knowledge of data quality and observability frameworks such as dbt Tests, Great Expectations, Soda, or Deequ.
• Experience with Informatica PowerCenter or comparable ETL/integration tools.
• Experience in regulated, security-sensitive, healthcare, telecommunications, or audit-oriented environments.
• Experience with modern data orchestration and cloud-native data engineering practices.
• Demonstrated experience in technical leadership, mentoring, or delivery coordination.

Required Qualifications

• 6-8 years of professional experience in Data Engineering.
• Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics, or a related discipline, or equivalent practical experience.
• Strong hands-on experience designing and operating enterprise data pipelines and data warehouses in production.
• Advanced SQL skills with strong practical experience in Python and/or PySpark for data processing and automation.
• Deep hands-on experience with Snowflake, including performance optimization, security, workload management, and scalable data modeling.
• Experience developing ETL/ELT solutions using dbt, Informatica PowerCenter, or comparable tools.
• Strong understanding of data quality, data validation, data profiling, and production data operations.
• Experience with Git, CI/CD, automated testing, and deployment for data engineering workloads.
• Strong analytical, problem-solving, communication, and stakeholder-management skills.