Lead Data Engineer

UST; Bengaluru; On-site; INR 2,500,000 - 4,200,000; Full time

Job summary

UST Bengaluru is seeking a Senior Data Engineer to design and deliver scalable batch and near-real-time data pipelines on Snowflake, using SQL, Python and PySpark. You will architect data models and own data quality and production readiness.

You will build CI/CD for data products, improve observability, integrate diverse data sources, and guide the team on standards and best practices while communicating with both technical and business partners.

Qualifications

• Bachelor's or master's degree in computer science, engineering, mathematics or a related discipline, or equivalent practical experience.
• Substantial professional experience designing and operating enterprise data pipelines and data warehouses in production.
• Advanced SQL skills and strong hands-on experience with Python and/or PySpark for data processing and automation.
• Deep experience with Snowflake, including performance optimization, security, workload management and scalable data modeling.
• Experience building ETL/ELT solutions with dbt, Informatica PowerCenter or comparable transformation and integration tools.
• Hands-on experience implementing CI/CD pipelines for data engineering workloads using Git-based workflows and automated testing/deployment tools.
• Experience building automated data quality systems with measurable rules, ing, lineage and issue-resolution workflows.
• Strong grounding in data profiling, cleansing, standardization, reconciliation and production data validation.
• Ability to gather ambiguous requirements, make sound design decisions and communicate effectively with technical and non-technical partners.

Responsibilities

• Design and deliver scalable pipelines. Build batch and near-real-time ETL/ELT pipelines using SQL, Python and PySpark, with Snowflake as a primary cloud data platform.
• Engineer reliable data models. Create dimensional, normalized and analytical models that support efficient querying, traceability and reuse across enterprise reporting and analytics.
• Own data quality by design. Implement automated profiling, schema checks, reconciliation, completeness, freshness, uniqueness and business-rule validation; publish quality results and route failures for action.
• Build CI/CD for data products. Create and maintain automated build, test and deployment pipelines for SQL, dbt, orchestration code and infrastructure configuration, including version control, peer review, environment promotion and safe rollback.
• Strengthen observability and operations. Monitor pipeline health, lineage, cost, performance and service-level objectives; troubleshoot incidents, perform root-cause analysis and eliminate recurring manual support.
• Integrate diverse sources. Ingest and standardize data from databases, APIs, files and event sources, applying secure access patterns and appropriate handling for sensitive information.
• Modernize legacy workflows. Migrate or refactor Informatica PowerCenter and other legacy ETL processes into maintainable cloud-native patterns using tools such as dbt and modern orchestration platforms.
• Partner from requirement to adoption. Translate stakeholder needs into technical designs, validate production outcomes and communicate tradeoffs, risks and delivery progress clearly.
• Lead through engineering practice. Set coding and testing standards, review designs and pull requests, mentor engineers and coordinate delivery across cross-functional teams.

Skills

Snowflake

SQL

Python

PySpark

dbt

ETL/ELT

Data modeling

Git

CI/CD for data

Data quality

Education

BTech / MTech

Tools

Informatica PowerCenter

dbt

Job description

Role Description

Job Code 15124 Locations Bengaluru Minimum Experience 6 Maximum Experience 8 Mandatory Skills Snowflake,SQL,Python,PySpark,dbt,ETL/ELT,data modeling,Git Skill to Evaluate Snowflake,SQL,Python,PySpark,dbt,ETL/ELT,data modeling,Git Experience 6 to 8 Years Location Bengaluru Job Description Required qualifications

• Bachelor's or master's degree in computer science, engineering, mathematics or a related discipline, or equivalent practical experience.
• Substantial professional experience designing and operating enterprise data pipelines and data warehouses in production.
• Advanced SQL skills and strong hands-on experience with Python and/or PySpark for data processing and automation.
• Deep experience with Snowflake, including performance optimization, security, workload management and scalable data modeling.
• Experience building ETL/ELT solutions with dbt, Informatica PowerCenter or comparable transformation and integration tools.
• Hands-on experience implementing CI/CD pipelines for data engineering workloads using Git-based workflows and automated testing/deployment tools.
• Experience building automated data quality systems with measurable rules, ing, lineage and issue-resolution workflows.
• Strong grounding in data profiling, cleansing, standardization, reconciliation and production data validation.
• Ability to gather ambiguous requirements, make sound design decisions and communicate effectively with technical and non-technical partners. Preferred qualifications
• Snowflake SnowPro Core or comparable cloud data certification.
• Experience with REST API, event and/or streaming data, and high-volume security or operational datasets.
• Familiarity with data observability and quality frameworks such as dbt tests, Great Expectations, Soda or Deequ.
• Experience delivering data solutions in regulated, security-sensitive, healthcare, telecommunications or audit-oriented environments.
• Demonstrated technical leadership, mentoring or delivery coordination across a small engineering team. Education Qualificaiton B Tech/ M Tech Job Title Senior Data Engineer (Senior Individual Contributor / Technical Lead)

Roles & Responsibilities

• Design and deliver scalable pipelines. Build batch and near-real-time ETL/ELT pipelines using SQL, Python and PySpark, with Snowflake as a primary cloud data platform.
• Engineer reliable data models. Create dimensional, normalized and analytical models that support efficient querying, traceability and reuse across enterprise reporting and analytics.
• Own data quality by design. Implement automated profiling, schema checks, reconciliation, completeness, freshness, uniqueness and business-rule validation; publish quality results and route failures for action.
• Build CI/CD for data products. Create and maintain automated build, test and deployment pipelines for SQL, dbt, orchestration code and infrastructure configuration, including version control, peer review, environment promotion and safe rollback.
• Strengthen observability and operations. Monitor pipeline health, lineage, cost, performance and service-level objectives; troubleshoot incidents, perform root-cause analysis and eliminate recurring manual support.
• Integrate diverse sources. Ingest and standardize data from databases, APIs, files and event sources, applying secure access patterns and appropriate handling for sensitive information.
• Modernize legacy workflows. Migrate or refactor Informatica PowerCenter and other legacy ETL processes into maintainable cloud-native patterns using tools such as dbt and modern orchestration platforms.
• Partner from requirement to adoption. Translate stakeholder needs into technical designs, validate production outcomes and communicate tradeoffs, risks and delivery progress clearly.
• Lead through engineering practice. Set coding and testing standards, review designs and pull requests, mentor engineers and coordinate delivery across cross-functional teams.