Data Engineer

Synechron; Bengaluru; In loco; INR 1.800.000 - 3.200.000; Tempo pieno

Descrizione del lavoro

Synechron is seeking a Senior Data Engineer in Bengaluru to analyze, design and implement enterprise-scale data pipelines. You will dissect Omnia-based Spark pipelines, translate business rules into scalable cloud solutions, and work with PySpark, Python and SQL to deliver robust data processing.

The role requires hands-on AWS experience, Snowflake modelling, dbt development, and Airflow orchestration. Collaboration with data designers and stakeholders is essential.

Competenze

• Strong hands-on experience designing, developing and supporting scalable data pipelines.
• Deep knowledge of Spark ETL, Omnia pipelines, and translating logic into scalable cloud solutions.
• Proficient with AWS services (S3, Glue, EMR, Lambda, Athena, Step Functions, SQS, IAM, Secrets Manager).
• Hands-on with Airflow for orchestration and end-to-end pipeline development.
• Snowflake data ingestion, modelling, transformation and performance tuning.
• dbt for modular data models, tests, documentation and dependencies.
• Excellent SQL for complex transformations, reconciliation and business logic understanding.
• Git-based version control, CI/CD with Jenkins and DevSecOps practices in data eng.
• Proficiency in Python for data processing and automation.
• Experience with Omnia or CBA data platforms is highly desirable.

Mansioni

• Analyze existing end-to-end data pipelines across platforms, focusing on Omnia PySpark processing.
• Review source code, SQL, mappings and data flows to interpret current logic.
• Derive and implement new transformation logic from analysis and business needs.
• Design and maintain reliable, scalable data pipelines using AWS, Snowflake and dbt.
• Develop batch and event-driven data processing solutions using cloud-native tech.
• Translate Omnia processing logic into modern cloud-based data architectures.
• Build ingestion, transformation and enrichment pipelines across lakes and warehouses.
• Write complex, optimized SQL for transformation, reconciliation and analytics.
• Identify and fix data quality issues and performance bottlenecks across environments.
• Collaborate with data designers and product owners to enforce data rules and durable engineering.

Conoscenze

AWS

Snowflake

dbt

Airflow

PySpark

Python

SQL

Git

CI/CD

Jenkins

DevSecOps

Omnia

CBA data

Strumenti

Snowflake

dbt

Airflow

Spark

Git

Jenkins

Descrizione del lavoro

## Data EngineerApply: Bengaluru - Thanissandra (BCIT): Full time: Posted Today: JR1045473**Role: Senior Data Engineer****Role Overview :**Experienced Data Engineer to analyze, design and develop enterprise-scale data pipelines across legacy and strategic cloud data platforms.A key component of the position is analyzing existing data pipelines, particularly Omnia-based Spark pipelines, understanding the underlying business and transformation logic, and translating that logic into scalable data solutions on modern cloud platforms. The role requires strong hands-on experience with AWS, Snowflake, dbt, Airflow, PySpark, Python and advanced SQL. **Technical Expertise :*** Strong hands-on experience designing, developing and supporting scalable enterprise data pipelines.* Strong Spark-etl experience, including the ability to analyze existing Omnia pipelines, understand their processing logic and derive new transformation and business rules.* Strong working knowledge of AWS services, including S3, Glue, EMR, Lambda, Athena, Step Functions, SQS, IAM, Secrets Manager and CloudWatch, combined with hands-on experience in orchestrating data pipelines using Apache Airflow.* Practical experience with Snowflake, covering data ingestion, data modelling, transformation and performance optimization.* Experience using dbt to develop data models, incremental processing, reusable macros, automated tests, documentation and dependency management.* Excellent SQL skills, with experience in complex data transformations, query optimization, reconciliation and understanding existing business logic.* Familiarity with Git, automated testing, CI/CD, Jenkins and DevSecOps practices within a data engineering environment.* Good Python programming skills for data processing, automation, and reusable framework components.* Previous CBA experience, particularly with Omnia or CBA data platforms and engineering standards, is highly desirable. **Key Responsibilities*** Conduct detailed analysis of existing end-to-end data pipelines across multiple platforms, with a particular focus on Omnia - PySpark-based processing.* Review existing source code, SQL, transformation rules, mappings and data flows to understand the current business and technical logic.* Derive, document, and implement new transformation logic based on the analysis of existing pipelines and business requirements.* Design, develop and maintain reliable, scalable, and reusable data pipelines using AWS, Snowflake and dbt.* Develop batch and, where required, event-driven data processing solutions using cloud-native and distributed data technologies.* Translate existing Omnia data processing logic into strategic cloud-based data engineering solutions.* Build ingestion, transformation, enrichment and data integration pipelines across data lakes, data warehouses and cloud lakehouse platforms.* Develop complex and performance-optimized SQL for data transformation, reconciliation, validation and analytical processing.* Analyze end-to-end data flows from source capture through transformation, enrichment and downstream consumption.* Work with data designers, product owners, and subject matter experts to translate business requirements into technical data rules and durable engineering solutions.* Implement data quality, reconciliation and balancing controls across data pipelines.* Troubleshoot data pipeline failures, transformation issues, data discrepancies and performance bottlenecks across multiple environments. ***S******YNECHRON’S DIVERSITY & INCLUSION STATEMENT***Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.