Data Engineer

Merck Group; Bengaluru; Presencial; INR 1 500 000 - 2 700 000; Tempo integral

Resumo da oferta

Merck Group in Bengaluru is seeking a data engineer to design and implement scalable data ingestion, transformation, and analytics-ready data pipelines. You will optimize SQL performance, model data architectures, and integrate dbt with CI/CD workflows.

The role emphasizes secure data access, governance, and collaboration across cloud environments (AWS/Azure/GCP) and Snowflake-powered platforms. 3–5+ years of hands-on data warehousing experience are expected.

Qualificações

• 3 to 5+ years of hands-on experience in data engineering or cloud data warehousing.
• Strong SQL and Python (Pandas, NumPy) skills.
• Experience with Snowflake features and dbt.
• Familiarity with AWS, Azure, or GCP and Apache Airflow.
• SnowPro Core or Snowflake certifications preferred.

Responsabilidades

• Design, build, and deploy scalable data ingestion and transformation workflows.
• Write, tune, and troubleshoot complex SQL queries, stored procedures, and views for performance.
• Design dimensional models and data schemas for analytics platforms.
• Implement Git-based DevOps and CI/CD pipelines, and dbt integrations.
• Ensure RBAC, data masking, encryption, and governance best practices are applied.

Conhecimentos

SQL

Python

dbt

Airflow

Ferramentas

Snowflake

Terraform

Palantir Foundry

AWS

Snowpipe

Airflow

Descrição da oferta de emprego

Work Your Magic with us!

Ready to explore, break barriers, and discover more? We know you’ve got big plans – so do we! Our colleagues across the globe love innovating with science and technology to enrich people’s lives with our solutions in Healthcare, Life Science, and Electronics. Together, we dream big and are passionate about caring for our rich mix of people, customers, patients, and planet. That`s why we are always looking for curious minds that see themselves imagining the unimageable with us.

Your role:

• Data Pipeline & Architecture: Design, build, and deploy scalable data ingestion and transformation workflows.
• SQL & Performance Optimization: Write, tune, and troubleshoot complex SQL queries, stored procedures, views, and Snowflake virtual warehouses for speed and cost efficiency.
• Data Modeling: Design dimensional models, star/snowflake schemas, fact/dimension tables, and Slowly Changing Dimensions (SCD).
• DevOps & CI/CD: Implement version control (Git/GitHub), CI/CD pipelines, state-based deployments, and dbt (data build tool) integrations.
• Security & Governance: Ensure role-based access control (RBAC), data masking, encryption, and compliance best practices are applied.

• Python: Backend service development, REST API integration, retry and error-handling mechanisms, data schema modelling and validation with Pydantic
  SQL and Snowflake: MERGE- and stream-based CDC pipelines, Tasks, stored procedures, Secure Views, Snowpipe and Snowpipe Streaming, Secure Data Sharing, RBAC, role and grant management, and warehouse performance tuning
  Terraform: Infrastructure as code for Snowflake databases, roles, grants, warehouses, and repeatable multi-environment deployments
  
• AWS: Lambda-based event-driven ingestion, Amazon S3 data lake storage, AWS Glue Data Catalog, and IAM-based access controls
  Palantir Foundry: Workshop-based GUI development and backend workflow orchestration for data-request, approval, and provisioning processes
  Version Control and CI Engineering: Git-based development workflows, pre-commit hooks, Ruff-based linting and formatting, pull-request templates, review gates, and structured AI-assisted development workflows
  Data Modelling and Interchange Formats: JSON-based configuration and request schemas, and synchronisation of hierarchical and relational data models across systems
• Platform and Domain Expertise
  Enterprise data provisioning: End-to-end ownership of data-request portals using Palantir Foundry Workshop, Foundry workflow orchestration, and Python backend services targeting Snowflake and AWS
  Data virtualisation and analytics integration: Output-port configuration, ownership and privilege logic, and hierarchical data-model synchronisation between source systems and downstream platforms
  Industrial and OT data integration: Event-driven ingestion of Aveva PI process-historian data using Lambda, Snowpipe, and Snowflake, with downstream delivery through Secure Data Shares
  Multi-stage approval workflows: Role-based approval lifecycles involving requesters, data stewards, data owners, engineering teams, and business verifiers
  Data access governance: Request and approval workflows, role-based access control, and self-service provisioning for business data owners
  Regulated-environment delivery: Dev-to-QA-to-production promotion workflows supporting validated and regulated, GxP-equivalent system requirements

Who you are:

• Experience: Typically 3 to 5+ years of hands-on experience in data engineering or cloud data warehousing.
• Core Technologies:
  • Snowflake Features: Virtual warehouses, clustering, micro-partitions, Time Travel, Streams & Tasks, Snowpipe, Dynamic Tables, and Snowflake Cortex (where applicable).
  • Programming & Scripting: Expert-level SQL and strong Python (Pandas, NumPy).
  • Transformation Tools: dbt (models, snapshots, tests, macros, Slim CI).
  • Cloud & Orchestration: Exposure to AWS, Azure, or GCP, plus tools like Apache Airflow.
• Certifications: Preferred credentials often include SnowPro Core or advanced Snowflake certifications.

What we offer:

We are curious minds that come from a broad range of backgrounds, perspectives, and life experiences. We celebrate all dimensions of diversity and believe that it drives excellence and innovation, strengthening our ability to lead in science and technology. We are committed to creating access and opportunities for all to develop and grow at your own pace. Join us in building a culture of inclusion and belonging that impacts millions and empowers everyone to work their magic and champion human progress!