Data Engineer - Snowflakes/AWS

Merck Healthcare; Bengaluru; Vor Ort; INR 1.500.000 - 2.700.000; Vollzeit

Zusammenfassung

Merck Healthcare is seeking a data engineer to design and operate scalable data pipelines in a cloud data warehousing environment. You will work with Snowflake, dbt, and Python to model data, optimize queries, and implement robust governance and security controls.

You will collaborate across teams to deliver end-to-end data ingestion, transformation, and analytics, leveraging AWS/Azure environments and Airflow for orchestration.

Qualifikationen

• 3–5+ years of hands-on data engineering or cloud data warehousing experience.
• Strong SQL and Python (Pandas, NumPy).
• Experience with dbt models, tests, and macros.
• Experience with Snowflake features and architecture (warehouses, time travel, streams).

Aufgaben

• Design, build, and deploy scalable data ingestion and transformation workflows.
• Write, tune, and troubleshoot SQL queries, stored procedures, views, and Snowflake warehouses for performance and cost efficiency.
• Design dimensional models, star/snowflake schemas, and SCDs.
• Implement Git/GitHub-based DevOps, CI/CD pipelines, and dbt integrations.
• Ensure RBAC, data masking, encryption, and governance practices.
• Backend data services in Python, REST API integration, and data schema validation.
• Collaborate on Palantir Foundry and cloud data provisioning workflows as part of data platforms.

Kenntnisse

SQL

Python

dbt

Snowflake

Airflow

Tools

Snowflake

dbt

AWS

Azure

Airflow

Jobbeschreibung

Work Your Magic with us!

Ready to explore, break barriers, and discover more? We know you’ve got big plans – so do we! Our colleagues across the globe love innovating with science and technology to enrich people’s lives with our solutions in Healthcare, Life Science, and Electronics. Together, we dream big and are passionate about caring for our rich mix of people, customers, patients, and planet. That`s why we are always looking for curious minds that see themselves imagining the unimageable with us.

Your Role

• Data Pipeline & Architecture: Design, build, and deploy scalable data ingestion and transformation workflows.
• SQL & Performance Optimization: Write, tune, and troubleshoot complex SQL queries, stored procedures, views, and Snowflake virtual warehouses for speed and cost efficiency.
• Data Modeling: Design dimensional models, star/snowflake schemas, fact/dimension tables, and Slowly Changing Dimensions (SCD).
• DevOps & CI/CD: Implement version control (Git/GitHub), CI/CD pipelines, state-based deployments, and dbt (data build tool) integrations.
• Security & Governance: Ensure role-based access control (RBAC), data masking, encryption, and compliance best practices are applied.
• Python: Backend service development, REST API integration, retry and error-handling mechanisms, data schema modelling and validation with Pydantic SQL and Snowflake: MERGE- and stream-based CDC pipelines, Tasks, stored procedures, Secure Views, Snowpipe and Snowpipe Streaming, Secure Data Sharing, RBAC, role and grant management, and warehouse performance tuning Terraform: Infrastructure as code for Snowflake databases, roles, grants, warehouses, and repeatable multi-environment deployments
• AWS: Lambda-based event-driven ingestion, Amazon S3 data lake storage, AWS Glue Data Catalog, and IAM-based access controls Palantir Foundry: Workshop-based GUI development and backend workflow orchestration for data-request, approval, and provisioning processes Version Control and CI Engineering: Git-based development workflows, pre-commit hooks, Ruff-based linting and formatting, pull-request templates, review gates, and structured AI-assisted development workflows Data Modelling and Interchange Formats: JSON-based configuration and request schemas, and synchronisation of hierarchical and relational data models across systems
• Platform and Domain Expertise Enterprise data provisioning: End-to-end ownership of data-request portals using Palantir Foundry Workshop, Foundry workflow orchestration, and Python backend services targeting Snowflake and AWS Data virtualisation and analytics integration: Output-port configuration, ownership and privilege logic, and hierarchical data-model synchronisation between source systems and downstream platforms Industrial and OT data integration: Event-driven ingestion of Aveva PI process-historian data using Lambda, Snowpipe, and Snowflake, with downstream delivery through Secure Data Shares Multi-stage approval workflows: Role-based approval lifecycles involving requesters, data stewards, data owners, engineering teams, and business verifiers Data access governance: Request and approval workflows, role-based access control, and self-service provisioning for business data owners Regulated-environment delivery: Dev-to-QA-to-production promotion workflows supporting validated and regulated, GxP-equivalent system requires

Who You Are

• Experience: Typically 3 to 5+ years of hands-on experience in data engineering or cloud data warehousing.
• Core Technologies:
  • Snowflake Features: Virtual warehouses, clustering, micro-partitions, Time Travel, Streams & Tasks, Snowpipe, Dynamic Tables, and Snowflake Cortex (where applicable).
  • Programming & Scripting: Expert-level SQL and strong Python (Pandas, NumPy).
  • Transformation Tools: dbt (models, snapshots, tests, macros, Slim CI).
  • Cloud & Orchestration: Exposure to AWS, Azure, or GCP, plus tools like Apache Airflow.
• Certifications: Preferred credentials often include SnowPro Core or advanced Snowflake certifications.

What we offer:

We are curious minds that come from a broad range of backgrounds, perspectives, and life experiences. We celebrate all dimensions of diversity and believe that it drives excellence and innovation, strengthening our ability to lead in science and technology. We are committed to creating access and opportunities for all to develop and grow at your own pace. Join us in building a culture of inclusion and belonging that impacts millions and empowers everyone to work their magic and champion human progress!