Data Engineer

Capgemini; Bengaluru; Hybrid; INR 1,200,000 - 1,800,000; Full time

Benefits offered by this job

Remote work option

Flexible hours

Career growth programs

Generative AI certifications

Job summary

Capgemini Invent is seeking a data engineering professional to design and build scalable data pipelines using Snowflake, Python, and PySpark in Bengaluru. You will optimize SQL, implement Snowpipe-based ingestion, and manage data across base tables with Streams and Tasks.

You will model data with DBT, enforce data quality checks, and collaborate with cross-functional teams to translate requirements into robust solutions. Strong DevOps and CI/CD practices are essential.

Qualifications

• Proficiency in Python and PySpark, SQL, including query optimization and performance tuning.
• Snowflake/AWS/Azure/GCP exposure.
• Familiarity with data warehousing concepts and ETL; experience with big data tools: Hadoop, Spark, Kafka.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Snowflake. Write efficient and maintainable code in Python and PySpark. Develop and optimize SQL queries for data extraction and transformation. Implement data ingestion strategies in Snowflake using Snowpipe and COPY to load data into base tables with Streams and Tasks.
• Extract data from various source systems and load data into Snowflake using Python, PySpark scripts. Configure role-based access on Snowflake objects and process data sharing and cloning between Snowflake accounts.
• Create models in DBT and integrate data quality rules and tests to ensure data accuracy, consistency, and completeness. Implement data quality checks and monitoring, ensuring data accuracy and reliability. Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Ensure data quality, governance, and security across all pipelines and storage layers. Hands‑on experience with CI/CD tools and DevOps practices.

Skills

Python

PySpark

SQL

Snowflake

AWS

Azure

GCP

Data warehousing

ETL

Hadoop

Spark

Kafka

MongoDB

Cassandra

DBT

CI/CD

Git

Tools

DBT

Snowflake

Git

CI/CD

Snowpipe

Job description

At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose.

Your role

• Design, develop, and maintain scalable data pipelines using Snowflake. Write efficient and maintainable code in Python and PySpark. Develop and optimize SQL queries for data extraction and transformation. Implement data ingestion strategies in Snowflake using Snowpipe and COPY command to load data into base tables with Streams and Tasks.
• Extract data from various source systems and load data into Snowflake using Python, Pyspark scripts. Configure role-based access on Snowflake objects and process data sharing and cloning between Snowflake accounts.
• Create models in DBT and integrate data quality rules and tests to ensure data accuracy, consistency, and completeness. Implement data quality checks and monitoring, ensuring data accuracy and reliability. Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Ensure data quality, governance, and security across all pipelines and storage layers. Hands‑on experience with CI/CD tools and DevOps practices.

Your Profile

• Proficiency in Python and PySpark, SQL skills, including query optimization and performance tuning.
• Snowflake/ AWS/ Azure/ GCP.
• Familiarity with data warehousing concepts and ETL processes. Experience with big data tools: Hadoop, Spark, Kafka, etc.
• Experience with relational databases such as Microsoft SQL Server, MySQL, PostgreSQL, Oracle, and NoSQL databases such as Hadoop, Cassandra, MongoDB. Excellent problem‑solving skills with an emphasis on sustainable and reusable development.

What You Will Love About Working Here

• We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance.
• At the heart of our mission is your career growth. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities.
• Equip yourself with valuable certifications in the latest technologies such as Generative AI.

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.