AWS Snowflake Engineer

Capgemini; Dadri, Hyderabad, Chennai District; Hybrid; INR 1,100,000 - 1,900,000; Full time

Job summary

Capgemini India is seeking a data engineer to design, develop, and optimize end-to-end data pipelines on AWS, leveraging Glue, Lambda, S3, Redshift, and Snowflake. You will build robust ingestion from databases, APIs, SharePoint, and files, and implement ETL/ELT workflows with a focus on performance and cost efficiency.

You will ensure data quality, implement monitoring and security best practices, and document deployment procedures.

Qualifications

• Design and implement data pipelines on AWS.
• Expertise in Amazon RDS and cloud databases.
• Design Snowflake data warehouse solutions.
• Collaborate with stakeholders to capture data requirements.
• Build data ingestion pipelines from databases, APIs, SharePoint, SFTP, Excel, and cloud apps.

Responsibilities

• Design, develop, and implement data pipelines on AWS using Glue, Data Pipeline, Lambda, Batch, S3, and Redshift.
• Demonstrate strong expertise in Amazon RDS and cloud-based database solutions.
• Design, implement, and optimize Snowflake data warehouse solutions, including data modeling, loading, transformation, and performance tuning.
• Collaborate with Data Engineers, Analysts, and stakeholders to understand data requirements and design scalable data solutions.
• Build data ingestion pipelines from various sources including databases, APIs, SharePoint, SFTP, Excel files, cloud applications.
• Develop and maintain ETL/ELT workflows using AWS and Snowflake.
• Optimize pipelines for performance, scalability, reliability, and cost efficiency.
• Ensure data quality and integrity via validation, cleansing, and monitoring.
• Monitor and troubleshoot data pipeline issues and implement fixes.
• Implement security best practices to ensure data privacy and governance.
• Document technical specs, deployment procedures, and operational guidelines.
• Stay updated on AWS, Snowflake, and modern data engineering practices and recommend innovative solutions.

Skills

AWS

Snowflake

ETL/ELT

Data modeling

Tools

AWS Glue

AWS Data Pipeline

AWS Lambda

Amazon S3

Amazon Redshift

Snowflake

Job description

1. Design, develop, and implement data pipelines on AWS using services such as AWS Glue, AWS Data Pipeline, AWS Lambda, AWS Batch, Amazon S3, Amazon Redshift, and related technologies.
2. Demonstrate strong expertise in Amazon RDS and cloud-based database solutions.
3. Design, implement, and optimize Snowflake data warehouse solutions, including data modeling, data loading, transformation, and performance tuning.
4. Collaborate with Data Engineers, Business Analysts, and business stakeholders to understand data requirements and design scalable data solutions.
5. Build data ingestion pipelines to collect data from various sources, including:

• Databases
• APIs
• SharePoint
• SFTP
• Excel files
• Cloud applications

• Develop and maintain ETL/ELT workflows using AWS services and Snowflake.
• Optimize and fine-tune data pipelines for performance, scalability, reliability, and cost efficiency.
• Ensure data quality and integrity throughout the data lifecycle by implementing validation, cleansing, and monitoring mechanisms.
• Monitor and troubleshoot data pipeline issues, identify bottlenecks, and implement corrective actions.
• Implement security best practices to ensure data privacy, compliance, and governance standards.
• Document technical specifications, deployment procedures, and operational guidelines.
• Stay updated on AWS, Snowflake, and modern data engineering best practices and recommend innovative solutions.