AWS Data Engineer

Capgemini; Newark (NJ); On-site; USD 120,000 - 160,000; Full time

Job summary

Capgemini Newark, NJ seeks a data engineer with strong AWS data lake and data warehouse experience to design and implement end-to-end data solutions. You will work with Python, SQL and shell scripting to ingest, transform, and load large enterprise datasets using S3, Glue, Athena, Redshift, and EMR.

The role requires CI/CD for data pipelines, streaming knowledge (Kinesis, Kafka), and data migrations from APIs and on-prem systems to AWS.

Qualifications

• Bachelor's degree in CS/SE or MIS, or equivalent.
• Experience with AWS data lakes and data warehouses on large enterprises.
• Proficient in Python, Shell scripting and SQL.
• Strong knowledge of AWS services including CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS.

Skills

Python

Shell scripting

SQL

AWS data lakes

AWS data warehouses

CloudFormation

S3

Athena

Glue

EMR/Spark

RDS

Redshift

DynamoDB

Lambda

Step Functions

IAM

KMS

Kinesis

Kafka

APIs data migration

Elasticsearch

Education

Bachelor's degree in Computer Science / Software Engineering / MIS

Job description

• Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and experience.
• Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises.
• Programming experience with Python, Shell scripting and SQL.
• Solid experience of AWS services such as CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
• Solid experience implementing solutions on AWS based data lakes.
• Should have good experience with AWS Services - API Gateway, Lambda, Step Functions, SQS, DynamoDB, S3, Elasticsearch.
• Experience in AWS data lake/data warehouse/business analytics.
• Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS.
• Knowledge of ETL/ELT.
• End-to-end data solutions (ingest, storage, integration, processing, access) on AWS.
• Architect and implement CI/CD strategy for EDP.
• Implement high velocity streaming solutions using Amazon Kinesis, SQS, and Kafka (preferred).
• Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift.
• Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift.
• Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-case.
• AWS Solutions Architect or AWS Developer Certification preferred.
• Good understanding of Lakehouse/data cloud architecture