AWS Data Engineer

Spectraforce Technologies; Newark (NJ); Hybrid; USD 120,000 - 170,000; Full time

Job summary

Spectraforce Technologies in Newark, NJ is seeking an AWS Data Engineer for a 7-month contract. Hybrid role requiring strong background designing and implementing data ingestion pipelines, data lakes and data warehouses on AWS.

You will work across teams to deliver scalable, reliable data solutions. The ideal candidate has Bachelor's degree and hands-on experience with AWS services, serverless development, and CI/CD for data platforms, with emphasis on streaming and migration to

Qualifications

• Bachelors degree in CS/SE/MIS or equivalent experience.
• Experience building data lakes, warehouses and apps on AWS for large enterprises.
• Proficient in Python, Shell, and SQL.
• Hands-on with AWS services including CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS.
• Experience delivering AWS data lake solutions.
• Familiar with API Gateway, Lambda, SQS, DynamoDB, Elasticsearch.
• Serverless development with AWS Lambda.
• Experience in ETL/ELT and end-to-end data solutions on AWS.

Responsibilities

• Design, build, and maintain efficient data architecture and code.
• Build reliable data ingestion pipelines within AWS and from on-prem to AWS.
• Ensure performance and quality on high-scale data projects.
• Participate in architecture and system design discussions.
• Hands-on development and unit testing of applications.
• Collaborate with teams to integrate components into enterprise web systems.
• Work with cross-functional teams across the SDLC to deliver projects.
• Identify and resolve performance issues; stay updated with new tech.
• Participate in code reviews to enforce standards.

Tools

CloudFormation

S3

Athena

Glue

EMR/Spark

RDS

Redshift

DynamoDB

Lambda

Step Functions

IAM

KMS

Secrets Manager

API Gateway

SQS

Elasticsearch

Kafka

Kinesis

Aurora

Job description

Job Title: AWS Data Engineer

Duration: 7 months

Location: Hybrid (Newark, NJ)

Qualifications

• Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and experience
• Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises
• Programming experience with Python, Shell scripting and SQL
• Solid experience of AWS services such as CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
• Solid experience implementing solutions on AWS based data lakes
• Good experience with AWS Services - API Gateway, Lambda, Step Functions, SQS, DynamoDB, S3, Elasticsearch
• Serverless application development using AWS Lambda
• Experience in AWS data lake/data warehouse/business analytics
• Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS
• Knowledge of ETL/ELT
• End-to-end data solutions (ingest, storage, integration, processing, access) on AWS
• Architect and implement CI/CD strategy for EDP
• Implement high velocity streaming solutions using Amazon Kinesis, SQS, and Kafka (preferred)
• Migrate data from traditional relational database systems, file systems, NAS shares to AWS relational databases such as Amazon RDS, Aurora, and Redshift
• Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift; Implement POCs on any new technology or tools to be implemented on EDP and onboard for real use-case
• AWS Solutions Architect or AWS Developer Certification preferred
• Good understanding of Lakehouse/data cloud architecture

Responsibilities

• Designing, building and maintaining efficient, reusable, and reliable architecture and code.
• Build reliable and robust Data ingestion pipelines (within AWS, onprem to AWS, etc.)
• Ensure the best possible performance and quality of high scale data engineering project
• Participate in the architecture and system design discussions
• Independently perform hands on development and unit testing of the applications.
• Collaborate with the development team and build individual components into complex enterprise web systems.
• Work in a team environment with product, production operation, QE/QA and cross functional teams to deliver a project throughout the whole software development cycle.
• Responsible to identify and resolve any performance issues
• Keep up to date with new technology development and implementation
• Participate in code review to make sure standards and best practices are met.