AWS Data Engineer: Scalable Cloud Data Pipelines

Quantum Technologies LLC; Dallas (TX); Vor Ort; USD 77.000 - 110.000; Vollzeit

Zusammenfassung

Quantum Technologies. LLC seeks an AWS Data Engineer to design, build, and maintain scalable data pipelines in the AWS cloud. You will collaborate with data scientists and analysts to deliver robust data solutions and optimize data ingestion, storage, and processing across data lakes and warehouses.

The role emphasizes serverless architectures, CI/CD for data platforms, and staying current with new tools and technologies. 7-month engagement in Dallas, TX with a competitive hourly rate of $68.

Qualifikationen

• Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent
• 5 years of experience implementing and supporting data lakes, data warehouses, and data applications on AWS for large enterprises
• Strong programming experience with Python, Shell scripting, and SQL
• Solid experience with AWS services: Glue, Redshift, S3, Lambda, EMR, Spark, Kinesis, and SQS; plus IAM, KMS, Secrets Manager, and Step Functions
• Experience in serverless application development and data pipeline orchestration
• Knowledge of ETL/ELT, data modeling and cloud-based data warehousing concepts
• Familiarity with data warehousing concepts and cloud-based architecture
• Strong problem-solving skills and teamwork ability
• Excellent communication and collaboration with stakeholders

Aufgaben

• Design, build, and maintain scalable data pipelines and architectures on AWS
• Create robust ingestion pipelines from on-premises to AWS and within AWS
• Develop and manage ETL/ELT processes to collect, process, and store data
• Architect end-to-end data solutions emphasizing data lakes and data warehouses
• Contribute to architecture discussions for high-scale data projects
• Perform hands-on development, unit testing, and code reviews
• Implement serverless applications using AWS Lambda, API Gateway, Step Functions
• Migrate data to AWS-based data lakes, RDS Aurora, and Redshift
• Implement high-velocity streaming with Kinesis and Kafka where applicable
• Develop and implement CI/CD strategies for data platforms
• Collaborate with cross-functional teams throughout the SDLC
• Identify and evaluate new tools and perform POCs for real-world use cases
• Monitor and optimize performance, cost, reliability, and scalability

Kenntnisse

Python

Shell scripting

SQL

AWS

Data pipelines

Ausbildung

Bachelor's degree in Computer Science / related field

Tools

Glue

Redshift

S3

Lambda

EMR

Spark

Kinesis

SQS

Kafka

CloudFormation

IAM

KMS

Secrets Manager

Step Functions

Jobbeschreibung

Quantum Technologies. LLC seeks an AWS Data Engineer to design, build, and maintain scalable data pipelines in the AWS cloud. You will collaborate with data scientists and analysts to deliver robust data solutions and optimize data ingestion, storage, and processing across data lakes and warehouses.

The role emphasizes serverless architectures, CI/CD for data platforms, and staying current with new tools and technologies. 7-month engagement in Dallas, TX with a competitive hourly rate of $68.