AWS Data Engineer

Select Minds LLC; Dallas (TX); Hybrid; Full time

Benefits offered by this job

Hybrid work options

Long Term

Opportunity for advancement

Job summary

A leading tech consulting firm based in Dallas is seeking a Senior Data Engineer to migrate data to AWS Redshift and build robust data pipelines. The ideal candidate will have strong Python skills and AWS experience, with opportunities for hybrid work arrangements. This long-term position offers a chance for advancement in a dynamic environment.

Qualifications

• Strong experience in Python and cloud technologies.
• Good understanding of Kafka, ETL, and data warehouses.
• Familiarity with CI/CD, boto3, and Kubernetes is a big plus.

Responsibilities

• Migrate data from On-prem to AWS Redshift with zero data loss.
• Build batch and real-time data pipelines using Python.
• Set up and manage Kafka on AWS for streaming data.
• Automate ETL workflows using AWS Glue and store data in S3.
• Design data solutions in Redshift for fast querying.
• Create reusable frameworks to reduce development time.
• Build dashboards and reports for client data insights.
• Use GitLab CI/CD for testing and deployments.
• Deploy containerized apps using Kubernetes.
• Follow best practices to ensure data security.

Skills

Python

Cloud technologies - AWS

Kafka

CI/CD

Kubernetes

Tools

AWS Redshift

AWS Glue

GitLab CI/CD

Job description

Benefits

• Hybrid
• Long Term
• Opportunity for advancement

Overview

Senior Data Engineer Location: Dallas, TX (Hybrid—3 days onsite). Duration: Long-term Project. Interview: In-person.

Responsibilities

• Migrate data from On-prem to AWS Redshift with zero data loss.
• Build batch and real-time data pipelines using Python.
• Set up and manage Kafka on AWS for streaming data.
• Automate ETL workflows using AWS Glue and store data in S3.
• Design data solutions in Redshift for fast querying.
• Create reusable frameworks to reduce development time and improve code.
• Build dashboards and reports for client data insights.
• Use GitLab CI/CD for testing and deployments.
• Work with AWS tools like Lambda, SQS, SNS, Glue, DynamoDB, OpenSearch, and more.
• Develop tools to generate mock data (JSON, CSV, XML).
• Deploy containerized apps using Kubernetes.
• Follow best practices to ensure data security and compliance.

Qualifications

• Strong experience in Python and cloud technologies - AWS.
• Good understanding of Kafka, ETL, and data warehouses.
• Familiarity with CI/CD, boto3, and Kubernetes is a big plus.
• Flexible work from home options available.