AWS Data Engineer — Big Data Pipelines

JobCubby; Atlanta (GA); Hybrid; USD 71,000 - 119,000; Full time

Benefits offered by this job

Docker containerization mentorship

Kubernetes orchestration

Professional development

Job summary

Publicis Sapient in Atlanta is seeking an Associate Data Engineer to design, develop, and maintain scalable Big Data solutions. You will work with large datasets, real-time processing frameworks, and cloud platforms to enable data-driven decision-making for our clients.

You will build pipelines with Hadoop, Spark, Kafka, and Flink, collaborate with data scientists and stakeholders, ensure data quality and security, and promote automated workflows while adopting new technologies.

Qualifications

• 5+ years of experience in Big Data engineering and analytics.
• Proficiency with Hadoop, Spark, Kafka, Flink, and Hive.
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Programming in Java, Scala, or Python for data processing and analytics.
• Hands-on with data modeling, ETL, and data warehousing.
• Strong understanding of distributed computing, data streaming, and real-time processing.
• Experience with SQL, NoSQL databases, and data lakes.
• Knowledge of data governance, security, and compliance best practices.
• Strong problem-solving and analytical skills.

Responsibilities

• Design, develop, and maintain Big Data pipelines using Hadoop, Spark, Kafka, and Flink.
• Implement data processing and transformation workflows to support analytics and machine learning.
• Optimize data storage, retrieval, and processing for performance and scalability.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver high-quality solutions.
• Ensure data quality, security, and compliance with industry standards and best practices.
• Automate data workflows and monitoring processes to improve efficiency and reliability.
• Stay updated with emerging Big Data technologies and best practices to drive innovation.

Skills

Big Data

Hadoop

Spark

Kafka

Flink

Hive

AWS

GCP

Azure

Java

Scala

Python

ETL

Data Warehousing

SQL

NoSQL

Data Lakes

Data Governance

Security

Education

Bachelor's degree in a technical field

Tools

Docker

Kubernetes

Job description

Publicis Sapient in Atlanta is seeking an Associate Data Engineer to design, develop, and maintain scalable Big Data solutions. You will work with large datasets, real-time processing frameworks, and cloud platforms to enable data-driven decision-making for our clients.

You will build pipelines with Hadoop, Spark, Kafka, and Flink, collaborate with data scientists and stakeholders, ensure data quality and security, and promote automated workflows while adopting new technologies.