Data Engineer: Big Data Pipelines & Cloud

The AES Group; San Francisco (CA); On-site; USD 130,000 - 170,000; Full time

Job summary

The AES Group in the Bay Area, CA is seeking a Data Engineer to join our team on a high-impact, long-term project. You will build, optimize, and maintain robust data pipelines that process massive volumes of web and advertising traffic, ensuring scalable infrastructure.

Collaborate with data scientists and product teams to deliver clean, structured data environments, leveraging Python/Java, SQL, and modern big data tools in AWS or GCP.

Qualifications

• Proficiency in Python or Java/Scala and SQL.
• Hands-on experience with Spark, Hadoop, Hive, or Kafka.
• Experience with public cloud platforms (AWS or GCP).
• Strong understanding of data warehousing concepts and data modeling.

Responsibilities

• Data Pipelines: Design, scale, and maintain reliable ETL/ELT data pipelines.
• Stream & Batch Processing: Process large-scale real-time and batch datasets.
• Storage Architecture: Optimize data warehouse storage, partitioning, and query performance.
• Collaboration: Work with Data Scientists and Product teams to deliver clean data environments.

Skills

Python

Java/Scala

SQL

Apache Spark

Hadoop

Hive

Kafka

AWS

GCP

Data warehousing

Job description

The AES Group in the Bay Area, CA is seeking a Data Engineer to join our team on a high-impact, long-term project. You will build, optimize, and maintain robust data pipelines that process massive volumes of web and advertising traffic, ensuring scalable infrastructure.

Collaborate with data scientists and product teams to deliver clean, structured data environments, leveraging Python/Java, SQL, and modern big data tools in AWS or GCP.