Senior Data Ingestion Engineer - Petabyte Pipelines

Adobe Inc.; San Jose (CA); On-site; USD 160,000 - 230,000; Full time

Job summary

Adobe Inc. in San Jose is seeking an experienced Software Development Engineer to build high‑performance data ingestion pipelines and scalable storage for petabytes of profile data. You will own design and implementation of key components and develop both streaming and batch data processing systems.

Experience with Spark, Hadoop, and Kafka, plus data lake storage like S3, is required or highly advantageous. Collaboration with cross‑functional teams and Agile practices are part of the role.

Qualifications

• BS/MS in Computer Science or related field or equivalent experience.
• 5+ years in a Software Development Engineer role.
• Experience with distributed processing systems and data lakes.
• Strong Java or Scala programming skills.

Responsibilities

• Collaborate with engineers and product managers to build high‑performance data ingestion pipelines.
• Own design and implementation of components handling petabytes of profile data.
• Develop systems for streaming and batch data processing.
• Design storage models using file and table formats.
• Participate in architecture, design & code reviews.

Skills

Java

Scala

Distributed processing

NoSQL databases

Education

B.S./M.S. in CS

Tools

Spark

Kafka

Hadoop

AWS S3

Job description

Adobe Inc. in San Jose is seeking an experienced Software Development Engineer to build high‑performance data ingestion pipelines and scalable storage for petabytes of profile data. You will own design and implementation of key components and develop both streaming and batch data processing systems.

Experience with Spark, Hadoop, and Kafka, plus data lake storage like S3, is required or highly advantageous. Collaboration with cross‑functional teams and Agile practices are part of the role.