Sr. Software Development Engineer

Adobe Inc.; San Jose (CA); On-site; USD 160,000 - 230,000; Full time

Job summary

Adobe Inc. in San Jose is seeking an experienced Software Development Engineer to build high‑performance data ingestion pipelines and scalable storage for petabytes of profile data. You will own design and implementation of key components and develop both streaming and batch data processing systems.

Experience with Spark, Hadoop, and Kafka, plus data lake storage like S3, is required or highly advantageous. Collaboration with cross‑functional teams and Agile practices are part of the role.

Qualifications

• BS/MS in Computer Science or related field or equivalent experience.
• 5+ years in a Software Development Engineer role.
• Experience with distributed processing systems and data lakes.
• Strong Java or Scala programming skills.

Responsibilities

• Collaborate with engineers and product managers to build high‑performance data ingestion pipelines.
• Own design and implementation of components handling petabytes of profile data.
• Develop systems for streaming and batch data processing.
• Design storage models using file and table formats.
• Participate in architecture, design & code reviews.

Skills

Java

Scala

Distributed processing

NoSQL databases

Education

B.S./M.S. in CS

Tools

Spark

Kafka

Hadoop

AWS S3

Job description

The Opportunity

Adobe Experience Platform (AEP) enables businesses to deliver the right experience at the right time to their customers. The Profile Stores layer is a key AEP service built as a multi‑cloud, multi‑tenant service to support thousands of customers, providing the ability to manage a high volume data pipeline and storage layer for real‑time customer profiles. Customer profiles are built on complex data models spanning both record structures supporting upserts at scale as well as time‑series analytics events that are ingested at very high volumes. With the need to support thousands of customers ingesting and storing petabytes of data, resiliency, data correctness, scalability and efficiency are paramount. This is a great opportunity for engineers to solve extremely interesting challenges of scale and build core services used by all Adobe Customer Experience Orchestration solutions. As part of building these services, you will work with an exceptionally talented and collaborative team, tackle complex data challenges and build highly performant services on various open‑source technologies.

The Role

This is an onsite position in the San Jose office.

• Collaborate with a team of engineers and product managers in building high‑performance data ingestion pipelines and data store to serve the use cases of Segmentation and Activation.
• Own responsibility for design and implementation of key components of ingesting and maintaining petabytes of Profile data.
• Develop systems to support high volume data ingestion pipelines handling both streaming and batch processing.
• Leverage popular file and table formats to design storage models to support the required ingestion volumes and data access patterns.
• Explore tradeoffs across different formats and schema layouts driven by workload and application characteristics.
• Deploy production services and iteratively improve them based on customer feedback.
• Follow Agile methodologies using industry‑leading CI/CD pipelines.
• Participate in architecture, design & code reviews.

Requirements

• B.S. or M.S. in Computer Science or a related field or equivalent experience.
• 5+ years of experience in a Software Development Engineer‑related occupation.
• Experience with distributed processing systems such as Apache Spark, Hadoop stack, or Apache Kafka.
• Experience with Data Lake cloud storages like Azure Data Lake Storage or AWS S3.
• Understanding of file formats like Apache Parquet and table formats such as Databricks Delta, Apache Iceberg, or Apache Hudi is preferred.
• Understanding of NoSQL databases such as Apache HBase, Cassandra, MongoDB, or Azure Cosmos DB is a plus.
• Practical experience in building resilient data pipelines at scale is preferred.
• Strong programming skills with extensive experience in Java or Scala.
• Leadership skills to collaborate and drive cross‑team efforts.
• Excellent communication skills.
• Adaptable to evolving priorities, accepting challenges outside one's comfort zone, learning new technologies, and delivering viable solutions within defined time boundaries.
• Ability to think through solutions from a short‑term and long‑term lens in an iterative development cycle.

Adobe is an equal opportunity employer. We welcome and encourage diversity in the workplace regardless of gender, race or color, ethnicity or national origin, age, disability, religion, sexual orientation, gender identity or expression, or veteran status.