Software Development Engineer II, Amazon Security Data Engineering

Amazon; Herndon (VA); On-site; USD 143,700 - 194,400; Full time

Job summary

Amazon is seeking a software engineer for its Security Platform Engineering team to design, build, and operate a petabyte-scale data lake and governance services. You will drive the technical vision for data catalogs, optimization, and indexing using Iceberg, Parquet, and Spark-based engines.

You will contribute to distributed systems for low-latency threat detection, implement access controls, and collaborate with AWS and other business units.

Qualifications

• 3+ years of non-internship professional software development experience.
• 3+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• 3+ years of designing and developing large-scale, multi-tiered, multi-threaded, distributed software applications, tools, systems, and services using: C#, C++, Java, or Perl experience.
• 3+ years of Object Oriented Design experience.
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field

Responsibilities

• Build and support Amazon Security's Data Lake, consisting of services for data cataloging, optimization, compaction, re-organization, and indexing at petabyte scale.
• Drive technical vision and contribute to the roadmap for systems that manage petabyte-scale data catalogs supporting Apache Iceberg, AWS Glue, Parquet, and other formats.
• Design and implement distributed systems for data optimization that balance multiple latency scenarios from minute-level batch detections to month-spanning threat hunting queries.
• Modernize security intelligence platform's data catalog and governance services to provide customers with rule-based access controls and fine-grained data access.
• Build and support query platform services for query scheduling, routing, auditing, and engine built on Apache Spark.
• Optimize data ingestion and cataloging systems that handle petabytes of data.
• Partner with data engineers and security teams across AWS, Amazon Stores, Digital, and other business units to understand requirements and deliver platform capabilities.
• Uphold software engineering best practices, code quality standards, and operational excellence.
• Explore and build AI driven data discovery capabilities using catalog metadata to help security analysts find relevant datasets faster.
• Work with AI/ML powered developer tools across the full SDLC, from AI assisted code reviews and design to agentic operational response

Skills

Java

Python

Scala

C++

Big data

Distributed systems

Multi-threading

Memory management

OO design

Communication

Education

Bachelor's degree in Computer Science/Engineering/Mathematics or related field

Tools

Apache Iceberg

Apache Spark

Trino

Flink

AWS Glue

Parquet

Job description

Description

If you thrive in a fast-paced, innovative environment and are interested in pushing the boundaries of what's possible with big data, cloud computing, and security analytics, Amazon Security Platform Engineering is the place for you. We support a suite of services and products to ingest, optimize and query petabyte scale datasets efficiently, and enable critical security use cases such as incident response, security analytics, threat-hunting, and low-latency detection.

Successful candidates for this position will possess:

Strong software engineering fundamentals, including expertise in object-oriented design, algorithms, and data structures

Experience building and operating scalable, fault-tolerant, distributed systems

Proficiency in Java, Scala or Python with a solid understanding of multi-threading and memory management

Knowledge of big data compute engines such as open table formats like Apache Iceberg, and Apache Spark, Trino, Flink.

Excellent verbal and written communication skills

Self-motivation and the ability to deliver high-quality results in a fast-paced environment.

Key job responsibilities

• Build and support Amazon Security's Data Lake, consisting of services for data cataloging, optimization, compaction, re-organization, and indexing at petabyte scale.


• Drive technical vision and contribute to the roadmap for systems that manage petabyte-scale data catalogs supporting Apache Iceberg, AWS Glue, Parquet, and other formats.


• Design and implement distributed systems for data optimization that balance multiple latency scenarios from minute-level batch detections to month-spanning threat hunting queries.


• Modernize security intelligence platform's data catalog and governance services to provide customers with rule-based access controls and fine-grained data access.


• Build and support query platform services for query scheduling, routing, auditing, and engine built on Apache Spark.


• Optimize data ingestion and cataloging systems that handle petabytes of data.


• Partner with data engineers and security teams across AWS, Amazon Stores, Digital, and other business units to understand requirements and deliver platform capabilities.


• Uphold software engineering best practices, code quality standards, and operational excellence.


• Explore and build AI driven data discovery capabilities using catalog metadata to help security analysts find relevant datasets faster.


• Work with AI/ML powered developer tools across the full SDLC, from AI assisted code reviews and design to agentic operational response

About the team

Why Amazon Security?

At Amazon, security is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for security across all of Amazon's products and services. We offer talented security professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores.

• Diverse Experiences: Amazon Security values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying.


• Inclusive Team Culture: In Amazon Security, it's in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest security challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices.


• Training & Career Growth: We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional.


• Work/Life Balance: We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there's nothing we can't achieve.

Basic Qualifications

• 3+ years of non-internship professional software development experience


• 3+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience


• 3+ years of designing and developing large-scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using: C#, C++, Java, or Perl experience


• 3+ years of Object Oriented Design experience


• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field

Preferred Qualifications

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits .

USA, VA, Herndon - 143,700.00 - 194,400.00 USD annually