Senior Data Infra Engineer – Hybrid & AWS

Aurora; San Francisco (CA); Hybrid; USD 162,000 - 260,000; Full time

Job summary

Aurora is seeking a Senior Data Platform Engineer to design, build, and maintain scalable data pipelines and the data warehouse ecosystem. You will work on core data lake and data warehouse infrastructure, ingesting data from diverse sources and ensuring quality, security, and accessibility at massive scale.

You will collaborate with data scientists, analysts, autonomy teams, and product teams to meet data needs, while optimizing performance and cost-efficiency in a hybrid, in-office model in

Qualifications

• Master's degree in Computer Science, Engineering, or related field or equivalent practical experience.
• 5+ years of professional experience in software engineering, with a focus on data-related projects.
• Proficiency in at least one programming language commonly used for data engineering (e.g., Python, Go or C++).
• Solid experience with big data processing frameworks like Presto/Trino, EMR, Apache Spark, Flink, Kinesis Data Streams, or similar technologies.
• Hands-on experience with cloud platforms (AWS, GCP, or Azure) and their data services (e.g., S3, Redshift, BigQuery, Glue).
• Strong knowledge of SQL and experience working with relational and NoSQL databases.
• Intermediate knowledge of data analytics infrastructure, including data transformation tools such as DBT and visualization frameworks and tools
• Experience with building and managing data pipelines using an orchestrator like Apache Airflow.
• Able to systematically approach open-ended questions to identify pragmatic data solutions that scale
• Able to work effectively in a highly cross-functional, fast-moving and high-stakes environment
• Proven ability to communicate technical, data-driven solutions to both technical and non-technical audiences across stakeholders

Responsibilities

• Design, build, and maintain robust and scalable data pipelines and ETL/ELT processes to ingest, transform, and load data from various sources into our data warehouse.
• Develop and manage data infrastructure components using AWS cloud services and infrastructure-as-code tools like Terraform.
• Collaborate with data scientists, analysts, autonomy engineering teams and product teams to understand their data needs and build solutions that meet their requirements.
• Optimize data processing systems for performance, reliability, and cost-efficiency.
• Implement monitoring, alerting, and logging for data pipelines and infrastructure to ensure operational stability.
• Champion best practices in data governance, data quality, and security.
• Work closely with other senior team members and management to improve the data ecosystem toolings, refine user experience, and continuously polish team roadmap.

Skills

Python

Go

C++

SQL

ETL/ELT

AWS

Terraform

Airflow

Big data

Data governance

Data visualization

Education

Master's degree in Computer Science, Engineering, or related field

Tools

Presto/Trino

EMR

Apache Spark

Flink

Kinesis Data Streams

S3

Redshift

BigQuery

Glue

DBT

Airflow

Job description

Aurora is seeking a Senior Data Platform Engineer to design, build, and maintain scalable data pipelines and the data warehouse ecosystem. You will work on core data lake and data warehouse infrastructure, ingesting data from diverse sources and ensuring quality, security, and accessibility at massive scale.

You will collaborate with data scientists, analysts, autonomy teams, and product teams to meet data needs, while optimizing performance and cost-efficiency in a hybrid, in-office model in