Data Engineer II - Design & Scale Complex Data Pipelines

Redfin; Seattle (WA); On-site; USD 120,000 - 162,000; Full time

Benefits offered by this job

Health benefits

401K

Paid time off

Job summary

Redfin is seeking a Data Engineer II to join our data platform team in Seattle. You will own critical data integrations and maintain data pipelines across multiple systems, collaborating with Engineering and Analytics to deliver reliable data products.

The ideal candidate builds scalable pipelines, writes robust SQL, and thrives on automating processes for accuracy and performance. Responsibilities include designing data exchange systems, gathering user requirements, and ensuring

Qualifications

• 3+ years experience with a modern programming language (Python, Java, etc). Python is preferred.
• 3+ years experience with complex SQL, data modeling and database technologies (Postgres, MySQL, Snowflake, etc). Snowflake is preferred.
• Experience creating and maintaining complex data pipelines.
• Experience with batch and streaming data pipelines.
• Experience with both structured and unstructured data sets.
• Ability to translate business needs into technical requirements.
• Experience with Git / Jenkins / Airflow / Docker / Kubernetes / Spark is a plus.
• Experience with API development is a plus.
• Experience with cloud services (AWS / Azure / GCP) is a plus. AWS is preferred.
• Ability to structure data to make it accessible, performant, and resilient to change.
• Experience with message queuing and stream processing data is a plus.
• Ability to handle ambiguity and function in a fast paced, constantly changing environment.
• Ability to communicate well with both technical and non technical users.

Responsibilities

• Design, develop, and support complex data pipelines from a wide variety of data sources for Redfin
• Work with other Engineering teams to design and implement data exchange systems from structured and unstructured data sources
• Work with end users to gather requirements and design solutions that provide the data they need in the format they need it in
• Ensure 100% accuracy in everything you do - make sure that all code and processes are fully tested and verified before running in production
• Identify and implement internal process improvements by automating manual processes and optimizing for accuracy, performance and coding best practices
• Support and maintain dev / test / prod environments
• Serve as part of weekly on call rotation

Skills

Python

Java

SQL

Data pipelines

Batch processing

Streaming data

Structured data

Unstructured data

Git

Jenkins

Airflow

Docker

Kubernetes

Spark

Cloud (AWS)

GCP

Azure

APIs

Communication

Tools

Git

Jenkins

Airflow

Docker

Kubernetes

Spark

Job description

Redfin is seeking a Data Engineer II to join our data platform team in Seattle. You will own critical data integrations and maintain data pipelines across multiple systems, collaborating with Engineering and Analytics to deliver reliable data products.

The ideal candidate builds scalable pipelines, writes robust SQL, and thrives on automating processes for accuracy and performance. Responsibilities include designing data exchange systems, gathering user requirements, and ensuring