Senior Data Engineer: Build Real-Time Data Lake & Pipelines

Way; Austin (TX); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Health insurance

Annual hotel and travel stipend

Office locations in Austin, SF, and巴黎

Job summary

Way is seeking an experienced Senior Data Engineer to establish and lead our data infrastructure as an early member of the data team. You will build our data platform from the ground up, implementing a data lake and pipeline architecture and driving a data-driven culture across the organization.

The ideal candidate will lead data engineering and analytics responsibilities, collaborating with product and business teams to define metrics, KPIs, and BI needs while promoting automation and best

Qualifications

• 5+ years hands-on data engineering experience with increasing responsibility.
• Proven track record of building data systems from scratch or leading data infrastructure transformations.
• Experience with cloud platforms (AWS, GCP, or Azure) and data services.
• Experience with data orchestration tools (Airflow, Prefect, Dagster).

Responsibilities

• Design and implement a comprehensive data lake architecture using modern cloud-native technologies
• Build scalable ETL/ELT pipelines for real-time and batch data processing across all data sources
• Establish data ingestion frameworks to collect data from application APIs, third-party services, and databases
• Architect automated data quality monitoring, validation, and alerting systems
• Create robust data warehousing solutions optimized for analytics and business intelligence
• Implement DataOps practices with automated testing and deployment pipelines (CI/CD for data)
• Develop and maintain Python-based data processing frameworks and utilities
• Build an automated data pipeline orchestration using Apache Airflow or similar tools
• Create streaming data processing solutions using Apache Kafka, Kinesis, or Pub/Sub
• Implement infrastructure as code for all data platform components (Terraform, CloudFormation)
• Establish feature stores and data models that support both operational and analytical workloads
• Optimize data storage costs and query performance across the entire platform
• Collaborate with product and business teams to identify key metrics, KPIs, and analytical requirements
• Build automated reporting dashboards and self-service business intelligence tools
• Support predictive modeling initiatives and A/B testing frameworks

Skills

Python

SQL

Cloud platforms

Data modeling

ETL pipelines

Data quality

DataOps

Tools

Apache Airflow

Kafka

Snowflake

BigQuery

Redshift

Databricks

Terraform

Delta Lake

Apache Iceberg

Apache Hudi

Kubernetes

Docker

Git

Job description

Way is seeking an experienced Senior Data Engineer to establish and lead our data infrastructure as an early member of the data team. You will build our data platform from the ground up, implementing a data lake and pipeline architecture and driving a data-driven culture across the organization.

The ideal candidate will lead data engineering and analytics responsibilities, collaborating with product and business teams to define metrics, KPIs, and BI needs while promoting automation and best