Data Engineer

Weekday (YC W21); New York (NY); On-site; USD 160,000 - 200,000; Full time

Job summary

Weekday (YC W21) in New York City is seeking a seasoned Data Engineer to design, build, and maintain scalable data pipelines and infrastructure powering analytics, reporting, ML, and business-critical apps.

The candidate will showcase strong Python and SQL skills, data modeling expertise, and experience with ETL/ELT pipelines across cloud platforms (AWS/GCP/Azure) and modern data warehouses. Collaborate with engineers, data scientists, and product teams to deliver reliable data assets.

Qualifications

• 4–5 years of professional experience in Data Engineering or closely related role.
• Strong proficiency in Python and SQL.
• Hands-on experience building ETL/ELT pipelines.
• Strong understanding of data modeling, data warehousing, and database design.
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Experience with modern data warehouses such as Snowflake, BigQuery, Redshift, or Databricks.
• Experience with data processing technologies such as Spark or PySpark.
• Experience with workflow orchestration tools such as Airflow, Dagster, or Prefect.
• Strong understanding of relational and NoSQL databases.
• Experience working with APIs, data ingestion frameworks, and distributed systems.
• Familiarity with Git, CI/CD, testing, and software engineering best practices.
• Strong analytical, debugging, and problem-solving skills.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build reliable data ingestion and transformation workflows from multiple sources.
• Develop efficient and complex SQL queries for large datasets.
• Design and maintain data models, schemas, and data warehouse architectures.
• Build batch and real-time data processing pipelines.
• Work with cloud-based data platforms and distributed data processing systems.
• Ensure data quality, consistency, availability, and reliability across pipelines.
• Optimize data pipelines and queries for performance and cost.
• Develop monitoring, alerting, and data-quality checks for production pipelines.
• Collaborate with data scientists, analysts, product managers, and software engineers.
• Support analytics, reporting experimentation, and machine-learning use cases.
• Participate in architecture discussions and technical design reviews.
• Maintain documentation for data pipelines, systems, and processes.
• Troubleshoot production data issues and implement long-term solutions.

Skills

Python

SQL

ETL/ELT pipelines

Data modeling and warehousing

Cloud platforms (AWS/GCP/Azure)

Data warehouses (Snowflake/BigQuery/RS

Tools

Databricks

Job description

Location: New York City, NY

Experience: 4–5 years

Employment Type: Full-time

Compensation: $160,000 – $200,000 per year + bonus/equity

About the Role

We are looking for a skilled and experienced Data Engineer to join our engineering and data team in New York City. You will be responsible for designing, building, and maintaining reliable data pipelines and scalable data infrastructure that power analytics, reporting, machine learning, and business-critical applications.

The ideal candidate has strong expertise in Python, SQL, data modeling, ETL/ELT pipelines, cloud platforms, and modern data engineering technologies. You should be comfortable working with large and complex datasets and collaborating closely with software engineers, data scientists, analysts, and product teams.

Requirements

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build reliable data ingestion and transformation workflows from multiple sources.
• Develop efficient and complex SQL queries for large datasets.
• Design and maintain data models, schemas, and data warehouse architectures.
• Build batch and real-time data processing pipelines.
• Work with cloud-based data platforms and distributed data processing systems.
• Ensure data quality, consistency, availability, and reliability across pipelines.
• Optimize data pipelines and queries for performance and cost.
• Develop monitoring, alerting, and data-quality checks for production pipelines.
• Collaborate with data scientists, analysts, product managers, and software engineers.
• Support analytics, reporting experimentation, and machine-learning use cases.
• Participate in architecture discussions and technical design reviews.
• Maintain documentation for data pipelines, systems, and processes.
• Troubleshoot production data issues and implement long-term solutions.

Must-Have Skills

• 4–5 years of professional experience in Data Engineering or a closely related role.
• Strong proficiency in Python and SQL.
• Hands-on experience building ETL/ELT pipelines.
• Strong understanding of data modeling, data warehousing, and database design.
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Experience with modern data warehouses such as Snowflake, BigQuery, Redshift, or Databricks.
• Experience with data processing technologies such as Spark or PySpark.
• Experience with workflow orchestration tools such as Airflow, Dagster, or Prefect.
• Strong understanding of relational and NoSQL databases.
• Experience working with APIs, data ingestion frameworks, and distributed systems.
• Familiarity with Git, CI/CD, testing, and software engineering best practices.
• Strong analytical, debugging, and problem-solving skills.

Nice-to-Have Skills

• Experience with Kafka, Kinesis, or other streaming technologies.
• Experience with dbt and modern analytics engineering practices.
• Hands-on experience with Databricks or Snowflake.
• Experience building real-time or near-real-time data pipelines.
• Knowledge of Kubernetes and containerized workloads.
• Experience with infrastructure-as-code tools such as Terraform.
• Familiarity with data governance, lineage, security, and privacy practices.
• Experience supporting machine-learning or AI data pipelines.
• AWS, GCP, Azure, or Databricks certifications.
• Experience working in a high-growth startup or large-scale technology environment.