Data Engineer: Scalable Pipelines & Medallion Architecture

Jobtailor; Foster City (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Jobtailor is seeking a data engineer in Foster City, CA to design and deliver scalable data solutions for intelligent decision-making. You will build large-scale ETL pipelines, leverage Medallion Architecture, and implement Airflow DAGs for end‑to‑end data processing.

The role requires 2+ years of relevant experience, strong Python/SQL, Hadoop ecosystem familiarity, and AWS data services. Office-based with 5–10% travel; agile ceremonies are part of the process.

Qualifications

• Minimum of 6 months of work experience or a bachelor's degree.
• 2+ years of work experience preferred.
• 1-3 yrs with a CS/data focus.
• Hands-on data engineering experience building/operating production data pipelines.
• Experience with Hadoop ecosystem and Spark, Python, Pandas.
• Proficiency in Python, SQL, and PySpark; Bash scripting.
• Experience with Airflow, SQL/NoSQL databases, and AWS services.
• Strong communication skills and ability to translate requirements into implementation.

Responsibilities

• Design and deliver scalable, high-quality data solutions for decision-making.
• Build and operate large-scale data pipelines and integrated data solutions.
• Ensure efficient data storage, processing, and presentation.
• Build and utilize advanced data modeling frameworks.
• Apply Medallion Architecture and modern patterns.
• Develop ETL processes generating card-level aggregated attributes.
• Write modular, configurable, scalable code with low-level design.
• Perform unit testing and document results.
• Build, schedule, and manage DAGs in Airflow; monitor pipelines.
• Identify and fix data issues; debug to minimize delays.
• Collaborate with stakeholders to clarify requirements; participate in agile ceremonies.

Skills

Data pipeline development

Apache Airflow management

Hadoop ecosystem experience

Python and SQL proficiency

Data quality assurance

Education

Bachelor's degree in CS/Data Engineering/Info Science/Electronics

Tools

Apache Airflow

Databricks

Snowflake

AWS S3

AWS EMR

AWS Glue Catalog

Job description

Jobtailor is seeking a data engineer in Foster City, CA to design and deliver scalable data solutions for intelligent decision-making. You will build large-scale ETL pipelines, leverage Medallion Architecture, and implement Airflow DAGs for end‑to‑end data processing.

The role requires 2+ years of relevant experience, strong Python/SQL, Hadoop ecosystem familiarity, and AWS data services. Office-based with 5–10% travel; agile ceremonies are part of the process.