Data Engineer

Jobtailor; Foster City (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Jobtailor is seeking a data engineer in Foster City, CA to design and deliver scalable data solutions for intelligent decision-making. You will build large-scale ETL pipelines, leverage Medallion Architecture, and implement Airflow DAGs for end‑to‑end data processing.

The role requires 2+ years of relevant experience, strong Python/SQL, Hadoop ecosystem familiarity, and AWS data services. Office-based with 5–10% travel; agile ceremonies are part of the process.

Qualifications

• Minimum of 6 months of work experience or a bachelor's degree.
• 2+ years of work experience preferred.
• 1-3 yrs with a CS/data focus.
• Hands-on data engineering experience building/operating production data pipelines.
• Experience with Hadoop ecosystem and Spark, Python, Pandas.
• Proficiency in Python, SQL, and PySpark; Bash scripting.
• Experience with Airflow, SQL/NoSQL databases, and AWS services.
• Strong communication skills and ability to translate requirements into implementation.

Responsibilities

• Design and deliver scalable, high-quality data solutions for decision-making.
• Build and operate large-scale data pipelines and integrated data solutions.
• Ensure efficient data storage, processing, and presentation.
• Build and utilize advanced data modeling frameworks.
• Apply Medallion Architecture and modern patterns.
• Develop ETL processes generating card-level aggregated attributes.
• Write modular, configurable, scalable code with low-level design.
• Perform unit testing and document results.
• Build, schedule, and manage DAGs in Airflow; monitor pipelines.
• Identify and fix data issues; debug to minimize delays.
• Collaborate with stakeholders to clarify requirements; participate in agile ceremonies.

Skills

Data pipeline development

Apache Airflow management

Hadoop ecosystem experience

Python and SQL proficiency

Data quality assurance

Education

Bachelor's degree in CS/Data Engineering/Info Science/Electronics

Tools

Apache Airflow

Databricks

Snowflake

AWS S3

AWS EMR

AWS Glue Catalog

Job description

• Design and deliver scalable, high-quality data solutions for intelligent decision-making
• Design and implement large-scale data pipelines and integrated data solutions
• Ensure efficient data storage, processing, and presentation
• Build and utilize advanced data modeling frameworks
• Apply modern architectural patterns such as Medallion Architecture
• Build and manage large-scale ETL processes generating card-level aggregated attributes
• Build modular and reusable code with configurability and scalability while adhering to low-level design
• Perform unit testing and document test results using standard templates
• Build, schedule, and manage DAGs in Apache Airflow
• Monitor data processing tasks in Airflow data pipelines
• Ensure data asset quality and reconcile data across pipeline stages
• Identify, discuss, and promptly fix data issues
• Debug execution errors to minimize delays and business impact
• Collaborate with stakeholders to understand and clarify requirements
• Participate actively in agile scrum ceremonies

Requirements

• Minimum of 6 months of work experience or a bachelor's degree
• 2 or more years of work experience preferred
• 1-3 yrs. work experience with a bachelor's degree with specialization in computer science, data engineering, information science, electronics, etc.
• Hands-on data engineering experience building and operating production data pipelines
• Extensive experience in Hadoop ecosystem and associated technologies, such as Apache Spark, Python, Pandas, and Open table formats such as Iceberg
• Proficiency in Python, SQL, and PySpark
• Experience with Unix/Linux systems and Bash scripting
• Experience with data pipeline and workflow management tools such as Airflow
• Experience with relational SQL and NoSQL databases
• Practical experience with AWS data engineering services, especially S3, EMR, and Glue Catalog
• Strong analytic skills working with structured and unstructured datasets
• Proficiency in managing and communicating data warehouse plans to internal clients
• Experience with data transformation, data structures, metadata, dependency, and workload management
• Experience designing for data quality, resiliency, and observability, including validation checks, monitoring, alerting, and runbooks
• Practical experience using generative AI tools such as Claude Code or similar systems, with appropriate review controls
• Strong communication skills and ability to convert ambiguous business requirements into technical implementation plans
• Experience with platforms such as Databricks or Snowflake
• Ability to work in an office setting
• Ability to travel 5-10% of the time

Core Competencies

Demonstrates expertise in designing and implementing scalable data solutions, including data pipelines and advanced data modeling frameworks. Proficient in utilizing tools such as Apache Airflow and AWS services to ensure data quality and efficient processing.

Highest-signal resume keywords

• Data Pipeline Development
• Apache Airflow Management
• Hadoop Ecosystem Experience
• Python and SQL Proficiency
• Data Quality Assurance

ATS Optimization Keywords

Hard Skills

• Data Engineering
• ETL Processes
• Data Modeling
• Data Transformation
• SQL
• Python
• PySpark
• Bash Scripting
• Hadoop
• AWS Services

Soft Skills

• Strong Communication Skills
• Analytic Skills
• Collaboration

Industry Keywords

• Data Solutions
• Data Quality
• Agile Scrum
• Medallion Architecture
• Generative AI Tools

Tools & Technologies

• Apache Airflow
• Databricks
• Snowflake
• AWS S3
• AWS EMR
• AWS Glue Catalog