Senior GCP Data Engineer: Airflow, Spark & FastAPI

Eitacies Inc; Austin (TX); On-site; USD 120,000 - 150,000; Full time

Job summary

A tech company is seeking a Senior Data Platform Engineer for an onsite position in Austin, Texas. The successful candidate will have over 6 years of experience in data engineering, strong Python skills, and production knowledge of Apache Spark and Google Cloud Platform. Responsibilities include designing data workflows, optimizing data pipelines, and developing REST APIs. This role is pivotal in ensuring data quality and performance, making it essential for candidates to have robust data modeling and data quality check implementations.

Qualifications

• 6+ years of data engineering experience.
• Strong Python programming skills.
• Production experience with Apache Spark PySpark.
• Experience with Airflow DAG development.
• Hands-on experience with GCP (Dataproc, BigQuery, Cloud Storage preferred).
• Experience building APIs using FastAPI.
• Strong SQL and data modeling knowledge.
• Experience implementing data quality checks (schema validation, row count checks, anomaly detection).

Responsibilities

• Design and maintain Airflow DAGs for production data workflows.
• Develop and optimize Spark / PySpark jobs on Dataproc.
• Build and optimize complex SQL transformations.
• Develop REST APIs using FastAPI Python to support data services.
• Implement robust data quality and validation frameworks.
• Monitor pipeline performance, retries, and SLA compliance.

Skills

Data engineering

Python programming

Apache Spark PySpark

Airflow DAG development

Google Cloud Platform

APIs development

SQL

Data quality checks

Job description

A tech company is seeking a Senior Data Platform Engineer for an onsite position in Austin, Texas. The successful candidate will have over 6 years of experience in data engineering, strong Python skills, and production knowledge of Apache Spark and Google Cloud Platform. Responsibilities include designing data workflows, optimizing data pipelines, and developing REST APIs. This role is pivotal in ensuring data quality and performance, making it essential for candidates to have robust data modeling and data quality check implementations.