Senior Data Engineer

Recruito; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Recruito is seeking a senior data platform engineer to design and maintain scalable data platforms on Google Cloud Platform. You will build batch, incremental, and streaming pipelines, optimize transformations with BigQuery and SQL, and develop metadata-driven frameworks to support enterprise data needs.

You will lead architecture decisions, review code, and own end-to-end delivery from design through production support, while reducing cloud costs and improving reliability.

Qualifications

• Experience building enterprise-scale data platforms.
• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL and Python skills.
• Experience with Dataform/dbt and Cloud Composer/Apache Airflow.
• Hands-on experience with Dataflow/Apache Beam and Dataproc/Spark.
• Strong understanding of batch, incremental, and streaming processing.
• Experience building reusable, metadata-driven frameworks.
• Expertise in data modeling, data contracts, and schema management.
• Knowledge of data quality, lineage, monitoring, and observability.
• Experience with Git, automated testing, and CI/CD.
• Strong architectural decision-making and problem-solving skills.

Responsibilities

• Design scalable data platforms and pipelines on GCP.
• Build batch, incremental, and streaming data pipelines.
• Develop and optimize transformations using BigQuery, SQL, and Dataform/dbt.
• Build reusable, metadata-driven data-processing frameworks.
• Develop distributed processing solutions using Dataflow, Beam, and Spark.
• Define data models, schemas, and data quality standards.
• Optimize performance, reliability, and cloud costs.
• Lead code reviews and guide engineers on technical decisions.
• Own solutions from design through deployment and production support.

Skills

Advanced SQL

Python

GCP experience

BigQuery

Tools

Dataflow

Apache Beam

Spark

dbt

Dataform

Airflow

Dataproc

Git

CI/CD

Job description

Responsibilities

• Design scalable data platforms and pipelines on GCP.
• Build batch, incremental, and streaming data pipelines.
• Develop and optimize transformations using BigQuery, SQL, and Dataform/dbt.
• Build reusable, metadata-driven data-processing frameworks.
• Develop distributed processing solutions using Dataflow, Beam, and Spark.
• Define data models, schemas, and data quality standards.
• Optimize performance, reliability, and cloud costs.
• Lead code reviews and guide engineers on technical decisions.
• Own solutions from design through deployment and production support.

Requirements

• Experience building enterprise-scale data platforms.
• Strong hands-on experience with GCP and BigQuery.
• Advanced SQL and Python skills.
• Experience with Dataform/dbt and Cloud Composer/Apache Airflow.
• Hands-on experience with Dataflow/Apache Beam and Dataproc/Spark.
• Strong understanding of batch, incremental, and streaming processing.
• Experience building reusable, metadata-driven frameworks.
• Expertise in data modeling, data contracts, and schema management.
• Knowledge of data quality, lineage, monitoring, and observability.
• Experience with Git, automated testing, and CI/CD.
• Strong architectural decision-making and problem-solving skills.