Senior GCP Data Engineer

Envision Technology Solutions

Minnesota City (MN)

On-site

USD 120,000 - 160,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Envision Technology Solutions is seeking a Senior Data Engineer in the United States to design and operate scalable data pipelines, focusing on batch and near-real-time workloads across customer, transaction, and marketing datasets.

You will work with architects, PMs, data scientists, and privacy teams to enforce data contracts and data quality, while optimizing BigQuery, Spark, and the GCP stack for performance and cost.

Qualifications

  • Advanced SQL and Python for ingestion, transformation, validation, reconciliation.
  • GCP services: BigQuery, Dataflow/Beam, Pub/Sub, Cloud Storage, Airflow.
  • Strong Spark and distributed data-processing, with partitioning and tuning.
  • Data modeling for customer, identity, commerce, campaign, audience, and measurement datasets.
  • Experience with data contracts, schema evolution, metadata, lineage, and data quality.
  • CI/CD with GitHub Actions and infrastructure-as-code; automated tests.
  • Pipeline observability: freshness, completeness, accuracy, latency, alerts.
  • Security: encryption, tokenization, RBAC, secrets management, retention controls.
  • Identity resolution knowledge and audience activation concepts.

Responsibilities

  • Design, build, and operate scalable batch and near-real-time pipelines.
  • Collaborate with architects, PMs, analysts, data scientists, privacy teams, and engineers.
  • Define data contracts, SLAs, and acceptance criteria.
  • Implement controls for schema, completeness, freshness, validity, and reconciliation.
  • Optimize BigQuery, Spark, and pipeline workloads for throughput and cost.
  • Develop tests and embed quality gates in CI/CD; establish runbooks and monitoring.

Skills

SQL
Python
GCP
BigQuery
Apache Spark
Data Modeling
Data Quality
CI/CD
Security
Identity Resolution

Education

Bachelor's or Master's in CS / DS / IS / Engineering

Tools

Dataflow/Beam
Pub/Sub
Cloud Storage
Airflow / Cloud Composer
GitHub Actions

Job description

Responsibilities
  • Advanced SQL and Python for ingestion, transformation, validation, reconciliation, and performance optimization.
  • Hands-on GCP experience with BigQuery, Dataflow / Apache Beam, Pub/Sub, Cloud Storage, and Cloud Composer / Airflow.
  • Strong Apache Spark and distributed data-processing experience, including partitioning, skew handling, incremental loads, and tuning.
  • Data modeling for customer, identity, commerce, campaign, audience, and measurement datasets.
  • Experience with data contracts, schema evolution, metadata, lineage, automated data-quality controls, and reconciliation.
  • CI/CD using GitHub / GitHub Actions or equivalent, automated tests, and infrastructure-as-code.
  • Pipeline observability covering freshness, completeness, accuracy, latency, volume, failures, dashboards, and alerts.
  • Secure data engineering using encryption, tokenization / hashing, role-based access, secrets management, and retention controls.
  • Working knowledge of identity resolution, identity graphs, customer 360, consent-aware processing, and audience activation.
  • Retail media, advertising, marketing technology, or campaign-measurement experience is preferred.
  • Design, build, and operate scalable batch and near-real-time pipelines for customer, transaction, product, digital interaction, campaign, and partner data.
  • Develop curated data products supporting identity resolution, audience segmentation, activation, analytics, attribution, and measurement.
  • Partner with architects, product managers, analysts, data scientists, privacy teams, and application engineers to define data contracts, SLAs, and acceptance criteria.
  • Implement controls for schema, uniqueness, referential integrity, completeness, freshness, validity, and source reconciliation.
  • Optimize BigQuery, Spark, and pipeline workloads for throughput, latency, reliability, and cost.
  • Build unit, integration, regression, and production-validation tests and embed quality gates in CI/CD.
  • Establish monitoring, alerting, dashboards, runbooks, incident triage, and permanent corrective actions.
  • Protect sensitive identifiers using approved access, encryption, tokenization / hashing, retention, auditability, and least-privilege design.
  • Perform code reviews, maintain technical and lineage documentation, and contribute to reusable engineering standards.
  • Support releases, backfills, migrations, performance tuning, and controlled recovery from pipeline failures.
Generic Managerial Skills, If any
  • Own delivery from design through production stabilization and communicate risks, dependencies, and status clearly.
  • Collaborate effectively across onsite and offshore teams in an Agile model.
  • Mentor engineers through design reviews, code reviews, and troubleshooting.
  • Demonstrate strong problem-solving, stakeholder communication, documentation, and prioritization.
Education

Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, Engineering, or related discipline

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior GCP Data Engineer - Scalable Pipelines & Analytics
Senior GCP Data Engineer - Scalable Pipelines & Analytics

Compunnel Inc. • Phoenix (AZ)

Hybrid
USD 120,000 - 180,000
GCP Data Engineer
GCP Data Engineer

TechDigital Group • Seattle (WA)

On-site
USD 130,000 - 160,000
GCP Data Engineer: Scalable Pipelines, IAM & Airflow
GCP Data Engineer: Scalable Pipelines, IAM & Airflow

TechDigital Group • Seattle (WA)

On-site
USD 130,000 - 160,000
GCP Data Engineer
GCP Data Engineer

Rivago Infotech Inc • New York (NY)

On-site
USD 120,000 - 160,000
GCP Data Engineer: Pipelines, APIs & Mentoring
GCP Data Engineer: Pipelines, APIs & Mentoring

Compunnel Inc. • Lisle (IL)

On-site
USD 90,000 - 120,000
GCP Data Engineer: Real-Time Pipelines & Kubernetes Lead
GCP Data Engineer: Real-Time Pipelines & Kubernetes Lead

TechDigital Group • Raleigh (NC)

On-site
USD 100,000 - 130,000
GCP Data Engineer
GCP Data Engineer

TechDigital Group • Secaucus (NJ)

On-site
USD 140,000 - 190,000
GCP Data Engineer
GCP Data Engineer

Quantum Technologies LLC • Tampa (FL)

On-site
USD 120,000 - 180,000
GCP Data Engineer
GCP Data Engineer

Valiant Technologies LLC • Tampa (FL)

On-site
USD 110,000 - 165,000
GCP Data Engineer
GCP Data Engineer

Arkhya Tech. Inc. • Concord (CA)

On-site
USD 140,000 - 190,000