GCP BigQuery Data Engineer

Virtues; Irving (TX); On-site; USD 100,000 - 110,000; Full time

Job summary

Virtues is seeking a GCP BigQuery Data Engineer to design cloud-native, scalable data platforms and build real-time ingestion pipelines on Google Cloud. This onsite role in Irving, TX focuses on event-driven streaming architectures that support near real-time analytics and enterprise reporting.

Responsibilities include designing and optimizing data warehouses with BigQuery, building streaming pipelines with NiFi and Kafka, and implementing ETL/ELT processes for scalable data lakes and analytics

Qualifications

• 7+ years of experience in Data Engineering, Data Warehousing, or Big Data platforms.

Responsibilities

• Design, develop, and optimize enterprise data warehouse solutions using Google BigQuery.
• Design, develop, and maintain real-time data ingestion pipelines using Apache NiFi to extract, transform, and route streaming data into Apache Kafka topics.
• Build scalable event-driven streaming architectures using Apache NiFi, Apache Kafka, and Google BigQuery to enable high-throughput, fault-tolerant, low-latency processing.
• Configure and optimize Apache NiFi processors for extraction, transformation, routing, filtering, schema validation, error handling, retry mechanisms, and reliable delivery to Kafka.
• Develop streaming ingestion solutions that allow Google BigQuery to consume Kafka event streams and transform near real-time events into analytical tables and reporting datasets.
• Design data models and implement ETL/ELT processes to move data from raw to curated and published layers.
• Design, create, and manage large-scale BigQuery datasets, including temporary/permanent and internal/external tables.
• Optimize BigQuery workloads using query tuning, partitioning, clustering, and cost optimization techniques to improve performance and reduce cloud costs.
• Monitor, troubleshoot, and optimize streaming workloads by tuning NiFi flows, Kafka topics/partitions, and BigQuery streaming ingestion to support high availability, data integrity, minimal latency, and cost-efficient processing.
• Build scalable data lake frameworks and ingestion pipelines using cloud-native GCP technologies.
• Develop reporting and visualization solutions using Looker, Looker Studio (Data Studio), Connected Sheets, and other BigQuery reporting tools.
• Collaborate with data architects, modelers, developers, DevOps engineers, project managers, and business stakeholders to deliver scalable enterprise analytics solutions and continuous improvements to the data platform.

Skills

Apache NiFi

Apache Kafka

GCP

BigQuery

Looker

Looker Studio

Connected Sheets

SQL

ETL/ELT

Python

Spark

Data Modeling

Data Warehousing

Data Lakes

Batch & Streaming Pipelines

Tools

Google Cloud Dataflow

Pub/Sub

GCS

Looker

Looker Studio

Connected Sheets

Job description

Virtues is seeking a GCP BigQuery Data Engineer to design cloud-native, scalable data platforms and build real-time ingestion pipelines on Google Cloud. This onsite role in Irving, TX focuses on event-driven streaming architectures that support near real-time analytics and enterprise reporting.

Responsibilities

• Design, develop, and optimize enterprise data warehouse solutions using Google BigQuery.
• Design, develop, and maintain real-time data ingestion pipelines using Apache NiFi to extract, transform, and route streaming data into Apache Kafka topics.
• Build scalable event-driven streaming architectures using Apache NiFi, Apache Kafka, and Google BigQuery to enable high-throughput, fault-tolerant, low-latency processing.
• Configure and optimize Apache NiFi processors for extraction, transformation, routing, filtering, schema validation, error handling, retry mechanisms, and reliable delivery to Kafka.
• Develop streaming ingestion solutions that allow Google BigQuery to consume Kafka event streams and transform near real-time events into analytical tables and reporting datasets.
• Design data models and implement ETL/ELT processes to move data from raw to curated and published layers.
• Design, create, and manage large-scale BigQuery datasets, including temporary/permanent and internal/external tables.
• Optimize BigQuery workloads using query tuning, partitioning, clustering, and cost optimization techniques to improve performance and reduce cloud costs.
• Monitor, troubleshoot, and optimize streaming workloads by tuning NiFi flows, Kafka topics/partitions, and BigQuery streaming ingestion to support high availability, data integrity, minimal latency, and cost-efficient processing.
• Build scalable data lake frameworks and ingestion pipelines using cloud-native GCP technologies.
• Develop reporting and visualization solutions using Looker, Looker Studio (Data Studio), Connected Sheets, and other BigQuery reporting tools.
• Collaborate with data architects, modelers, developers, DevOps engineers, project managers, and business stakeholders to deliver scalable enterprise analytics solutions and continuous improvements to the data platform.

Requirements

• 7+ years of experience in Data Engineering, Data Warehousing, or Big Data platforms.
• Must-have skills: Apache NiFi and Apache Kafka.
• Strong hands-on experience with Google Cloud Platform (GCP), including BigQuery, Cloud Dataflow, Pub/Sub, and Google Cloud Storage (GCS).
• Hands-on experience designing and supporting real-time streaming pipelines using Apache NiFi, Apache Kafka, and Google BigQuery.
• Experience using BigQuery Console/Query Editor for data management, performance tuning, and SQL development.
• Strong experience designing scalable data models, ETL/ELT pipelines, data lake architectures, and enterprise data warehouse solutions.
• Experience with batch and streaming data ingestion using GCP services.
• Thorough understanding of BigQuery cost structure, including storage, ingestion, and query costs, with experience implementing query optimization, partitioning, clustering, and other cost optimization techniques.
• Experience managing large-scale datasets, including temporary/permanent and internal/external BigQuery tables.
• Experience developing reporting and visualization solutions using Looker, Looker Studio (Data Studio), and Connected Sheets.
• Excellent verbal and written communication skills, with the ability to collaborate with technical teams, business stakeholders, senior management, and executive leadership.
• Experience working in Agile environments and collaborating with cross-functional teams to deliver enterprise data engineering solutions.

Preferred Qualifications

• Google Cloud certifications.
• Experience delivering enterprise-scale analytics and cloud data solutions.

Technologies

• Google BigQuery
• Apache NiFi
• Apache Kafka
• Google Cloud Dataflow
• Google Pub/Sub
• SQL
• ETL/ELT
• Google Cloud Storage (GCS)
• Looker
• Looker Studio (Data Studio)
• Connected Sheets
• CI/CD
• DevOps
• Agile/Scrum
• Python
• Spark
• Data Modeling
• Data Warehousing
• Data Lakes
• Batch & Streaming Pipelines

Compensation and Location

Salary: USD 100,000 - 110,000 per year.

Work location: In person (Irving, TX).