Senior GCP Data Engineer: Big Data Pipelines & Cloud

Fixity Technologies; Charlotte (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

Fixity Technologies is seeking an experienced Senior Data Engineer to design, develop, and optimize scalable data pipelines for large-scale processing. You will build cloud-based data engineering solutions on Google Cloud Platform (GCP) using BigQuery, DataProc, Airflow, and Pub/Sub, and you will implement batch and real-time processing with Kafka and GCP Pub/Sub.

Proficiency in Spark/Hadoop/Hive and Python is required.

Qualifications

• 8+ years of experience in software development and/or Big Data Engineering.

Responsibilities

• Design, develop, and optimize scalable data pipelines for large-scale data processing and transformation.

Skills

GCP

BigQuery

Apache Airflow

Cloud Composer

DataProc

Pub/Sub

Apache Spark

Hadoop

Hive

Python

Shell Scripting

SQL

Jenkins

Git

GitHub

CI/CD

distributed systems

SDLC

Agile/Scrum

Tools

GCP services

BigQuery

Airflow/Cloud Composer

DataProc

Pub/Sub

Kafka

Job description

Fixity Technologies is seeking an experienced Senior Data Engineer to design, develop, and optimize scalable data pipelines for large-scale processing. You will build cloud-based data engineering solutions on Google Cloud Platform (GCP) using BigQuery, DataProc, Airflow, and Pub/Sub, and you will implement batch and real-time processing with Kafka and GCP Pub/Sub.

Proficiency in Spark/Hadoop/Hive and Python is required.