GCP Data Engineer- Only for FTE

KPG99 INC; Phoenix (AZ); On-site; USD 140,000 - 190,000; Full time

Job summary

KPG99 INC in Phoenix, AZ seeks a Senior Big Data Engineer to design and optimize large-scale data pipelines using GCP, Spark, Airflow, and SQL. You will build reliable data models, implement ETL processes, and collaborate with data scientists and product teams.

The ideal candidate has 6+ years of hands-on software development experience with Big Data solutions, strong coding skills in Python and SQL, and a track record of delivering scalable data systems in cloud environments.

Qualifications

• 6+ years of hands-on software development experience with Big Data & Analytics solutions.
• Proficiency in SQL and database design for performance and scalability.
• Experience with GCP services and data pipelines using Big Data technologies (Airflow, PubSub, DataProc).
• Strong OOP, CI/CD, and source control (Git/Jenkins).
• Experience with distributed systems and data modeling for scalability.
• Familiarity with Agile/Scrum methodologies and cloud certifications is a plus.

Skills

Big Data & Analytics

SQL

Python

OOP Concepts

Agile/Scrum

Education

GCP Professional Data Engineer certification

Tools

BigQuery

Airflow

DataProc

PubSub

Kafka

Spark

Hadoop Hive

Shell scripting

Git

Jenkins

XLR

GitHub

GitLab

Bitbucket

SVN

Job description

Python; PySpark; GCP; ETL - Big Data / Data Warehousing; Data Analytics; SQL; Git (GitHub, GitLab, BitBucket, SVN)

• 6+ years of hands-on software development experience with Big Data & Analytics solutionsGCP Cloud - Big Query, Airflow, DataProc, PubSub,Hadoop Hive, Spark, Python, shell scripting, lStrong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.
• Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.Experience of building event processing pipelines with Kafka or GCP PubSub.Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.
• Strong Object-Oriented Programming skills and design patterns.Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).
• lGood knowledge and experience with configuration management tools like GitHubAbility to analyze complex data engineering problems, propose effective solutions, and implement them effectively.Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big DataLooks proactively beyond the obvious for continuous improvement opportunities.
• Communicate effectively with product and cross functional team.
• Willingness to learn new technologies and leverage them to their optimal potential.Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.
• Certifications in cloud platform (GCP Professional Data Engineer) is a plus.