GCP Data Engineer

IntoneCCM; Hartford (CT); On-site; USD 83,000 - 165,000; Part time

Job summary

CVS Health in Hartford, CT is seeking a contract Data Engineer to design and implement robust data pipelines on GCP, leveraging BigQuery, Pub/Sub, Cloud Storage, and real-time processing with Kafka.

Responsibilities include building and managing workflows with Cloud Composer, writing scalable Python code, and crafting complex SQL queries. AI tool experience (GPT/Gemini) and offshore/onshore collaboration are valued.

Qualifications

• Minimum 7 years hands-on experience with GCP data services.
• Minimum 3 years hands-on experience with data pipelines, Cloud Composer, Python, BigQuery SQL, Apache Kafka, MongoDB, Shell scripting, data profiling, API integration, and data flow documentation.
• Experience with AI tools such as GPT and Gemini with clear business use cases on resume.
• Strong understanding of distributed systems and data engineering best practices.
• Excellent communication and documentation skills.
• Ability to work independently and collaboratively in a fast-paced environment.
• Experience in Agile methodology.
• Experience with onshore/offshore support model, collaborating with offshore teams.
• Preferred: GCP Professional Data Engineer Certification.
• Preferred: SAFe methodology experience.
• Preferred: Experience with GitHub, RTC, and automation tools.

Responsibilities

• Design and implement robust data pipelines on GCP using BigQuery, Cloud Storage and Pub/Sub.
• Develop and manage workflows with Cloud Composer for scheduling and orchestration.
• Write clean, scalable Python code for data processing tasks.
• Craft complex BigQuery SQL queries with window functions and performance tuning.
• Build and maintain real-time data processing with Apache Kafka.
• Model and manage MongoDB databases with scalable schema design.
• Use Shell scripting and perform Linux administration tasks supporting data infra.
• Conduct data profiling and implement validation for data quality.
• Develop API integration scripts for service automation and data exchange.
• Troubleshoot data issues with strong analytical skills, communicate findings.
• Create data flow diagrams to document architecture and pipelines for stakeholders.

Skills

GCP data services

Cloud Composer

Python

BigQuery SQL

Apache Kafka

MongoDB

Shell scripting

Data profiling

API integration

Data flow documentation

GPT/Gemini AI tools

Distributed systems

Agile methodology

Communication skills

Documentation

Independent / collaborative work

Onshore/offshore support

GitHub/RTC/automation tools

Tools

BigQuery

Cloud Storage

Pub/Sub

Airflow

Job description

This is a contract position with CVS based in Hartford, Connecticut. The role requires onsite presence three days per week and offers the opportunity to transition to a permanent position. A ROPES assessment is required for this opportunity.

Responsibilities

• Design and implement robust data pipelines using Google Cloud Platform (GCP) services such as BigQuery, Cloud Storage, and Pub/Sub
• Develop and manage workflows using Cloud Composer (Apache Airflow) for efficient scheduling and orchestration
• Write clean, efficient, and scalable code in Python, leveraging advanced programming techniques
• Craft complex SQL queries in BigQuery, including window functions, CTEs, and performance tuning strategies
• Build and maintain real-time data processing systems using Apache Kafka
• Model and manage NoSQL databases, particularly MongoDB, with a focus on scalable schema design
• Utilize Shell scripting and perform Linux system administration tasks to support data infrastructure
• Conduct data profiling and implement validation techniques to ensure data quality and integrity
• Develop and maintain API integration scripts for seamless service automation and data exchange
• Troubleshoot and resolve data-related issues with strong analytical and problem-solving skills
• Create and maintain data flow diagrams to clearly communicate architecture and pipeline logic to stakeholders

Qualifications

• Required: Minimum 7 years of proven hands-on experience with GCP data services
• Required: Minimum 3 years of proven hands-on experience with the core technical responsibilities listed above (data pipelines, Cloud Composer, Python, BigQuery SQL, Apache Kafka, MongoDB, Shell scripting, data profiling, API integration, and data flow documentation)
• Required: Demonstrated experience with AI tools such as GPT and Gemini, with clear business use cases highlighted on resume
• Required: Strong understanding of distributed systems and data engineering best practices
• Required: Excellent communication and documentation skills
• Required: Ability to work independently and collaboratively in a fast-paced environment
• Required: Experience in Agile methodology
• Required: Experience working in an onshore/offshore support model, collaborating with offshore teams
• Preferred: GCP Google Cloud Professional Data Engineer Certification
• Preferred: Experience with SAFe methodology
• Preferred: Experience working with GitHub, RTC, and automation tools