Senior GCP Data Engineer – Real-Time Pipelines & AI Tools

Artifintech; Hartford (CT); In loco; USD 110.000 - 170.000; Tempo pieno

Descrizione del lavoro

Artifintech in Hartford, CT seeks an experienced Data Engineer to design and operate data pipelines on GCP, leveraging BigQuery, Cloud Storage, Pub/Sub, and real-time processing with Kafka. You will write Python, optimize complex SQL, and build scalable NoSQL models (MongoDB) while collaborating with offshore/onshore teams in an Agile/SAFe environment.

Strong communication is essential. 7+ years of hands-on experience required; certification in GCP is a plus.

Competenze

• 7+ years hands-on experience with data engineering and GCP data services.
• Experience building robust data pipelines with BigQuery, Pub/Sub, and Cloud Storage.
• Strong knowledge of distributed systems and data engineering best practices.
• Excellent communication and documentation skills.
• Ability to work independently and in fast-paced environments.
• Experience with Agile and SAFe methodologies.
• Experience collaborating with offshore/onshore teams.
• Experience with GitHub, RTC, automation tools is preferred.

Mansioni

• Design and implement robust data pipelines on GCP using BigQuery, Cloud Storage, and Pub/Sub.
• Develop workflows with Cloud Composer for scheduling and orchestration.
• Write clean Python code and advanced SQL queries in BigQuery.
• Build real-time data processing with Apache Kafka.
• Model NoSQL databases like MongoDB with scalable schemas.
• Write shell scripts and perform Linux system administration to support data infra.
• Create data flow diagrams to communicate architecture to stakeholders.
• Develop API integration scripts for service automation and data exchange.
• Troubleshoot data issues with strong analytical skills.

Conoscenze

AI/ML experience

GPT expertise

Gemini experience

Communication skills

Formazione

GCP Professional Data Engineer Certification

Strumenti

BigQuery

Cloud Storage

Pub/Sub

Cloud Composer

Apache Airflow

Python

SQL

Kafka

MongoDB

Shell scripting

Linux

Descrizione del lavoro

Artifintech in Hartford, CT seeks an experienced Data Engineer to design and operate data pipelines on GCP, leveraging BigQuery, Cloud Storage, Pub/Sub, and real-time processing with Kafka. You will write Python, optimize complex SQL, and build scalable NoSQL models (MongoDB) while collaborating with offshore/onshore teams in an Agile/SAFe environment.

Strong communication is essential. 7+ years of hands-on experience required; certification in GCP is a plus.