GCP Data Lead: Build Scalable Big Data Pipelines

HCL Technologies Limited; United States; On-site; USD 150,000 - 190,000; Full time

Job summary

HCL Technologies Limited is seeking a GCP Data Lead in the United States to architect and deliver scalable data pipelines on Google Cloud, leveraging BigQuery, Dataproc, Pub/Sub, and Airflow.

You will collaborate with product and engineering teams, design data models, implement CI/CD for data workflows, and optimize performance across large-scale analytics platforms. Strong Python, SQL, and distributed systems background required.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Computer Engineering, or related technical discipline.
• 5+ years of hands-on software development experience with Big Data and analytics solutions.
• Hands-on experience with Google Cloud Platform technologies (BigQuery, Dataproc, Pub/Sub, and Airflow).
• Strong Python and shell-scripting skills.
• Strong experience designing, developing, and optimizing scalable data pipelines for large-scale data processing, transformation, and analysis.
• Proficiency in SQL and relational database systems, including designing and optimizing data models for performance and scalability.
• Experience building event-driven processing pipelines using Kafka or Google Cloud Pub/Sub.
• Knowledge of distributed, multi-tiered systems and algorithms.
• Strong object-oriented programming skills and knowledge of design patterns.
• Experience with CI/CD pipelines, automated testing frameworks, and source-code management tools such as Jenkins and Git.
• Ability to analyze complex data-engineering problems, propose effective solutions, and implement them successfully.
• Ability and willingness to learn, adopt, and build solutions using established enterprise Big Data frameworks.
• Effective communication with product and cross-functional teams.
• Understanding of SDLC methodologies and familiarity with Agile and Scrum ceremonies.
• Proactive approach to identifying improvement opportunities and willingness to learn and apply new technologies.

Skills

GCP and Big Data

Programming and scripting

Data engineering

SQL and data modelling

Event processing

Distributed systems

Software engineering

CI/CD and source control

Problem solving

Enterprise frameworks

Collaboration

Delivery methodology

Continuous improvement

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

BigQuery

Dataproc

Pub/Sub

Airflow

Hadoop

Hive

Spark

Python

Shell scripting

Jenkins

Git

XLR

Kafka

Job description

HCL Technologies Limited is seeking a GCP Data Lead in the United States to architect and deliver scalable data pipelines on Google Cloud, leveraging BigQuery, Dataproc, Pub/Sub, and Airflow.

You will collaborate with product and engineering teams, design data models, implement CI/CD for data workflows, and optimize performance across large-scale analytics platforms. Strong Python, SQL, and distributed systems background required.