Data Engineer - GCP

Tata Consultancy Services Limited; Toronto; On-site; CAD 110,000 - 150,000; Full time

Job summary

Tata Consultancy Services Canada Inc. is seeking an experienced GCP Data Engineer with 8-10 years of expertise in designing, developing, and optimizing large-scale data platforms and cloud-native data solutions.

Strong hands‑on experience in PySpark, Python, BigQuery, Dataproc, Kafka, Airflow, SQL, and GCP services, with proven success in building scalable ETL/ELT pipelines, batch and real-time data processing frameworks, and enterprise data warehouse solutions.

Qualifications

• 8-10 years of experience in designing, developing and optimizing large-scale data platforms on cloud
• Hands-on with PySpark, Python, BigQuery, Dataproc, Kafka, Airflow, SQL and GCP services
• Experience building scalable ETL/ELT pipelines and real-time data processing frameworks

Responsibilities

• Design and develop scalable ETL/ELT pipelines using PySpark, Python, and Spark SQL.
• Build and optimize data processing solutions on GCP using Dataproc, BigQuery, and GCS.
• Develop and maintain Kafka-based streaming pipelines for real-time ingestion and processing.
• Create and manage BigQuery datasets, tables, views, and complex queries.
• Implement workflow orchestration with Cloud Composer (Airflow).
• Manage cloud security through IAM roles and access policies.
• Collaborate with Data Architects, DevOps, Data Scientists and stakeholders.
• Provide production support, troubleshooting and performance optimization.

Skills

PySpark

Python

BigQuery

Dataproc

Kafka

Airflow

SQL

GCP services

Tools

Git

CI/CD

Terraform

Shell scripting

Cloud Composer

Job description

Tata Consultancy Services (TCS) is an equal opportunity employer, andembraces diversity in race, nationality, ethnicity, gender, age, physicalability, neurodiversity, and sexual orientation, to create a workforce thatreflects the societies we operate in. Our continued commitment to Culture andDiversity is reflected in our people stories across our workforce and implementedthrough equitable workplace policies and processes.

Tata Consultancy Services (BSE: 532540, NSE: TCS) is thetechnology partner of choice for industry-leading organizations worldwide.Since its inception in 1968, TCS has upheld the highest standards ofinnovation, engineering excellence and customer service.

It has set an aspiration to become the world's largestAI-led technology services company and is enabling its clients to transformthemselves across the full AI stack, from infrastructure to intelligence.

Rooted in the heritage of the Tata Group, TCS is focused oncreating long term value for its clients, its investors, its employees, and thecommunity at large. With a highly skilled workforce spread across 56 countriesand 194 service delivery centers across the world, the company has beenrecognized as a top employer in six continents. With the ability to rapidlyapply and scale new technologies, the company has built long term partnershipswith its clients. Many of these relationships have endured into decades andnavigated every technology cycle, from mainframes in the 1970s to artificialintelligence today.

Role Description:

Experienced GCP Data Engineer with 8-10 years of expertise in designing,developing, and optimizing large-scale data platforms and cloud-native datasolutions. Strong hands‑on experience in PySpark, Python, BigQuery, Dataproc,Kafka, Airflow, SQL, and GCP services, with proven success in building scalableETL/ELT pipelines, batch and real-time data processing frameworks, andenterprise data warehouse solutions. Adept at data modeli ng, performance tuning,cloud security, automation, and collaborating with cross-functional teams todeliver high-quality data solutions that support analytics, reporting, andbusiness intelligence initiatives.

Key Skills & Responsibilities

• 1. Design and develop scalable ETL/ELT pipelines using PySpark, Python, andSpark SQL.
• 2. Build and optimize large-scale data processing solutions on GCP usingDataproc, BigQuery, and GCS.
• 3. Develop and maintain Kafka-based streaming pipelines for real-time data ingestionand processing.
• 4. Create, manage, and optimize BigQuery datasets, tables, views, storedprocedures, and complex SQL queries.
• 5. Implement workflow orchestration and scheduling using Cloud Composer(Airflow).
• 6. Manage cloud security through IAM roles, service accounts, and accesscontrol policies following best practices.
• 7. Work extensively with Hive, Hadoop, Spark, and distributed data processingtechnologies.
• 8. Develop automation and monitoring solutions using Python and Shell scriptingto improve operational efficiency.
• 9. Utilize Git, CI/CD pipelines, and Terraform for code management, deployment, and infrastructure automation.
• 10. Collaborate with Data Architects, DevOps teams, Data Scientists, andbusiness stakeholders while providing production support, troubleshooting, andperformance optimization.

Tata Consultancy Services Canada Inc. iscommitted to meeting the accessibility needs of all individuals in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). Should you require accommodation during the recruitment and selection process, please inform Human Resources.