Senior Data Engineer: PySpark, Airflow & GCP

Tata Consultancy Services Limited

Sunnyvale (CA)

On-site

USD 110,000 - 150,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Support: Parental Leaves
Aut & Home Insurance
Commuter Benefits & Training Reimburse
Vacation, Sick Leave & Holidays
401K Plan, Bonus, College Fund

Job summary

Tata Consultancy Services in Sunnyvale, CA is seeking a Data Engineer to design, build, and optimize scalable ETL/ELT pipelines using PySpark and GCP. You will automate ingestion from multiple sources, ensure high performance, and manage Airflow workflows for batch and streaming data.

You will work with Hive and Kafka, implement data quality and governance, support incidents, and contribute to continuous improvements of data platforms and analytics readiness for AI/ML and business operations.

Qualifications

  • Must have hands-on experience with scalable data pipelines and cloud data platforms.
  • Proficiency in PySpark, Python, Airflow, Hive and Kafka is required.
  • Experience with GCP data services and large-scale data processing is essential.

Responsibilities

  • Design, build, and maintain scalable ETL/ELT pipelines.
  • Automate data ingestion from multiple data sources and ensure high performance.
  • Develop and optimize PySpark-based data processing frameworks.
  • Build and manage Airflow workflows and orchestration pipelines.
  • Work with Hive and Kafka-based batch and streaming data solutions.
  • Implement data quality, monitoring, governance, and operational controls.
  • Support production incidents, performance tuning, and continuous improvements.

Tools

GCP
PySpark
Python
Airflow
Hive
Kafka

Job description

Tata Consultancy Services in Sunnyvale, CA is seeking a Data Engineer to design, build, and optimize scalable ETL/ELT pipelines using PySpark and GCP. You will automate ingestion from multiple sources, ensure high performance, and manage Airflow workflows for batch and streaming data.

You will work with Hive and Kafka, implement data quality and governance, support incidents, and contribute to continuous improvements of data platforms and analytics readiness for AI/ML and business operations.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer - GCP, PySpark, Airflow & Kafka
Data Engineer - GCP, PySpark, Airflow & Kafka

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Leave
+3
Data Engineer - Scalable ETL, PySpark & Airflow on GCP
Data Engineer - Scalable ETL, PySpark & Airflow on GCP

Tata Consultancy Services Limited • Herndon (VA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive.
Comprehensive Medical Coverage:Medical
Dental & Vision
+7
Senior Data Engineer: Build Scalable ETL Pipelines on GCP
Senior Data Engineer: Build Scalable ETL Pipelines on GCP

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 110,000 - 150,000
Discretionary Annual Incentive
Comprehensive Medical Coverage: Health
Dental & Vision
+6
AWS Data Engineer | Spark, PySpark, Hive & Airflow
AWS Data Engineer | Spark, PySpark, Hive & Airflow

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 90,000 - 110,000
Senior GCP Data Engineer: Big Data Pipelines & Analytics
Senior GCP Data Engineer: Big Data Pipelines & Analytics

Tata Consultancy Services Limited • Phoenix (AZ)

On-site
USD 100,000 - 120,000
Medical Coverage
Dental & Vision
401K Plan
+2
Spark Data Engineer (PySpark) - Cloud ETL & Pipelines
Spark Data Engineer (PySpark) - Cloud ETL & Pipelines

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 90,000 - 120,000
Discretionary Annual Incentive
Medical Coverage (Health, Dental & Vis
401K Plan
+1
Senior Data Platform Engineer - Azure, PySpark & Lakehouse
Senior Data Platform Engineer - Azure, PySpark & Lakehouse

Tata Consultancy Services Limited • San Jose (CA)

On-site
USD 80,000 - 115,000
Annual Incentive
Medical Coverage
Parental Leave
+5
Senior PySpark Data Engineer - Hadoop & ETL
Senior PySpark Data Engineer - Hadoop & ETL

Tata Consultancy Services Limited • Charlotte (NC)

On-site
USD 100,000 - 110,000
Discretionary annual incentive
Comprehensive medical coverage
Family leaves
+4
Cloud Data Platform Engineer (Azure, PySpark, CDP)
Cloud Data Platform Engineer (Azure, PySpark, CDP)

Tata Consultancy Services Limited • San Jose (CA)

On-site
USD 80,000 - 115,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Support & Leaves
+1
Senior Data Engineer | Python & PySpark Cloud Pipelines
Senior Data Engineer | Python & PySpark Cloud Pipelines

MACHINE LEARNING TECHNOLOGIES LLC • Atlanta (GA)

Hybrid
USD 93,000 - 133,000