Senior Data Engineer

Tata Consultancy Services Limited

Sunnyvale (CA)

On-site

USD 110,000 - 150,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Support: Parental Leaves
Aut & Home Insurance
Commuter Benefits & Training Reimburse
Vacation, Sick Leave & Holidays
401K Plan, Bonus, College Fund

Job summary

Tata Consultancy Services in Sunnyvale, CA is seeking a Data Engineer to design, build, and optimize scalable ETL/ELT pipelines using PySpark and GCP. You will automate ingestion from multiple sources, ensure high performance, and manage Airflow workflows for batch and streaming data.

You will work with Hive and Kafka, implement data quality and governance, support incidents, and contribute to continuous improvements of data platforms and analytics readiness for AI/ML and business operations.

Qualifications

  • Must have hands-on experience with scalable data pipelines and cloud data platforms.
  • Proficiency in PySpark, Python, Airflow, Hive and Kafka is required.
  • Experience with GCP data services and large-scale data processing is essential.

Responsibilities

  • Design, build, and maintain scalable ETL/ELT pipelines.
  • Automate data ingestion from multiple data sources and ensure high performance.
  • Develop and optimize PySpark-based data processing frameworks.
  • Build and manage Airflow workflows and orchestration pipelines.
  • Work with Hive and Kafka-based batch and streaming data solutions.
  • Implement data quality, monitoring, governance, and operational controls.
  • Support production incidents, performance tuning, and continuous improvements.

Tools

GCP
PySpark
Python
Airflow
Hive
Kafka

Job description

  • Design, build, and maintain scalable ETL/ELT pipelines.
  • Automate data ingestion from multiple data sources.
  • Ensure reliable, high-performance, and scalable data movement.
  • Develop and optimize PySpark-based data processing frameworks.
  • Build and manage Airflow workflows and orchestration pipelines.
  • Work with Hive and Kafka-based batch and streaming data solutions.
  • Implement data quality, monitoring, governance, and operational controls.
  • Support production incidents, performance tuning, and continuous improvement initiatives
  • The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.
Must Have Technical/Functional Skills
  • Google Cloud Platform (GCP)
  • PySpark
  • Python
  • Airflow
  • Hive
  • Kafka
Data Pipeline Development, Enhancement & Maintenance
  • Design, build, and maintain scalable ETL/ELT pipelines.
  • Automate data ingestion from multiple data sources.
  • Ensure reliable, high-performance, and scalable data movement.
  • Develop and optimize PySpark-based data processing frameworks.
  • Build and manage Airflow workflows and orchestration pipelines.
  • Work with Hive and Kafka-based batch and streaming data solutions.
  • Implement data quality, monitoring, governance, and operational controls.
  • Support production incidents, performance tuning, and continuous improvement initiatives
Roles & Responsibilities
  • The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.
Preferred Experience
  • Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
  • Strong experience with large-scale GCP Data Engineering solutions.
  • Hands-on experience with Airflow DAG development and support
  • Experience building and optimizing Spark/PySpark data pipelines.
  • Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
  • Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable
TCS Employee Benefits Summary
  • Discretionary Annual Incentive.
  • Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
  • Family Support: Maternal & Parental Leaves.
  • Insurance Options: Aut& Home Insurance, Identity Theft Protection.
  • Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
  • Time Off: Vacation, Time Off, Sick Leave & Holidays.
  • Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
Salary Range -$110,000-$150,000 a year
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 110,000 - 150,000
Discretionary Annual Incentive
Comprehensive Medical Coverage: Health
Dental & Vision
+6
Engineer Software Design
Engineer Software Design

Tata Consultancy Services Limited • Herndon (VA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive.
Comprehensive Medical Coverage:Medical
Dental & Vision
+7
Engineer Software Design
Engineer Software Design

Tata Consultancy Services Limited • Sunnyvale (CA)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family Leave
+3
PySpark Data Engineer
PySpark Data Engineer

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 90,000 - 120,000
Discretionary Annual Incentive
Medical Coverage (Health, Dental & Vis
401K Plan
+1
Senior PySpark Data Developer
Senior PySpark Data Developer

Tata Consultancy Services Limited • New York (NY)

On-site
USD 110,000 - 170,000
Discretionary Annual Incentive
Medical Coverage
Parental Leaves
+1
Senior Data Engineer: Cloud Pipelines, Analytics & ML Data
Senior Data Engineer: Cloud Pipelines, Analytics & ML Data

Compunnel Inc. • Charlotte (NC)

On-site
USD 120,000 - 150,000
Gcp Data Engineer
Gcp Data Engineer

Tata Consultancy Services Limited • Charlotte (NC)

On-site
USD 100,000 - 120,000
Discretionary Annual Incentive
Comprehensive Medical Coverage
Family & Parental Leaves
+4
Senior Data Engineer
Senior Data Engineer

Diverse Lynx, LLC • Oregon

Hybrid
USD 120,000 - 150,000
Senior Data Engineer
Senior Data Engineer

Compunnel Inc. • Charlotte (NC)

On-site
USD 120,000 - 150,000
Data Engineer
Data Engineer

Tata Consultancy Services Limited • Irving (TX)

On-site
USD 125,000 - 140,000