Engineer Software Design

Tata Consultancy Services Limited; Sunnyvale (CA); On-site; USD 100,000 - 120,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Family Leave

401K Plan

Certification & Training Reimbursement

Vacation & Holidays

Job summary

Tata Consultancy Services in Sunnyvale, CA is seeking a Data Engineer to design, build, govern, and optimize data platforms and pipelines to support analytics, reporting, AI/ML, and business operations.

You will develop scalable ETL/ELT processes using GCP, PySpark, Python, Airflow, Hive, and Kafka, automate ingestion, ensure reliability, and participate in incidents and improvements.

Qualifications

• Must have hands-on experience with scalable ETL/ELT pipelines on GCP.
• Proficient in PySpark and Python for data processing.
• Experience building and maintaining Airflow workflows.
• Knowledge of Hive for batch processing and Kafka-based ingestion.
• Understanding of data quality, governance, and monitoring.

Responsibilities

• Design, build, govern, and optimize data platforms, pipelines, and integrations for analytics, reporting, AI/ML, and business operations.

Tools

Google Cloud Platform (GCP)

PySpark

Python

Airflow

Hive

Kafka

Job description

Must Have Technical/Functional Skills

• Google Cloud Platform (GCP)
• PySpark
• Python
• Airflow
• Hive
• Kafka

Data Pipeline Development, Enhancement & Maintenance

• Design, build, and maintain scalable ETL/ELT pipelines.
• Automate data ingestion from multiple data sources.
• Ensure reliable, high-performance, and scalable data movement.
• Develop and optimize PySpark-based data processing frameworks.
• Build and manage Airflow workflows and orchestration pipelines.
• Work with Hive and Kafka-based batch and streaming data solutions.
• Implement data quality, monitoring, governance, and operational controls.
• Support production incidents, performance tuning, and continuous improvement initiatives

Roles & Responsibilities

• The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.

Preferred Experience

• Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
• Strong experience with large-scale GCP Data Engineering solutions.
• Hands-on experience with Airflow DAG development and support
• Experience building and optimizing Spark/PySpark data pipelines.
• Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
• Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable

TCS Employee Benefits Summary:

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Aut& Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

#LI-KR3

Salary Range-$100,000-$120,000 a year

Location

Sunnyvale, CA

Function

TECHNOLOGY

Role

Engineer

ID

438686

Skills

Kafka | Python | ETL Testing | Hive

Salary Range

$100,000 - $120,000