Senior Data Engineer — Spark & Cloud Pipelines

Tata Consultancy Services; Irving (TX); On-site; USD 100,000 - 130,000; Full time

Job summary

Tata Consultancy Services is seeking a Data Engineer in Irving, TX to design, build, and optimize scalable data pipelines using PySpark, Spark SQL, and HiveQL. You will work across AWS, Azure, or GCP data platforms to ensure reliable, fast pipelines that feed BI and analytics teams.

The role emphasizes performance tuning, data modeling, and collaboration with data scientists and analysts to translate business requirements into robust data solutions.

Qualifications

• Proficient in Python and PySpark for large-scale data processing.
• Strong SQL/HiveQL knowledge and data modeling basics.
• Experience with cloud data platforms and data lake architectures.

Responsibilities

• Design, build, and optimize scalable ETL/ELT data pipelines using PySpark and Spark SQL.
• Manage cloud data infrastructure on AWS, Azure, or GCP.
• Tune performance, resolve bottlenecks, and reduce data skew in Spark jobs.
• Ingest high-volume data from relational and unstructured sources.
• Automate data workflows with Airflow or native schedulers.
• Collaborate with data scientists and analysts to meet business needs.

Skills

Python

PySpark

Spark SQL

HiveQL

Education

Bachelor's degree in Computer Science

Tools

AWS EMR

Azure Databricks

GCP

Apache Airflow

Job description

Tata Consultancy Services is seeking a Data Engineer in Irving, TX to design, build, and optimize scalable data pipelines using PySpark, Spark SQL, and HiveQL. You will work across AWS, Azure, or GCP data platforms to ensure reliable, fast pipelines that feed BI and analytics teams.

The role emphasizes performance tuning, data modeling, and collaboration with data scientists and analysts to translate business requirements into robust data solutions.