Senior Big Data Tech Lead - GCP, Spark & Flink

Tata Consultancy Services; Phoenix (AZ); On-site; USD 100,000 - 150,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Family Support: Leaves

401K Plan

Certification & Training Reimbursement

Vacation & Holidays

Job summary

Tata Consultancy Services in Phoenix, AZ, is seeking a BigData Tech Lead with 10+ years of experience to design, implement and optimize large-scale data platforms.

You will work with Apache Flink and Beam, SQL, and Google Cloud Platform services (BigQuery, Dataflow, Cloud Composer) to build robust batch and real-time pipelines, mentor engineers, and collaborate with cross-functional teams to deliver high-impact solutions.

Qualifications

• Experience with cloud platforms like GCP and building scalable microservices.
• Proficient in SQL and complex data transformations on large datasets.
• Hands-on with Hadoop/Spark architectures and data processing pipelines.
• Strong programming skills in Java or Python/Scala.
• Experience writing and optimizing Spark/Hive SQL queries.

Responsibilities

• Design and develop optimized data pipelines for batch and real-time processing.
• Lead and mentor data engineering teams on best practices and architecture.
• Collaborate with cross-functional stakeholders to define data solutions.
• Deliver scalable, reliable and maintainable data platforms.

Skills

Apache Flink

Apache Beam

SQL

Hadoop

Spark

Core Java/Python/Scala

UNIX shell scripting

Data pipelines

GCP (BigQuery)

Dataflow

Cloud Composer

GitHub/Bitbucket

CI/CD

Tools

GitHub/Bitbucket

CI/CD pipelines

BigQuery

Dataflow

Cloud Composer

Job description

Tata Consultancy Services in Phoenix, AZ, is seeking a BigData Tech Lead with 10+ years of experience to design, implement and optimize large-scale data platforms.

You will work with Apache Flink and Beam, SQL, and Google Cloud Platform services (BigQuery, Dataflow, Cloud Composer) to build robust batch and real-time pipelines, mentor engineers, and collaborate with cross-functional teams to deliver high-impact solutions.