BigData Tech lead

Tata Consultancy Services; Phoenix (AZ); On-site; USD 100,000 - 150,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Family Support: Leaves

401K Plan

Certification & Training Reimbursement

Vacation & Holidays

Job summary

Tata Consultancy Services in Phoenix, AZ, is seeking a BigData Tech Lead with 10+ years of experience to design, implement and optimize large-scale data platforms.

You will work with Apache Flink and Beam, SQL, and Google Cloud Platform services (BigQuery, Dataflow, Cloud Composer) to build robust batch and real-time pipelines, mentor engineers, and collaborate with cross-functional teams to deliver high-impact solutions.

Qualifications

• Experience with cloud platforms like GCP and building scalable microservices.
• Proficient in SQL and complex data transformations on large datasets.
• Hands-on with Hadoop/Spark architectures and data processing pipelines.
• Strong programming skills in Java or Python/Scala.
• Experience writing and optimizing Spark/Hive SQL queries.

Responsibilities

• Design and develop optimized data pipelines for batch and real-time processing.
• Lead and mentor data engineering teams on best practices and architecture.
• Collaborate with cross-functional stakeholders to define data solutions.
• Deliver scalable, reliable and maintainable data platforms.

Skills

Apache Flink

Apache Beam

SQL

Hadoop

Spark

Core Java/Python/Scala

UNIX shell scripting

Data pipelines

GCP (BigQuery)

Dataflow

Cloud Composer

GitHub/Bitbucket

CI/CD

Tools

GitHub/Bitbucket

CI/CD pipelines

BigQuery

Dataflow

Cloud Composer

Job description

Job Description

Role - BigData Tech lead

Experience Required -10+ Years

Must Have Technical/Functional Skills

Apache Flink, Apache Beam , SQL

Google Cloud Platform (Big Query, Dataflow, Cloud Composer)

Roles & Responsibilities

• Knowledge of cloud platforms like GCP, building Microservices and scalable solutions, will be an advantage
• Experience with GitHub/Bitbucket and leveraging CI/CD pipelines
• Excellent technical and analytical aptitude
• Good communication skills
• Excellent Project management skills.
• Results driven
• Hands-on experience with programming using Core Java or Python/Scala
• Expert on Hadoop and Spark Architecture and its working principle
• Hands-on experience on writing and understanding complex SQL(Hive/PySpark-dataframes), optimizing joins while processing huge amount of data
• Experience in UNIX shell scripting
• Ability to design and develop optimized Data pipelines for batch and real time data processing
• Should have experience in analysis, design, development, testing, and implementation of system applications
• Demonstrated ability to develop and document technical and functional specifications and analyze software and system processing flows.

Base Salary Range : $100,000 to $150,000 Per Annum

TCS Employee Benefits Summary

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.