Hadoop Developer

Tata Consultancy Services; Charlotte (NC); On-site; USD 100,000 - 120,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Medical Coverage: Medical & Health, 9/

Dental & Vision

401K Plan

Parental Leaves

Certification & Training Reimbursement

Job summary

Tata Consultancy Services Limited in Charlotte, NC seeks an Engineer to lead data platform architectures and real-time streaming initiatives. The role focuses on building scalable data pipelines while ensuring security and enterprise standards.

You will mentor junior engineers, collaborate with product teams, and apply Hadoop, PySpark, Python, and Kafka expertise to deliver high-performance solutions in an agile environment.

Qualifications

• Minimum 10 years of experience.
• Strong experience with Hadoop ecosystem (HDFS, Hive, Spark).
• Proficiency in PySpark for distributed data processing.
• Hands-on experience with Apache Kafka for real-time streaming.

Responsibilities

• Define end-to-end architecture for data platforms and streaming systems.
• Design and implement data ingestion, transformation, and processing pipelines.
• Mentor junior developers and data engineers; lead agile ceremonies and design discussions.

Skills

Hadoop ecosystem

PySpark

Python

Apache Kafka

Education

Bachelor of Computer Science

Tools

Docker

Kubernetes

AWS

Azure

GCP

Job description

Job Description

Must Have Technical/Functional Skills

Primary Skill: Hadoop ecosystem (HDFS, Hive, Spark), PySpark, Python, Apache Kafka

Experience: Minimum 10 years

Roles & Responsibilities

Architectural Leadership:

• Define end-to-end architecture for data platforms, streaming systems, and web applications.
• Ensure alignment with enterprise standards, security, and compliance requirements.
• Evaluate emerging technologies and recommend adoption strategies.

Data Engineering:

• Design and implement data ingestion, transformation, and processing pipelines using Hadoop, PySpark, and related tools.
• Optimize ETL workflows for large-scale datasets and real-time streaming.
• Integrate Apache Kafka for event-driven architectures and messaging.

Collaboration & Leadership:

• Work closely with product owners, business analysts, and DevOps teams.
• Mentor junior developers and data engineers.
• Participate in agile ceremonies, code reviews, and design discussions.

Required Skills & Qualifications:

Technical Expertise:

• Strong experience with Hadoop ecosystem (HDFS, Hive, Spark).
• Proficiency in PySpark for distributed data processing.
• Advanced programming skills in Python.
• Hands-on experience with Apache Kafka for real-time streaming.

Architectural Skills:

• Expertise in designing scalable, secure, and high-performance systems.
• Familiarity with microservices, API design, and cloud-native architectures.

Additional Skills:

• Knowledge of CI/CD pipelines, containerization (Docker/Kubernetes).
• Exposure to cloud platforms (AWS, Azure, GCP).

Soft Skills:

• Strong analytical and problem-solving abilities.
• Excellent communication and leadership skills.
• Ability to thrive in a fast-paced, agile environment.

TCS Employee Benefits Summary:

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range: $100,000-$120,000 a year

Location

Charlotte, NC

Function

TECHNOLOGY

Role

Engineer

ID

436968

Skills

Big Data | Hadoop

Salary Range

$100,000 - $120,000

Desired Candidate Profile

Qualifications

BACHELOR OF COMPUTER SCIENCE