Hadoop developer

Tata Consultancy Services; Charlotte (NC); On-site; USD 95,000 - 115,000; Full time

Benefits offered by this job

Annual incentive

Medical coverage

Parental leaves

Insurance options

Certification & training

Job summary

Tata Consultancy Services in Charlotte, NC is seeking an experienced Hadoop Spark Developer with 10+ years of Big Data expertise, including Java, Spark, Hadoop, and Hive. Responsibilities include designing, developing, and maintaining large-scale data processing solutions and collaborating with stakeholders.

The role emphasizes building ETL/ELT pipelines, real-time streaming with Kafka, data quality frameworks, and code reviews.

Qualifications

• 10+ years of experience in Hadoop/Spark ecosystem.
• Proficient in Spark, Java, Hive, SQL.
• Unix, Kafka, Python skills.

Responsibilities

• Design, develop, and maintain scalable Big Data solutions using Hadoop and Spark.
• Build and optimize ETL/ELT pipelines using Java, Spark, Hive, and Python.
• Process and analyze large datasets in distributed environments.
• Develop high-performance Spark jobs and optimize existing workloads.
• Create and manage Hive tables, partitions, views, and complex queries.
• Implement data quality, data validation, and reconciliation frameworks.
• Perform code reviews and ensure adherence to coding standards and best practices.
• Utilize Microsoft Copilot to accelerate development, automate code generation, troubleshooting, documentation, and testing activities.
• Strong experience building both batch and real-time streaming applications with Kafka.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and DevOps teams.
• Troubleshoot production issues and perform root cause analysis.
• Design data ingestion frameworks for structured, semi-structured, and unstructured data.
• Participate in Agile ceremonies including sprint planning, estimation, and retrospectives.
• Mentor junior developers and provide technical leadership.

Skills

Spark

Java

Hive

SQL

Unix

Kafka

Python

Education

Bachelor's degree

Job description

Job Description

Must Have Technical/Functional Skills

• Primary Skill: Spark, Java, Hive, SQL
• Secondary: Unix, Kafka, Python
• Experience: 10+ years

Roles & Responsibilities

We are seeking a highly experienced Hadoop Spark Developer with 10+ years of expertise in Big Data technologies, including Java, Spark, Hadoop Ecosystem and Hive. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining large-scale data processing solutions. Experience with Microsoft Copilot for AI-assisted development and productivity enhancement is highly desirable. The developer should hold a bachelor's or master's degree. The candidate should possess strong analytical skills, hands-on experience in distributed data processing, and the ability to work closely with business stakeholders, architects, and data engineering teams.

• Design, develop, and maintain scalable Big Data solutions using Hadoop and Spark.
• Build and optimize ETL/ELT pipelines using Java, Spark, Hive, and Python.
• Process and analyze large datasets in distributed environments.
• Develop high-performance Spark jobs and optimize existing workloads.
• Create and manage Hive tables, partitions, views, and complex queries.
• Implement data quality, data validation, and reconciliation frameworks.
• Perform code reviews and ensure adherence to coding standards and best practices.
• Utilize Microsoft Copilot to accelerate development, automate code generation, troubleshooting, documentation, and testing activities.
• Strong experience building both batch and real-time streaming applications with Kafka.
• Collaborate with Data Architects, Data Scientists, Business Analysts, and DevOps teams.
• Troubleshoot production issues and perform root cause analysis.
• Design data ingestion frameworks for structured, semi-structured, and unstructured data.
• Participate in Agile ceremonies including sprint planning, estimation, and retrospectives.
• Mentor junior developers and provide technical leadership.

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
• Salary Range: $95,000 - 115,000 a year

Qualifications:

• BACHELOR OF COMPUTER SCIENCE