Senior Developer

Tata Consultancy Services; Charlotte (NC); On-site; USD 110,000 - 120,000; Full time

Benefits offered by this job

Discretionary Annual Incentive.

Comprehensive Medical Coverage

Family Support: Parental Leaves

Insurance Options: Auto & Home

Certification & Training Reimbursement

Vacation, Sick Leave & Holidays

Job summary

Tata Consultancy Services in Charlotte seeks a senior data/platform engineer with 10+ years in data engineering and platform architecture. You will design surveillance data processing platforms using Hadoop, Spark and PySpark, govern data flows, and guide cross-functional teams.

Role emphasizes building scalable ETL/ELT pipelines, real-time processing, and strong communication to influence across organizations. Regulatory-focused data platforms experience is valued.

Qualifications

• Deep Expertise in PySpark, including performance tuning and optimization.
• Strong Python development experience in large-scale distributed environment.
• Solid knowledge of Hadoop ecosystem (HDFS, Hive/Impala, YARN).
• Experience designing and governing enterprise data platforms with regulatory considerations.
• Expertise in ELT/ETL, batch and real-time data processing.
• Proficiency in Java, Scala and SQL.
• Understanding of non-functional requirements and production support.

Responsibilities

• Enterprise & Surveillance Architecture.
• Define target state architecture and roadmap for surveillance data platforms leveraging Hadoop, Spark, PySpark and Python.
• Establish standard architectural patterns for alert generation, enrichment, aggregation and reporting pipelines.
• Ensure architecture aligns with enterprise standards and regulatory expectations.
• Solution design and delivery support.
• Translate requirements into scalable technical designs for surveillance platforms.
• Review and approve detailed designs to ensure alignment with intent and standards.
• Provide hands-on architectural guidance during development, testing and implementation.
• Design and standardize Spark/PySpark frameworks for surveillance workflows.
• Modernize and optimize large-scale Hadoop workloads for performance and control.
• Implement audit-friendly lineage and governance patterns.
• Support end-to-end delivery across the software lifecycle.

Skills

PySpark

Python

Hadoop

Java/Scala/SQL

Data architecture

Communication

Education

Bachelor's degree in Computer Science

Tools

Kafka

Airflow

Oozie

Control-M

Job description

Job Description

Must Have Technical/Functional Skills

Primary Skill: Data Engineering, Platform Engineering or architecture roles. Deep Expertise in Pyspark.

Experience: 10+ yrs

Roles & Responsibilities

Bachelor’s or master’s degree in computer science or related field.

Required Hard and Soft Skills / Experience

• Deep Expertise in PySpark, including performance tuning and optimization


• Strong python development experience in large-scale distributed environment


• Solid knowledge of Hadoop ecosystem (HDFS,Hive/Impala, YARN)


• Proven experience designing and governing enterprise, regulatory facing data platforms.


• Expertise in designing data lakes, ELT/ETL pipelines, batch and real time data processing solution


• Proficiency in programming languages such as Java, Scala and SQL


• Strong understanding of non-functional requirements and production support models


• Clear written and verbal communication skills with ability to influence across organizations

Preferred Skills / Experience

• Financial services experience, particularly in Market Surveillance, AML, Fraud, Or Risk Technology


• Experience supporting regulatory or audit facing platforms


• Kafka and Spark Structured streaming exposure


• Familiarity with Orchestration tools(Airflow,Control-M,Oozie)


• Knowledge of data governance, lineage, and data quality controls

Daily Responsibilities/Priorities

• Enterprise & Surveillance Architecture


• Define target state architecture and strategic roadmap for surveillance data processing platforms leveraging Hadoop,Spark, Pyspark and Python


• Establish standard architectural patterns for alert generation, enrichment, aggregation and reporting pipelines.


• Ensure architecture aligns with enterprise technology standards and Surveillance control expectation


• Solution Design and Delivery Support


• Translate surveillance business requirements(e.g, market misconduct detection, regulatory coverage) into scalable technical designs


• Review and approve detailed technical designs, ensuring alignment with functional intent, regulatory requirements, and architectural standards.


• Provide hands-on architectural guidance to engineering teams during development, testing and implementation


• Design and standardization of Spark/Pyspark frameworks supporting surveillance alert generation and enrichment


• Modernization and Optimization of large-scale Hadoop surveillance workloads to improve performance, stability and control coverage


• Implementation of enterprise-consistent architecture patterns enabling audit readiness, lineage, and regulatory traceability


• Support end-to-end delivery across the SDLC, minimizing rework and technical debt

Architecture Governance & Leadership

• Participate in and lead architecture review forums and design walkthroughs


• Mentor senior engineers and promote adoption of Standard framework and best practices


• Influence enterprise surveillance platform strategy through architecture governance

Documentation & Communication

• Maintain high-quality documentation, including business requirement, data definitions and process flows


• Proactively communicate risks, dependencies and potential timeline impact to stakeholders

TCS Employee Benefits Summary

• Discretionary Annual Incentive.


• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.


• Family Support: Maternal & Parental Leaves.


• Insurance Options: Auto & Home Insurance, Identity Theft Protection.


• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.


• Time Off: Vacation, Time Off, Sick Leave & Holidays.


• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Salary Range $110,000-$120,000 a year

Qualifications:

BACHELOR OF COMPUTER SCIENCE