Data Engineer

Tata Consultancy Services; Atlanta (GA); On-site; USD 100,000 - 135,000; Full time

Benefits offered by this job

Discretionary Incentive

Medical Coverage

Dental & Vision

Disability Insurance

Pet Insurance

Commuter Benefits

Certification Reimbursement

Vacation & Holidays

401K Plan

Performance Bonus

College Fund

Student Loan Refinancing

Job summary

Tata Consultancy Services in Atlanta, GA is seeking a data engineer to design, build, and maintain scalable data pipelines and ETL solutions for large-scale analytics.

You'll work with Data Scientists and BI teams, apply Spark, Hadoop, Kafka, and cloud platforms (AWS/GCP), and mentor junior engineers while driving best practices.

Qualifications

• Experience designing scalable data pipelines and ETL solutions.
• Strong SQL querying and data modeling skills.
• In-depth knowledge of Hadoop, Spark, Kafka, Hive, HDFS, Sqoop.
• Proficiency with cloud platforms (AWS/GCP) and data warehousing.
• Experience streaming data with Kafka and Spark Streaming.
• Familiarity with Databricks notebooks and Delta Lake ecosystem.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL solutions.
• Build and optimize data ingestion frameworks for structured and unstructured data.
• Implement data quality, validation, governance processes.
• Develop data infrastructure integrating heterogeneous sources.
• Support analytics, reporting, ML, and BI initiatives.
• Collaborate with Data Scientists and BI teams to curate data.
• Lead and mentor junior data engineers.
• Drive adoption of best practices and standards in Data Eng.

Skills

Databricks

Python

PySpark

Spark SQL

Apache Spark

ETL Development

Hadoop

Kafka

Hive

HDFS

Sqoop

SQL

Data Modeling

Data Ingestion

Data Transformation

Data Cleansing

Data Validation

AWS

GCP

Data Warehouse

Big Data Processing

Kafka Streaming

Spark Streaming

Agile

SCrum

Kanban

SAFe

Tools

Databricks Delta Lake

Delta Live Tables

Unity Catalog

S3

Glue

Lambda

Kinesis/MSK

Tableau

Power BI

Spotfire

Oracle

DB2

BigQuery

Git

Job description

Job Description

• Python and PySpark
• Spark SQL and Apache Spark
• Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop
• Strong SQL querying and data modeling
• Data ingestion, transformation, cleansing, and validation
• Data warehouse and large-scale data processing experience
• Experience with streaming data using Kafka and Spark Streaming
• Agile methodologies (Scrum, Kanban, SAFe)

Must Have Technical/Functional Skills

• Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)
• Python and PySpark
• Spark SQL and Apache Spark
• ETL/Data Pipeline Development
• Big Data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop
• Strong SQL querying and data modeling
• Data ingestion, transformation, cleansing, and validation
• Cloud platforms (AWS/GCP)
• Data warehouse and large-scale data processing experience
• Experience with streaming data using Kafka and Spark Streaming
• Agile methodologies (Scrum, Kanban, SAFe)

Preferred Skills

• Scala / Scala-Spark
• IBM DataStage, Tibco, or similar ETL tools
• AWS services: S3, Glue, Lambda, Kinesis/MSK
• BI Tools: Tableau, Power BI, Spotfire, OAC
• Databases: Teradata, DB2, BigQuery, Mainframe
• Git version control
• Data governance and retention modeling

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL solutions.
• Build and optimize data ingestion frameworks for structured and unstructured data.
• Implement data quality, validation, standardization, and governance processes.
• Develop data infrastructure integrating multiple heterogeneous data sources.
• Support analytics, reporting, machine learning, and business intelligence initiatives.
• Work with Data Scientists and BI teams to prepare and curate data for analytics.
• Develop scalable solutions for Big Data processing and streaming workloads.
• Analyze data and provide actionable insights and recommendations to business stakeholders.
• Gather requirements from business teams and translate them into technical solutions.
• Lead and mentor junior data engineers.
• Collaborate across multiple projects as the Data Engineering SME.
• Drive adoption of Data Engineering best practices and standards.

Salary Range-$100,000-$135,000 a year

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.