Ai Data Engineer

Tata Consultancy Services; Tampa (FL); On-site; USD 90,000 - 115,000; Full time

Job summary

Tata Consultancy Services Limited in Tampa, FL seeks an Engineer to design, develop and maintain scalable end-to-end data pipelines. You will build data processing solutions using Python, SQL and PySpark, integrating data from databases, APIs, and files for AI/ML and GenAI applications.

You will implement data-quality checks, optimize Spark workloads, and collaborate with AI/ML engineers and business stakeholders. A strong CI/CD/DevOps mindset and production support expertise are required.

Qualifications

• Bachelor's degree in Computer Science or equivalent.
• Strong hands-on experience with Python and SQL.
• Experience designing and developing ETL/ELT pipelines for large-scale data processing.
• Experience with AWS data services and Spark/PySpark.

Responsibilities

• Design, develop, and maintain scalable end-to-end ETL/ELT and data pipelines.
• Build data processing and transformation solutions using Python, SQL and PySpark.
• Develop batch and streaming pipelines to integrate data from databases, APIs, files and enterprise source systems.
• Implement data-quality checks, validation, reconciliation, deduplication and exception handling.
• Optimize data pipelines, SQL queries and Spark workloads for high-volume processing.
• Prepare and transform datasets required for AI/ML and Generative AI applications.
• Develop data ingestion and preparation pipelines supporting LLM/RAG solutions and vector databases.
• Collaborate with AI/ML engineers, application teams and business stakeholders on downstream data requirements.
• Implement automated deployment and monitoring through CI/CD and DevOps practices.
• Support production deployments, monitoring, troubleshooting and root-cause analysis for data-related issues.

Skills

Python

SQL

Data engineering

PySpark

Spark

Airflow

Data quality

Batch processing

CI/CD

Docker

Kubernetes

Education

Bachelor's degree in Computer Science

Tools

Apache Airflow

Kafka

Docker

Kubernetes

Job description

Job Description

Must Have Technical/Functional Skills

• *Strong hands-on experience in Python, SQL and data engineering.
• *Experience designing and developing ETL/ELT pipelines for large-scale data processing.
• *Strong knowledge of PySpark / Spark for data transformation and processing.
• *Experience with AWS data services such as Glue, S3, Redshift and related cloud services.
• *Hands-on experience with Apache Airflow or equivalent orchestration tools.
• *Strong understanding of data transformation, cleansing, validation, reconciliation and data-quality frameworks.
• *Experience working with structured, semi-structured and unstructured data.
• *Knowledge of batch and streaming data pipelines, preferably Kafka.
• *Experience building data pipelines supporting AI/ML, GenAI, LLM and RAG use cases.
• *Experience with CI/CD, Git, Docker and Kubernetes/EKS for production deployments.
• *Good understanding of data modeling, performance optimization and production support.

Roles & Responsibilities

• Design, develop and maintain scalable end-to-end ETL/ELT and data pipelines.
• Build data processing and transformation solutions using Python, SQL and PySpark.
• Develop batch and streaming pipelines to integrate data from databases, APIs, files and enterprise source systems.
• Implement data-quality checks, validation, reconciliation, deduplication and exception handling.
• Optimize data pipelines, SQL queries and Spark workloads for high-volume processing.
• Prepare and transform datasets required for AI/ML and Generative AI applications.
• Develop data ingestion and preparation pipelines supporting LLM/RAG solutions and vector databases.
• Collaborate with AI/ML engineers, application teams and business stakeholders on downstream data requirements.
• Implement automated deployment and monitoring through CI/CD and DevOps practices.
• Support production deployments, monitoring, troubleshooting and root-cause analysis for data-related issues.

Salary Range: $90,000 to $115,000 per year

Location

Tampa, FL

Function

TECHNOLOGY

Role

Engineer

ID

436563

Skills

Artificial Intelligence

Salary Range

$90,000 - $115,000

Desired Candidate Profile

Qualifications

BACHELOR OF COMPUTER SCIENCE