Senior Databricks Data Engineer

Tata Consultancy Services Limited; Boynton Beach (FL); On-site; USD 100,000 - 130,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Commuter Benefits

Certification & Training Reimbursement

Healthcare: Medical Coverage

Dental & Vision Coverage

Disability Planning & Insurance

401K Plan

Family Support: Leaves

Job summary

Tata Consultancy Services Limited in Boynton Beach, FL seeks a Senior Data Engineer to lead design and implementation of large-scale data pipelines. You will work with Databricks and PySpark to build scalable batch and real-time data solutions and mentor junior engineers.

Role requires 8+ years in data engineering, Delta Lake mastery, and strong SQL/PLSQL skills. On-site role with a focus on cost optimization and reliable data platforms.

Qualifications

• 8+ years in Data Engineering, ETL/ELT and Data Warehousing.
• 5+ years with Databricks, PySpark and Apache Spark.
• Strong Delta Lake, Lakehouse architecture and large-scale data processing expertise.
• Advanced SQL and PL/SQL development skills.
• Experience designing scalable batch and real-time data pipelines.
• Knowledge of dimensional/relational data modeling.
• Experience with Spark SQL, DataFrames, Hive and distributed computing.
• Hands-on AWS services experience (S3, IAM, Glue, Lambda, Redshift).
• CI/CD, Git, DevOps and deployment automation knowledge.
• Excellent analytical and stakeholder management skills; prior leadership experience.

Responsibilities

• Gather and analyze business/technical requirements from stakeholders.
• Design end-to-end data solutions, architectures, and scalable data models.
• Develop and optimize ingestion, transformation, and analytics pipelines with Databricks and PySpark.
• Create and maintain conceptual, logical, and physical data models.
• Lead design activities and document High-Level/Low-Level designs.
• Implement data quality, governance, and performance optimization practices.
• Collaborate with Product Owners, Architects, and BAs to define scope and plans.
• Provide technical leadership and mentoring to onshore/offshore teams.
• Conduct design/code reviews and troubleshoot to ensure quality.
• Drive platform performance, scalability, reliability and cost optimization.
• Support production deployments, issue resolution and root-cause analysis.
• Contribute to technology roadmaps and innovative solutions.

Skills

Databricks

PySpark

Apache Spark

Delta Lake

Lakehouse architecture

Advanced SQL

PL/SQL

ETL/ELT

Data Warehousing

Data modeling

CI/CD

Git

AWS

Tools

Hive

DataFrames

Spark SQL

Data quality

Job description

Job Description

Must Have Technical/Functional Skills

• 8+ years of experience in Data Engineering, ETL/ELT, and Data Warehousing.
• 5+ years of hands-on experience with Databricks, PySpark, and Apache Spark.
• Strong expertise in Delta Lake, Lakehouse Architecture, and large-scale data processing.
• Advanced SQL and PL/SQL development skills.
• Experience designing and implementing scalable batch and real-time data pipelines.
• Strong knowledge of dimensional, relational, and analytical data modeling.
• Experience with Spark SQL, DataFrames, Hive, and distributed computing frameworks.
• Hands-on experience with AWS services, including S3, IAM, Glue, Lambda, and Redshift.
• Strong understanding of Databricks-AWS integrations and cloud-native architectures.
• Experience with data quality, performance tuning, optimization, and troubleshooting.
• Knowledge of CI/CD, Git, DevOps practices, and deployment automation.
• Excellent analytical, problem-solving, communication, and stakeholder management skills.
• Prior experience leading technical teams and mentoring junior engineers.

Roles & Responsibilities

• Gather and analyze business and technical requirements from stakeholders and clients.
• Design end-to-end data solutions, architectures, and scalable data models.
• Develop and optimize data ingestion, transformation, and analytics pipelines using Databricks and PySpark.
• Create and maintain conceptual, logical, and physical data models.
• Lead solution design activities, including High-Level and Low-Level Design documentation.
• Implement data quality, monitoring, governance, and performance optimization best practices.
• Collaborate with Product Owners, Architects, and Business Analysts to define project scope and delivery plans.
• Provide technical leadership, mentoring, and guidance to onshore and offshore teams.
• Conduct design reviews, code reviews, and troubleshooting sessions to ensure solution quality.
• Drive continuous improvements in platform performance, scalability, reliability, and cost optimization.
• Support production deployments, issue resolution, and root cause analysis activities.
• Contribute to technology roadmaps and recommend innovative solutions aligned with business goals.

TCS Employee Benefits Summary:

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Aut& Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

#LI-KR3

Salary Range-$100,000-$130,000 a year

Location

Boynton Beach, FL

Function

TECHNOLOGY

Role

Senior Engineer

ID

435473

Skills

Data Warehouse | SQL | Apache Spark

Salary Range

$100,000 - $130,000