Senior Databricks Architect — Cloud Data Platform Leader

Tata Consultancy Services Limited; New York (NY); On-site; USD 130,000 - 150,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Parental Leaves

401K Plan

Certification & Training Reimbursement

Vacation & Holidays

Job summary

Tata Consultancy Services Limited in New York, NY is seeking an experienced Engineer to lead data architecture and cloud data platform initiatives. You will design scalable Databricks pipelines, collaborate with cross-functional teams, and ensure security, governance and performance across analytic workloads.

The role requires 8+ years in data engineering or architecture, strong SQL, PySpark, and experience with AWS data services.

Qualifications

• Bachelor's degree in CS or equivalent with 8+ years of experience.
• Strong experience in data architecture and large-scale cloud data platforms.
• Proficiency in Databricks, PySpark, SQL and data governance.

Responsibilities

• Architect scalable data processing frameworks using Databricks for analytical workloads.
• Design ETL/ELT pipelines for ingestion, transformation, and distribution of data.
• Implement data governance, security and privacy controls across platforms.
• Develop CI/CD workflows and IaC practices for data pipelines and environments.
• Collaborate with product owners and engineers to translate requirements into architecture.
• Mentor team members on Spark optimization and cloud-native design patterns.

Skills

AWS

Python

Education

Bachelor's degree in computer science or similar field

Tools

Databricks

Unity Catalog

AWS IAM

Job description

Tata Consultancy Services Limited in New York, NY is seeking an experienced Engineer to lead data architecture and cloud data platform initiatives. You will design scalable Databricks pipelines, collaborate with cross-functional teams, and ensure security, governance and performance across analytic workloads.

The role requires 8+ years in data engineering or architecture, strong SQL, PySpark, and experience with AWS data services.