Senior Data Platform Engineer — Lakehouse & AI Pipelines

Tata Consultancy Services; San Jose (CA); On-site; USD 80,000 - 115,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Medical Coverage

Parental Leaves

401K Plan

Performance Bonus

Job summary

Tata Consultancy Services in San Jose, CA seeks a data engineering professional to design end-to-end data solutions using PySpark, SQL, and modern data architectures. You will build pipelines, implement Medallion Architecture, and optimize data platforms for reporting and AI workloads.

You will collaborate with stakeholders to define data models, governance, and security, while driving Microsoft Fabric adoption and CDP initiatives. Strong communication skills required.

Qualifications

• Experience delivering data solutions using PySpark and SQL.
• Design and implement end-to-end data pipelines and data models.
• Strong knowledge of Azure data services and BI tooling.
• Ability to collaborate with stakeholders across teams.

Responsibilities

• Collaborate with stakeholders to translate data needs into scalable technical solutions.
• Design and develop data pipelines using PySpark, SQL, and Medallion Architecture.
• Build Lakehouse and Data Warehouse solutions for reporting and AI workloads.
• Define data models, quality standards, governance, security and performance optimization.
• Provide guidance on Microsoft Fabric adoption and modernization initiatives.
• Design and implement CDP solutions including identity resolution and real-time activation.
• Work with cross-functional teams to deliver high-quality data solutions.
• Support CI/CD, code reviews, testing, deployment and production support.
• Prepare technical documentation and implementation guidelines.
• Stay current with Microsoft Fabric and Azure data technologies.

Skills

PySpark

Data Warehouse

Data Concepts

Data Modelling

Big Data Platforms

Data Engineering

Azure Data Factory

CI/CD

Communication

Azure DevOps

Git

Advanced SQL

Tools

Microsoft Fabric

OneLake

Delta Tables

Power BI

Job description

Tata Consultancy Services in San Jose, CA seeks a data engineering professional to design end-to-end data solutions using PySpark, SQL, and modern data architectures. You will build pipelines, implement Medallion Architecture, and optimize data platforms for reporting and AI workloads.

You will collaborate with stakeholders to define data models, governance, and security, while driving Microsoft Fabric adoption and CDP initiatives. Strong communication skills required.