Databricks Solution Architect: Cloud Data Lakehouse Leader

Tata Consultancy Services; Irving (TX); On-site; USD 125,000 - 140,000; Full time

Job summary

Tata Consultancy Services in Irving, TX seeks a Databricks Solution Architect to design end-to-end data solutions, guide teams, and promote best practices in data engineering and ML workflows.

Lead PoC developments, evaluate platform features, ensure security and compliance, and plan migrations from on-prem to cloud Databricks. Strong PySpark, Python, and cloud experience are required to succeed.

Qualifications

• Proven expertise in implementing Lakehouse and Delta Lake using Databricks.
• Strong PySpark and Python experience.
• Databricks (AWS) Platform Architect Certification.
• Familiarity with ML Ops/LLM Ops and distributed systems.
• Experience with Big Data platform like Cloudera Hadoop and Could platforms like AWS, GCP.
• Solid understanding of system design patterns, scalability, observability, and performance tuning.
• Strong analytical and problem-solving skills.
• Passion for exploring and building with emerging technologies.

Responsibilities

• Work closely with Data Scientists, Business Team and Engineers to translate requirements into technical architectures and integrate Databricks with enterprise systems.
• Recommend best practices around data engineering, ML/analytics workflows and guide teams in architecture patterns.
• Define project scope, set technical direction, and troubleshoot complex problems.
• Lead the development of proof-of-concept solutions and evaluate new platform features with security/compliance in mind.
• Plan and execute migration strategies to move on-prem data warehouses/data lakes to cloud Databricks.
• Develop reusable frameworks for Data Engineering and Machine Learning Pipelines.
• Explore AI/Gen AI solutions and provide adoption roadmaps.

Skills

PySpark

Python

Lakehouse/Delta Lake

Databricks Certification

ML Ops/LLM Ops

System design patterns

Observability/Performance tuning

Data engineering

Tools

AWS

GCP

Cloudera Hadoop

Databricks

Docker & Containers

EKS

Job description

Tata Consultancy Services in Irving, TX seeks a Databricks Solution Architect to design end-to-end data solutions, guide teams, and promote best practices in data engineering and ML workflows.

Lead PoC developments, evaluate platform features, ensure security and compliance, and plan migrations from on-prem to cloud Databricks. Strong PySpark, Python, and cloud experience are required to succeed.