Full Stack Tech Lead

Tata Consultancy Services; Bloomfield (CA); On-site; USD 110,000 - 125,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Medical Coverage

Job summary

Tata Consultancy Services in Bloomfield, CA, is seeking a data engineer to design, build, and optimize large-scale data pipelines using Databricks and Spark. You will develop ETL/ELT processes, ingest data from multiple sources, and leverage AWS services to support cloud analytics workloads.

The role requires strong SQL, Databricks expertise, and experience with data governance, security, and Agile practices.

Qualifications

• Experience designing and implementing large-scale data pipelines.
• Strong SQL skills with complex queries and indexing.
• Proficiency with Databricks and Spark.
• Experience with AWS data services.
• Knowledge of data governance and security practices.

Responsibilities

• Design, develop, and optimize large-scale data processing solutions using Databricks.
• Build and maintain robust ETL/ELT pipelines using Python and Databricks.
• Develop scalable data ingestion, transformation, and integration frameworks from multiple data sources.
• Leverage Databricks features including Delta Lake, Spark SQL, Unity Catalog, Workflows, and ML capabilities.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Work with AWS services to support cloud-based data platforms and analytics workloads.
• Implement data quality checks, monitoring, performance tuning, and operational support processes.
• Participate in architecture discussions and contribute to best practices for data engineering and cloud adoption.
• Ensure compliance with security, governance, and data management standards.
• Actively participate in Agile ceremonies including sprint planning, backlog grooming, daily standups, sprint reviews, and retrospectives.

Skills

Databricks

Python Spark

AWS

Production experience

Query optimization

SQL

Indexing

Schema design

Education

Bachelor of Computer Science

Tools

EC2

S3

Lambda

Glue

EKS

Jenkins

GitLab

GitHub Actions

Job description

Job Description

Must Have Technical/Functional Skills

• Databricks
• Python – Spark
• AWS

• Production experience, query optimization

• SQL – Complex queries, indexing, schema design

Cloud & DevOps

• AWS – Experience with EC2, S3, Lambda, Glue, EKS
• CI/CD – Jenkins / GitLab / GitHub Actions

Good-to-have Skills

• LangChain or LLM orchestration frameworks
• OpenAI API integration experience
• Prompt engineering and embeddings
• Semantic search implementation

Key Responsibilities

• Design, develop, and optimize large-scale data processing solutions using Databricks.
• Build and maintain robust ETL/ELT pipelines using Python and Databricks.
• Develop scalable data ingestion, transformation, and integration frameworks from multiple data sources.
• Leverage Databricks features including Delta Lake, Spark SQL, Unity Catalog, Workflows, and ML capabilities.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Work with AWS services to support cloud-based data platforms and analytics workloads.
• Implement data quality checks, monitoring, performance tuning, and operational support processes.
• Participate in architecture discussions and contribute to best practices for data engineering and cloud adoption.
• Ensure compliance with security, governance, and data management standards.
• Actively participate in Agile ceremonies including sprint planning, backlog grooming, daily standups, sprint reviews, and retrospectives.

Generic Managerial Skills, If any

• Translate requirements into technical designs
• Work with product, data, and AI teams
• Communicate technical decisions to stakeholders

Salary Range: $110,000 -$125,000 year

TCS Employee Benefits Summary

• Discretionary Annual Incentive.
• Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
• Family Support: Maternal & Parental Leaves.
• Insurance Options: Auto & Home Insurance, Identity Theft Protection.
• Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
• Time Off: Vacation, Time Off, Sick Leave & Holidays.
• Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Qualifications:

BACHELOR OF COMPUTER SCIENCE