Senior Data Engineer: Databricks + AWS

Robotic Technologies Automation; Sunnyvale (CA); On-site; USD 83,000 - 102,000; Full time

Job summary

Robotic Technologies Automation in Sunnyvale, CA is seeking a Senior Data Engineer with strong Databricks, Python, PySpark, SQL, and AWS expertise to design scalable data pipelines and enterprise data platforms.

You will implement ETL/ELT workflows, optimize Spark jobs, and collaborate with stakeholders to deliver data lakehouse architectures, governance, and AI-powered data solutions.

Qualifications

• 8+ years of experience in Data Engineering.
• Strong experience with Python, PySpark, and SQL.
• Hands-on experience with Databricks and Delta Lake.
• Strong knowledge of ETL/ELT pipeline development.
• Experience with AWS services such as S3, Glue, Lambda, Redshift, and EMR.
• Experience with Snowflake or other cloud data warehouses.
• Strong knowledge of data modeling and data warehousing.
• Experience with Apache Spark and distributed data processing.
• Knowledge of data integration and orchestration tools.
• Experience with Git, Jenkins, and CI/CD pipelines.
• Strong understanding of data quality, performance tuning, and optimization.
• Recent / Modern Skills (2025–2026)
• Databricks Unity Catalog and data governance.
• Databricks Lakehouse architecture.
• Databricks Workflows and Delta Live Tables.
• Databricks Genie and Genie Agents.
• Model Context Protocol (MCP) integration.
• Conversational analytics and Text-to-SQL.
• AI-powered data engineering and automation.
• Vector search and RAG integration with enterprise data.
• Data pipeline observability and monitoring.
• Infrastructure as Code using Terraform.

Responsibilities

• Design, develop, and maintain scalable data engineering pipelines.
• Build ETL/ELT workflows using Python, PySpark, and SQL.
• Develop data processing solutions using Databricks and AWS.
• Implement data lakehouse and data warehouse architectures.
• Optimize Spark jobs and large-scale data processing workloads.
• Implement data quality checks and data governance practices.
• Integrate enterprise data sources into centralized data platforms.
• Collaborate with architects, analysts, and business stakeholders.
• Support production deployments, troubleshooting, and performance tuning.

Skills

Python

PySpark

SQL

Databricks

Delta Lake

AWS

Snowflake

Data modeling

Data warehousing

Spark

ETL/ELT

CI/CD

Git

Terraform

Data governance

Data observability

Databricks Unity Catalog

Databricks Lakehouse

Databricks Workflows

Delta Live Tables

Genie

MCP integration

Text-to-SQL

AI-powered data engineering

Vector search

Data pipeline observability

Terraform (IaC)

Tools

Databricks Unity Catalog

Databricks Genie

Databricks Workflows

Delta Live Tables

Delta Lake

Terraform

Job description

Robotic Technologies Automation in Sunnyvale, CA is seeking a Senior Data Engineer with strong Databricks, Python, PySpark, SQL, and AWS expertise to design scalable data pipelines and enterprise data platforms.

You will implement ETL/ELT workflows, optimize Spark jobs, and collaborate with stakeholders to deliver data lakehouse architectures, governance, and AI-powered data solutions.