Senior Data Engineer

Ravh-IT; Irvine (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Ravh-IT in Irvine, CA is seeking an experienced Databricks Technical Lead to design, develop, and implement scalable data engineering solutions. The role requires leadership, hands-on development, and collaboration with stakeholders to deliver enterprise-grade data pipelines.

The ideal candidate has 10+ years in IT or data engineering, deep Databricks and Spark expertise, and strong Python/SQL skills. Onsite work in Irvine with cloud experience on Azure or AWS is expected.

Qualifications

• 10-15+ years of experience in IT, data engineering, or data platform development.
• Strong hands-on experience with Databricks and Apache Spark.
• Expertise in Python, PySpark, and advanced SQL.
• Experience with Delta Lake, Lakehouse architecture, and data modeling.
• Experience developing and optimizing large-scale data pipelines.

Responsibilities

• Lead the design and development of scalable data pipelines using Databricks and Spark.
• Develop and optimize ETL/ELT workflows using Python, PySpark, and SQL.
• Design and implement data lakehouse architectures and Delta Lake solutions.
• Build, maintain, and optimize batch and streaming data processing pipelines.
• Collaborate with cloud platforms like Azure or AWS to develop reliable data engineering solutions.
• Implement data quality checks, error handling, monitoring, and performance tuning.
• Lead technical discussions, code reviews, and solution design sessions.
• Mentor developers and establish coding standards and best practices.
• Collaborate with data architects, business analysts, and cross-functional teams to translate requirements into technical solutions.
• Ensure data security, governance, scalability, and reliability across data platforms.

Skills

Databricks

Apache Spark

Python

PySpark

SQL

Data modeling

Tools

Azure

AWS

Job description

We are seeking an experienced Databricks Technical Lead to lead the design, development, and implementation of scalable data engineering solutions. The ideal candidate will have strong expertise in Databricks, Apache Spark, Python, SQL, cloud data platforms, and modern data architecture. This role requires technical leadership, hands-on development, performance optimization, and collaboration with business and technical stakeholders.

Key Responsibilities

• Lead the design and development of scalable data pipelines using Databricks and Apache Spark.
• Develop and optimize ETL/ELT workflows using Python, PySpark, and SQL.
• Design and implement data lakehouse architectures and Delta Lake solutions.
• Build, maintain, and optimize batch and streaming data processing pipelines.
• Work with cloud platforms such as Azure or AWS to develop reliable data engineering solutions.
• Implement data quality checks, error handling, monitoring, and performance tuning.
• Lead technical discussions, code reviews, and solution design sessions.
• Mentor developers and establish coding standards and best practices.
• Collaborate with data architects, business analysts, and cross-functional teams to translate requirements into technical solutions.
• Ensure data security, governance, scalability, and reliability across data platforms.

Required Skills

• 10-15+ years of experience in IT, data engineering, or data platform development.
• Strong hands-on experience with Databricks and Apache Spark.
• Expertise in Python, PySpark, and advanced SQL.
• Experience with Delta Lake, Lakehouse architecture, and data modeling.
• Strong understanding of ETL/ELT processes and large-scale data processing.
• Experience developing and optimizing complex data pipelines.
• Hands-on experience with Azure or AWS cloud data services.
• Knowledge of Git, CI/CD pipelines, and automated deployment practices.
• Strong troubleshooting, analytical, and performance-tuning skills.
• Proven experience leading technical teams and delivering enterprise-level data solutions.

Preferred Qualifications

• Experience with Databricks Unity Catalog, workflow orchestration, and governance.
• Knowledge of data warehousing and modern data architecture patterns.
• Experience with streaming technologies such as Kafka or Structured Streaming.
• Familiarity with data security, access controls, and regulatory compliance.
• Excellent communication, stakeholder management, and technical leadership skills.

Work Location

This is an onsite position in Irvine, California. Candidates must be willing to work onsite and be eligible for W2 employment.