Databricks Data Engineer & Scientist for Lakehouse AI

Parra Consulting Group, Inc. .; Oregon; Hybrid; USD 140,000 - 190,000; Full time

Job summary

Parra Consulting Group is seeking a Databricks Data Engineer and Scientist to support a federal client’s enterprise Lakehouse on Databricks and AWS.

You will design, build, and maintain end‑to‑end data pipelines, develop Bronze–Silver–Gold models, optimize performance, and apply Python/PySpark and AI-assisted coding tools.

Qualifications include 8+ years of ETL/pipeline experience, a Bachelor’s in a STEM field, US citizenship, and the ability to obtain a Public Trust.

Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Math, Physics, or Information Systems and Technology.
• 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
• U.S. citizenship required to support this federal contract.

Responsibilities

• Design, build, and maintain robust, end-to-end data pipelines for an enterprise Lakehouse on Databricks and AWS.
• Develop and manage ETL processes with comprehensive logging and error handling to troubleshoot and resolve issues.
• Design data models within a Bronze, Silver, and Gold medallion architecture, selecting dimensional or relational approaches as appropriate.
• Analyze and decompose complex legacy SQL, rebuild pipelines and models, and compare results to validate accuracy and performance.
• Optimize pipeline performance and scalability using techniques such as partitioning, bucketing, and caching.
• Use AI-assisted coding tools to accelerate pipeline analysis, data engineering, and data modeling tasks.
• Help ensure data security and compliance, and collaborate with cross-functional teams to deliver analytics-ready data.

Skills

ETL development

Data pipeline design

Data modeling

AI-assisted coding

Cross-functional collaboration

Education

Bachelor's degree in Computer Science or related field

Tools

Databricks

Python

PySpark

AWS (S3, Glue, RDS)

Unity Catalog

Job description

Parra Consulting Group is seeking a Databricks Data Engineer and Scientist to support a federal client’s enterprise Lakehouse on Databricks and AWS.

You will design, build, and maintain end‑to‑end data pipelines, develop Bronze–Silver–Gold models, optimize performance, and apply Python/PySpark and AI-assisted coding tools.

Qualifications include 8+ years of ETL/pipeline experience, a Bachelor’s in a STEM field, US citizenship, and the ability to obtain a Public Trust.