Databricks Data Engineer and Scientist

Parra Consulting Group, Inc. .; Gaithersburg (MD); In loco; USD 150.000 - 210.000; Tempo pieno

Descrizione del lavoro

Parra Consulting Group, Inc. is seeking a Databricks Data Engineer and Scientist to support a federal client’s enterprise Lakehouse. You will design, build, and maintain data pipelines with AI-assisted tools and apply analytics to turn raw data into actionable insights across the business.

The role requires 8+ years of ETL and data pipeline experience, Databricks and PySpark coding, and AWS services knowledge. U.S. citizenship is required for the federal contract.

Competenze

• Bachelor’s degree in a relevant technical field.
• 8+ years of ETL and data pipeline experience.
• U.S. citizenship and ability to obtain Public Trust.
• Experience with Databricks and Python/PySpark.
• Experience with AWS services (Glue, S3, RDS).

Mansioni

• Design, build, and maintain robust end-to-end data pipelines for a Lakehouse on Databricks and AWS.
• Develop and manage ETL processes with logging and error handling.
• Design data models within Bronze, Silver, Gold medallion architecture.
• Analyze and decompose complex legacy SQL and rebuild pipelines.
• Optimize pipeline performance using partitioning, bucketing, and caching.
• Collaborate with cross-functional teams to deliver analytics-ready data.

Conoscenze

ETL pipelines

Python

PySpark

Data modeling

SQL

AWS

Formazione

Bachelor’s degree in Computer Science/Engineering/Math/Physics/IST

Strumenti

Databricks

Delta Lake

Unity Catalog

Claude Code

Cursor

AWS Glue

S3

RDS

Descrizione del lavoro

Parra Consulting Group (PCG) is seeking a Databricks Data Engineer and Scientist to support a federal client’s critical enterprise Lakehouse. This position designs, builds, and maintains robust data pipelines and applies data science, AI, and analytics skills to turn raw data into actionable insights across the business.

This successful candidate will be responsible for the following:

• Design, build, and maintain robust, end-to-end data pipelines for an enterprise Lakehouse on Databricks and AWS.
• Develop and manage ETL processes with comprehensive logging and error handling to troubleshoot and resolve issues.
• Design data models within a Bronze, Silver, and Gold medallion architecture, selecting dimensional or relational approaches as appropriate.
• Analyze and decompose complex legacy SQL, rebuild pipelines and models, and compare results to validate accuracy and performance.
• Optimize pipeline performance and scalability using techniques such as partitioning, bucketing, and caching.
• Use AI-assisted coding tools to accelerate pipeline analysis, data engineering, and data modeling tasks.
• Help ensure data security and compliance, and collaborate with cross-functional teams to deliver analytics-ready data.

Required Qualifications

• Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Math, Physics, or Information Systems and Technology.
• 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines.
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
• S. citizenship required to support this federal contract.

Required Experience

• Experience designing, developing, deploying, and managing ETL processes and data pipelines with comprehensive logging and error handling.
• 3+ years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools, including Delta Lake and Unity Catalog.
• 3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations.
• Experience using AI-assisted coding tools, such as Claude Code or Cursor, to support data engineering and data modeling tasks.
• Experience with data modeling, including selecting dimensional or relational models, within a Bronze, Silver, and Gold medallion architecture.
• Experience using AWS services, including AWS Glue, Amazon S3, and Amazon RDS.
• Knowledge of ETL performance optimization techniques, such as partitioning, bucketing, and caching, and experience troubleshooting query performance.
• Ability to analyze and decompose complex legacy SQL, rebuild data pipelines or models, and compare results to validate accuracy and performance.