Databricks Data Engineer (1157930)

The Judge Group; Arlington (VA); Hybride; USD 76 000 - 90 000; Plein temps

Avantages offerts par ce poste

Medical, dental, and vision insurance

Résumé du poste

The Judge Group is currently seeking a Databricks Data Engineer to support a newly awarded contract with a large federal agency. This is a hybrid position.

It is a w2 only opportunity, no third parties. You will build and maintain Databricks ingestion, transformation, orchestration, migration, and data-processing pipelines, developing scalable ETL/ELT solutions using Databricks, Spark, PySpark, Python, SQL, and Delta Lake.

Qualifications

• Bachelor's degree in computer science, data engineering, information systems, engineering, analytics, or a related field.
• Minimum 4 years of experience in data engineering, software development, ETL/ELT, cloud analytics, or a related technical discipline.
• Hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, and workflow orchestration.
• Experience building, testing, and maintaining batch data pipelines, transformations, and curated data layers.
• Experience with enterprise data migration, source-to-target mapping, reconciliation, validation, and data-quality controls.
• Familiarity with Git, CI/CD, automated testing, deployment, monitoring, logging, and troubleshooting.
• Strong analytical, problem-solving, documentation, collaboration, and communication skills.
• Ability to obtain and maintain a Federal Public Trust.

Responsabilités

• Build and maintain Databricks ingestion, transformation, orchestration, migration, and data-processing pipelines.
• Develop scalable ETL/ELT solutions using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake.
• Implement source-to-target mappings, transformation rules, business logic, data contracts, metadata, lineage, and data-quality requirements.
• Develop migrated data feeds, APIs, curated datasets, reporting layers, and reusable data products.
• Create reconciliation routines and support source-to-target validation, exception analysis, and defect resolution.
• Build unit, integration, regression, performance, and data-quality tests and document technical results.
• Support Git-based development, CI/CD, release management, deployment, monitoring, troubleshooting, and stabilization.
• Analyze and resolve job failures, cluster-performance issues, pipeline bottlenecks, schema changes, and data-quality problems.
• Collaborate with data architects, engineering leads, integration engineers, profiling analysts, testers, security specialists, governance leads, and source-system teams.
• Document reusable code, technical patterns, runbooks, troubleshooting procedures, and lessons learned for later modernization waves.
• Participate in Agile ceremonies, backlog refinement, sprint planning, demonstrations, and technical reviews.

Connaissances

Analytical

Problem-solving

Documentation

Collaboration

Communication

Formation

Bachelor's degree in computer science, data engineering, information systems, engineering, analytics, or related field

Outils

Databricks

Apache Spark

PySpark

Python

SQL

Delta Lake

Workflow orchestration

Description du poste

Location: Arlington, VA

Salary: $55.00 USD Hourly - $65.00 USD Hourly

Description: The Judge Group is currently seeking a Databricks Data Engineer to support a newly awarded contact with a large federal agency. This is a hybrid position. It is a w2 only opportunity, no third parties.

What You Will Do

• Build and maintain Databricks ingestion, transformation, orchestration, migration, and data-processing pipelines.
• Develop scalable ETL/ELT solutions using Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake.
• Implement source-to-target mappings, transformation rules, business logic, data contracts, metadata, lineage, and data-quality requirements.
• Develop migrated data feeds, APIs, curated datasets, reporting layers, and reusable data products.
• Create reconciliation routines and support source-to-target validation, exception analysis, and defect resolution.
• Build unit, integration, regression, performance, and data-quality tests and document technical results.
• Support Git-based development, CI/CD, release management, deployment, monitoring, troubleshooting, and stabilization.
• Analyze and resolve job failures, cluster-performance issues, pipeline bottlenecks, schema changes, and data-quality problems.
• Collaborate with data architects, engineering leads, integration engineers, profiling analysts, testers, security specialists, governance leads, and source-system teams.
• Document reusable code, technical patterns, runbooks, troubleshooting procedures, and lessons learned for later modernization waves.
• Participate in Agile ceremonies, backlog refinement, sprint planning, demonstrations, and technical reviews.

What You Will Need

• Bachelor's degree in computer science, data engineering, information systems, engineering, analytics, or a related field.
• Minimum 4 years of experience in data engineering, software development, ETL/ELT, cloud analytics, or a related technical discipline.
• Hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, and workflow orchestration.
• Experience building, testing, and maintaining batch data pipelines, transformations, and curated data layers.
• Experience with enterprise data migration, source-to-target mapping, reconciliation, validation, and data-quality controls.
• Familiarity with Git, CI/CD, automated testing, deployment, monitoring, logging, and troubleshooting.
• Strong analytical, problem-solving, documentation, collaboration, and communication skills.
• Ability to obtain and maintain a Federal Public Trust.

What Would Be Nice To Have

• Databricks Certified Data Engineer Associate or Professional certification.
• Experience with Unity Catalog, Delta Live Tables, Databricks Workflows, or Auto Loader.
• Experience developing APIs, integration pipelines, or data-consumption services.
• Experience with Azure DevOps, GitHub Actions, Jenkins, Terraform, or related engineering tools.
• Experience with streaming technologies such as Kafka or Azure Event Hubs.
• Experience migrating HR, ERP, workforce, financial, or sensitive enterprise data.
• Experience supporting federal agencies, regulated organizations, or public-sector modernization.
• Experience with Microsoft Azure or Amazon Web Services.

Medical, dental, and vision insurance are available to qualified candidates who meet eligibility requirements.

Contact: [e-mail]