Data Engineer: Scalable Pipelines with Databricks

The Marlin Alliance, Inc.; San Diego (CA); Hybrid; USD 85,000 - 100,000; Full time

Job summary

The Marlin Alliance seeks a Data Engineer to build and maintain scalable data pipelines for our Navy client. You will transform raw operational data into analytics-ready datasets, ensuring governance and quality while collaborating across teams in a hybrid San Diego environment.

Responsibilities include designing data pipelines with Databricks and PySpark, optimizing Python and SQL workflows, and translating ambiguous problems into actionable analytics.

Qualifications

• Bachelor's degree in Computer Science, Data Science, or a related field or equivalent professional experience.
• 3 to 5 years of experience in data engineering or a related field.
• Demonstrated experience with Databricks and PySpark for large scale data processing.
• Strong programming ability in Python.
• Experience writing, optimizing, and troubleshooting SQL queries.
• Ability to infer and define valuable analytics from datasets without established analytic requirements.
• Strong written and verbal communication skills, with the ability to convey technical concepts to diverse audiences.
• Must be a US citizen and able to obtain and maintain a Secret security clearance.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks and PySpark.
• Develop and optimize Python and SQL based data transformation workflows.
• Analyze datasets that may lack predefined analytic requirements and independently identify valuable, actionable insights.
• Collaborate with stakeholders to translate ambiguous data problems into clear analytic solutions.
• Ensure data quality, governance, and consistency across pipelines and outputs.
• Document data engineering processes, pipeline architecture, and analytic approaches.
• Partner with cross-functional team members to integrate data outputs into downstream applications and dashboards.
• Troubleshoot and resolve data pipeline issues to maintain reliable, timely data delivery.

Skills

Databricks

PySpark

Python

SQL

Analytical thinking

Communication

Education

Bachelor's degree in Computer Science, Data Science, or related field

Tools

Palantir Foundry

Databricks

React

Job description

The Marlin Alliance seeks a Data Engineer to build and maintain scalable data pipelines for our Navy client. You will transform raw operational data into analytics-ready datasets, ensuring governance and quality while collaborating across teams in a hybrid San Diego environment.

Responsibilities include designing data pipelines with Databricks and PySpark, optimizing Python and SQL workflows, and translating ambiguous problems into actionable analytics.