Remote Data Engineer: Spark, Databricks & AWS

LDSS Corp; Dayton (OH); On-site; USD 120,000 - 135,000; Full time

Benefits offered by this job

401(k)

401(k) matching

Dental insurance

Flexible spending account

Health insurance

Health savings account

Life insurance

Paid time off

Job summary

LDSS Corp in Dayton, Ohio, is seeking a Data Engineer for a remote role supporting DoD MRO programs. You will build ETL pipelines with Databricks and Spark, model data with Spark SQL, and integrate sources via APIs while leveraging AWS S3 and SharePoint data.

Ideal candidates have 2–3 years with Databricks/Spark, 4 years Python/SQL, and experience across Microsoft 365 and Jira. Collaboration with SMEs and cross-functional teams is essential to translate business rules into data transforms.

Qualifications

• Bachelor’s degree in computer science, engineering, or a related field.
• Databricks and Spark experience required or preferred.
• AWS cloud services, including S3, with hands-on data engineering experience.
• Python and SQL proficiency for data analysis and transformation.
• Experience with SharePoint data and Microsoft Graph integration.

Responsibilities

• Build and optimize ETL data pipelines using Databricks and Apache Spark.
• Develop scalable data models leveraging Spark Data Frames and Spark SQL for diverse data sources.
• Use Python and Regular Expressions for data cleansing and preparation for ingestion.
• Integrate data via APIs and automate data exchange across systems.
• Programmatically manage SharePoint data (lists, libraries, files, metadata) using Python and Graph API.
• Leverage Microsoft 365 tools for documentation and collaboration.
• Develop and maintain AWS infrastructure including S3.
• Collaborate with cross-functional teams and SMEs to translate business rules into data transformations.

Tools

Databricks

Apache Spark

AWS

S3

SharePoint

Microsoft Graph

Jira

APIs

Microsoft 365

Job description

LDSS Corp in Dayton, Ohio, is seeking a Data Engineer for a remote role supporting DoD MRO programs. You will build ETL pipelines with Databricks and Spark, model data with Spark SQL, and integrate sources via APIs while leveraging AWS S3 and SharePoint data.

Ideal candidates have 2–3 years with Databricks/Spark, 4 years Python/SQL, and experience across Microsoft 365 and Jira. Collaboration with SMEs and cross-functional teams is essential to translate business rules into data transforms.