Data Engineer - Databricks, Spark & AWS (Hybrid)

LDSS Corp; Dayton (OH); Hybrid; USD 120,000 - 130,000; Full time

Benefits offered by this job

401(k)

401(k) matching

Dental insurance

Flexible spending account

Health insurance

Health savings account

Life insurance

Paid time off

Referral program

Tuition reimbursement

Vision insurance

Job summary

LDSS Corp is seeking a data engineer to build and optimize ETL pipelines using Databricks and Apache Spark, with strong SQL and data modeling skills. The role involves AWS (S3), Python data analysis, SharePoint/Graph API management, and Jira-based project tracking. Hybrid work with collaboration across DoD-focused teams and SMEs.

The ideal candidate has DoD experience, security clearance eligibility, and a background in supply chain/data transformation for Air Force MRO programs.

Qualifications

• Experience with Databricks, Spark, and cloud-based data engineering
• Strong SQL skills and data modeling
• Proficiency with AWS S3 and Python for data analysis
• Experience with SharePoint and Microsoft 365 integrations
• Ability to develop and consume APIs for data exchange
• Familiarity with Jira or similar agile tools

Responsibilities

• Build and optimize ETL pipelines using Databricks and Spark
• Develop scalable data models with Spark DataFrames and Spark SQL
• Use Python and regex for data cleansing and parsing
• Use APIs for data exchange and automation across systems
• Manage SharePoint data via Python and Graph API
• Utilize Microsoft 365 for documentation and collaboration
• Develop AWS data pipelines including S3 workloads
• Collaborate with cross-functional teams to transform legacy data
• Work with SMEs to derive business rules for data transformations

Skills

Databricks

Apache Spark

SQL

AWS

Python

SharePoint

Microsoft Graph

APIs

Jira

Data modeling

ETL

Education

Bachelor's degree in computer science/engineering

Tools

Microsoft Graph API

SharePoint

Job description

LDSS Corp is seeking a data engineer to build and optimize ETL pipelines using Databricks and Apache Spark, with strong SQL and data modeling skills. The role involves AWS (S3), Python data analysis, SharePoint/Graph API management, and Jira-based project tracking. Hybrid work with collaboration across DoD-focused teams and SMEs.

The ideal candidate has DoD experience, security clearance eligibility, and a background in supply chain/data transformation for Air Force MRO programs.