Remote Databricks Data Engineer & OCDS Specialist

Healing Strides Counseling; United States; Hybrid; USD 110,000 - 150,000; Full time

Job summary

Scicom Infrastructure Services seeks a Data Engineer / Business Analyst to drive a large Databricks implementation bridging business, governance, and engineering teams. You will define data contracts, map source data, and build reliable data pipelines for procurement and OCDS data.

Responsibilities include requirements gathering, data profiling, source-to-target mappings, and ensuring data quality and traceability across the contracting lifecycle using Delta Lake and Spark.

Qualifications

• Bachelor’s degree in computer science, information systems, data analytics, or related field.
• At least five years of combined data-engineering, data-analysis, business-analysis, or data-integration experience.
• Hands-on experience implementing Open Contracting Data Standard.
• Experience mapping procurement data to OCDS schemas.
• Strong knowledge of OCDS releases, records, schemas, and validation.
• Three+ years of hands-on Databricks experience.
• Proficiency in Spark, PySpark, Python, SQL, and Delta Lake.
• Experience gathering requirements and writing user stories and acceptance criteria.
• Agile delivery experience and effective stakeholder communication.
• Ability to document rules, data dictionaries, and process flows.

Responsibilities

• Lead requirements sessions with procurement, contracting, analytics, governance, and technical stakeholders.
• Define and maintain data contracts between data producers and consumers.
• Translate business requirements into user stories, mappings, and specs.
• Map source-system data to OCDS fields and structures.
• Develop Databricks data pipelines using Spark, PySpark, Python, and SQL.
• Implement Bronze-Silver-Gold data layers and Delta Lake schemas.
• Validate data against OCDS schemas and data-quality checks.
• Support data governance and access-control planning.
• Collaborate with agile teams on backlogs, sprints, and testing.

Skills

Data contracts

Databricks

Python

SQL

Apache Spark

REST APIs

Agile delivery

Stakeholder communication

Data modeling

Education

Bachelor’s degree in CS/IS/Data analytics

Tools

Delta Lake

JSON Schema

CI/CD tools

Job description

Scicom Infrastructure Services seeks a Data Engineer / Business Analyst to drive a large Databricks implementation bridging business, governance, and engineering teams. You will define data contracts, map source data, and build reliable data pipelines for procurement and OCDS data.

Responsibilities include requirements gathering, data profiling, source-to-target mappings, and ensuring data quality and traceability across the contracting lifecycle using Delta Lake and Spark.