Remote Spark Data Integration Engineer

Lakefusion; United States; Remote; USD 27,000 - 43,000; Full time

Benefits offered by this job

Remote work from India

End-to-end project ownership

Small focused team

Job summary

Lakefusion is seeking an Integration Engineer to design, build, and maintain data connectors for a master data management platform. The role spans full-stack development and data engineering, focusing on reliable integrations that scale customer data across the platform.

Hybrid role with external vendor collaboration, API orchestration, and configuration interfaces. You will work with Spark Structured Streaming, backend services, and frontend experiences to empower customers to manage

Qualifications

• Experience building data integrations or large-scale ETL/ELT pipelines.
• Strong proficiency with Apache Spark, especially Spark Structured Streaming.
• Backend development for APIs and orchestration services.
• Frontend development for configuration interfaces.
• Solid understanding of REST APIs, authentication patterns, and data schemas.
• Excellent communication for vendor relationships and cross-team collaboration.

Responsibilities

• Design, build, and maintain end-to-end integrations with enterprise data providers such as Dun & Bradstreet, ZoomInfo, and Smarty.
• Develop Spark Structured Streaming pipelines to ingest and process high-volume third-party data.
• Build backend services and APIs that orchestrate integration workflows, authentication, and data delivery.
• Create frontend configuration experiences that let customers set up and manage integrations.
• Work directly with external vendors to understand APIs, authentication models, schemas, rate limits, and delivery mechanisms, then translate those into working integrations.
• Collaborate with Product, Platform, and Delivery teams to ensure each integration meets customer requirements.

Skills

Spark Structured Streaming

APIs & orchestration

Frontend configuration interfaces

Data integration

Vendor management

Tools

Databricks Unity Catalog

Databricks Workflows

Databricks Jobs

Azure

AWS

Terraform

Docker

Job description

Lakefusion is seeking an Integration Engineer to design, build, and maintain data connectors for a master data management platform. The role spans full-stack development and data engineering, focusing on reliable integrations that scale customer data across the platform.

Hybrid role with external vendor collaboration, API orchestration, and configuration interfaces. You will work with Spark Structured Streaming, backend services, and frontend experiences to empower customers to manage