Lead Data Platform Engineer - Real-Time Kafka & Spark

IVID TEK INC; Fairmeadows (NC); Hybrid; USD 120,000 - 160,000; Full time

Job summary

IVID TEK INC is seeking a Principal Data Engineer to design and implement enterprise-grade data processing systems. This role requires expertise in real-time data pipelines using Java, Kafka, and Spark, along with strong microservices and API development experience.

The ideal candidate will have over 10 years in data engineering or backend development, with significant familiarity with event-driven architectures. The position allows for considerable autonomy to architect scalable solutions.

Qualifications

• 10–12+ years experience in Java/backend or data engineering.
• Hands-on experience building real-time data pipelines using Kafka and Spark.
• Strong Microservices and API development experience.

Responsibilities

• Architect and implement data platforms and distributed processing systems.
• Build production-ready Spark applications for both batch and real-time processing.
• Design Kafka-based event streaming and ingestion pipelines.

Skills

Java/backend or data engineering

Real-time data pipelines (Kafka, Spark Streaming/Flink)

Relational databases (Redshift, PostgreSQL, Snowflake)

NoSQL databases (MongoDB or similar)

Microservices experience (Spring Boot, REST APIs)

API development and consumption

SQL and data modeling skills

AWS experience (S3, Glue, EMR, Redshift)

CI/CD, Git, Docker/Kubernetes familiarity

Job description

IVID TEK INC is seeking a Principal Data Engineer to design and implement enterprise-grade data processing systems. This role requires expertise in real-time data pipelines using Java, Kafka, and Spark, along with strong microservices and API development experience.

The ideal candidate will have over 10 years in data engineering or backend development, with significant familiarity with event-driven architectures. The position allows for considerable autonomy to architect scalable solutions.