Lead Data Engineer: Cloud Data Pipelines & Streaming

MassGenics, Inc.; Charlotte, Northern (NC, KY); Hybrid; USD 96,000 - 110,000; Full time

Benefits offered by this job

Medical & pharmacy coverage

Dental/vision insurance

401(k)

HSA & FSA

Life Insurance

Pet Insurance

Short/Long term Disability

Accident & Critical illness coverage

Pre‑paid legal & ID theft protection

Sick time

Paid leaves

Job summary

Innova Solutions is seeking a Lead Data Engineer for a contract role based in multiple US locations, including Charlotte, NC. You will lead the design and development of scalable data pipelines using Spark streaming, Kafka, and Flink, and help implement a lakehouse architecture with governance controls.

Ideal candidates have 10+ years in data engineering, strong cloud experience with Hadoop and Google Cloud, and hands-on expertise in Python, PySpark, Airflow and SQL.

Qualifications

• 10+ years of experience in data engineering, including hands-on Hadoop and Google Cloud data solutions.
• Hands-on experience developing data flows using Kafka, Flink, and Spark streaming.
• Experience with Data Lakehouse architecture and design.
• Experience with Python, PySpark, Apache Kafka, Airflow, SQL, GCP Cloud Storage, BigQuery, Data Proc, Cloud Composer.
• NoSQL databases including columnar, graph, document, KV stores and related formats.

Responsibilities

• Creating/supporting Spark-based processing and Kafka streaming.
• Developing data flows using Kafka, Flink, and Spark streaming.
• Designing and implementing scalable, modular data pipelines with built-in data governance controls.

Skills

Spark

Kafka

Flink

Python

Airflow

SQL

BigQuery

NoSQL

Tools

Hadoop

Google Cloud

Data Lakehouse

Kafka

Spark Streaming

GCP Cloud Storage

Parquet/Iceberg/Delta

Data Proc

Cloud Composer

Job description

Innova Solutions is seeking a Lead Data Engineer for a contract role based in multiple US locations, including Charlotte, NC. You will lead the design and development of scalable data pipelines using Spark streaming, Kafka, and Flink, and help implement a lakehouse architecture with governance controls.

Ideal candidates have 10+ years in data engineering, strong cloud experience with Hadoop and Google Cloud, and hands-on expertise in Python, PySpark, Airflow and SQL.