Senior Data Engineer — AI-Driven Customer Data Platform

JobCubby; Philadelphia, Northern (Philadelphia County, KY); Hybrid; USD 127,000 - 160,000; Full time

Benefits offered by this job

Healthcare benefits

Dental benefits

Tuition reimbursement

Commuter benefits

Remote friendly workplace

Job summary

NBME seeks a Data Engineer to join a skilled team of data engineers and BI developers. You will build and optimize cloud-based data pipelines on AWS, prepare data for AI/ML models, and lead data engineering initiatives across the organization.

You will collaborate with data scientists, AI researchers, and enterprise architects to ensure data quality and availability for analytics, data services, and master data management. Remote-friendly workplace with competitive compensation.

Qualifications

• Bachelor’s degree required.
• 7+ years of application development experience.
• 4+ years of big data technologies experience.
• 4+ years with AWS cloud computing.
• 4+ years of development in Python, SQL, Scala, or Java.
• 4+ years with distributed data tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, MySQL).
• 4+ years on real-time data and streaming apps.
• 4+ years with NoSQL (Mongo, Cassandra).
• 4+ years data warehousing (Redshift).
• 6+ years UNIX/Linux, 7+ years Agile, 7+ years SQL optimization.
• 4+ years PySpark; 3+ years orchestration (AirFlow, KubeFlow, Step Functions, Luigi).

Responsibilities

• Code, test, deploy and monitor cloud data pipelines and feature stores.
• Collaborate with data scientists and architects to prepare data for AI/ML.
• Lead evaluation of batch/streaming data tech and MLOps tools.
• Act as technical lead to showcase modern data techniques.
• Help refine data engineering processes and best practices.
• Educate ETL developers on cloud-based data initiatives.
• Ensure data integrity and security across data services.
• Deliver high-quality data for analytics and master data management.
• Coordinate technical solution designs and delivery.
• Build automated, scalable, test-driven pipelines with Git/CI/CD in AWS.
• Collaborate with Data Engineers, DevOps and architects on DataOps improvements.

Skills

AWS cloud computing

Python/Scala/Java

Big data tech

Data streaming

SQL optimization

Agile engineering

UNIX/Linux

PySpark

Education

Bachelor's Degree

Tools

MapReduce

Hadoop

Hive

EMR

Kafka

Spark

MySQL

AirFlow

KubeFlow

AWS Step Functions

Luigi

Job description

NBME seeks a Data Engineer to join a skilled team of data engineers and BI developers. You will build and optimize cloud-based data pipelines on AWS, prepare data for AI/ML models, and lead data engineering initiatives across the organization.

You will collaborate with data scientists, AI researchers, and enterprise architects to ensure data quality and availability for analytics, data services, and master data management. Remote-friendly workplace with competitive compensation.