Senior Data Engineer (Customer Data Products)

National Board of Medical Examiners; Philadelphia (Philadelphia County); Hybride; USD 127 000 - 160 000; Plein temps

Avantages offerts par ce poste

Healthcare

Dental

Prescription

Vision plans

401(k) w/match

Tuition Reimbursement Plan

Commuter Benefit: Public Transit or...

Remote Friendly Workplace

Résumé du poste

National Board of Medical Examiners (NBME) is seeking a Data Engineer to join a team of data engineers and BI developers. You will build data lakes, pipelines, and scalable data solutions on AWS to support analytics, AI/ML, and decision-making for internal staff and external customers.

You will collaborate with data scientists, engineers, and stakeholders to prepare structured and unstructured data for model training and feature extraction, while advancing data engineering practices and

Qualifications

• Bachelor's Degree required.
• 7+ years of application development experience.
• 4+ years in big data technologies.
• 4+ years with AWS cloud.
• 4+ years in application development (Python/SQL/Scala/Java).
• 4+ years with distributed data tools (MapReduce/Hadoop/Hive/EMR/Kafka/Spark/MySQL).
• 4+ years in real-time data and streaming apps.
• 4+ years with NoSQL (Mongo/Cassandra).
• 4+ years data warehousing (Redshift).
• 6+ years UNIX/Linux experience.
• 7+ years Agile engineering practices.
• 7+ years SQL optimization.
• 4+ years PySpark.
• 3+ years orchestration (AirFlow/KubeFlow/AWS Step Functions/Luigi).
• Preferred: Generative AI, LLM data prep pipelines, vector databases, feature stores, APIs, and DaaS.

Responsabilités

• Code, test, deploy, orchestrate, monitor, document, and troubleshoot cloud-based data engineering processes, feature stores, and vector databases.
• Collaborate with data scientists, AI researchers, data and enterprise architects, and stakeholders to prepare data for AI/ML model training and features.
• Lead evaluation and experimentation with batch/streaming data techs and MLOps to stay aligned with industry innovation.
• Act as a technical lead to showcase AI and data tech capabilities across NBME.
• Refine processes for the data engineering practice.
• Educate ETL developers on cloud-based data engineering initiatives.
• Ensure data integrity and security across corporate data.
• Guarantee high data quality for Data Services, Analytics and MDM.
• Coordinate solutions and own designs, development, testing, delivery.
• Build automated, scalable, test-driven data pipelines.
• Utilize Git, CI/CD, and release management to enhance AWS pipelines.
• Collaborate on DataOps tool/framework improvements.

Connaissances

Python

SQL

Scala

Java

PySpark

Spark

Hadoop

Kafka

Redshift

MongoDB

Cassandra

Unix/Linux

AirFlow

KubeFlow

AWS

Formation

Bachelor's Degree

Outils

Git

CI/CD

EMR

Linux scripting

Description du poste

POSITION OVERVIEW

NBME is looking for a Data Engineer to join a highly skilled and growing team of data engineers and BI developers. This team successfully launched and enhances a data product on AWS for medical doctors. Read more here.

In this role, you will use your data engineering and problem‑solving skills to deliver meaningful insights to internal staff and external customers. The Data Engineer will help modernize, expand, and optimize NBME's data platform by building data lakes, complex data integration pipelines, and scalable data solutions that support analytics, AI/ML, and business decision‑making. This role will apply AI‑assisted engineering practices to improve development efficiency and share best practices for AI assisted engineering across theIT organization.

This role has been designated as a hybrid position, though it may require occasional onsite work for team collaboration or meetings. We are currently prioritizing applicants within a 50‑mile radius from our Philadelphia office.

Please note that onsite interviews and onboarding at our Philadelphia office may be required for this role. If applicable, advance notice will be provided to support planning.

At NBME, we continue to innovate and improve how we fulfill the evolving needs of the healthcare community. This commitment starts and ends with the people at NBME. By recruiting and empowering talented individuals from various disciplines and backgrounds, which includes professionals with diverse life experiences, abilities, and perspectives, NBME can take a well‑informed, robust approach to advancing medical education and assessment for years to come.

RESPONSIBILITIES

• Code, test, deploy, orchestrate, monitor, document, and troubleshoot cloud‑based data engineering processes, feature stores, and vector databases in accordance with best practices and security standards throughout the development lifecycle.
• Partner closely with data scientists, AI researchers, data and enterprise architects, and business stakeholders to identify, extract, clean, and format structured and unstructured data for AI/ML model training, fine‑tuning, and feature extraction.
• Lead evaluation, research, and experimentation efforts with batch and streaming data technologies, LLM data preparation frameworks, and MLOps tools to keep pace with industry innovation
• Act as a technical lead to showcase the capabilities of emerging AI and data technologies, enabling the widespread adoption of modern data techniques across the organization.
• Significantly contribute to the definition and refinement of processes and procedures for the data engineering practice
• Educate and develop ETL developers on data engineering cloud‑bases initiatives to enable transition to data engineer and practice
• Assures the integrity and accuracy of the corporate data, with particular attention to data security.
• Responsible for ensuring high data quality for Data Services, Analytics and Master Data Management.
• Helps coordinate technical solutions, takes responsibility for designs, development, testing and delivery of solutions.
• Build automated, scalable, test‑driven data pipelines.
• Utilize software development practices such as version control via Git, CI/CD, and release management to enhance existing CI/CD pipelines in AWS.
• Collaborate with Data Engineers, DevOps engineers and architects on improvement opportunities for DataOps tools and frameworks.

QUALIFICATIONS

• Bachelor's Degree
• At least 7 years of experience in application development (Internship experience does not apply)
• At least 4 years of experience in big data technologies
• At least 4 years' experience with cloud computing using AWS
• 4+ years of experience in application development including Python, SQL, Scala, or Java
• 4+ years' experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, MySQL etc.)
• 4+ year experience working on real‑time data and streaming applications
• 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
• 4+ years of data warehousing experience (Redshift)
• 6+ years of experience with UNIX/Linux including basic commands and shell scripting
• 7+ years of experience with Agile engineering practices
• 7+ years of experience with SQL optimization
• 4+ years of experience with PySpark
• 3+ year of experience with process orchestration including AirFlow, KubeFlow, AWS step functions, or Luigi

Preferred Qualifications

• Proven experience implementing Generative AI, LLM data preparation pipelines, and Vector Databases (e.g., Pinecone, Milvus, pgvector).
• Strong experience building and maintaining Feature Stores for machine learning models.
• Experience building highly scalable, secure, and production‑ready APIs and Data‑as‑a‑Service (DaaS) platforms.
• AWS Certified Data Engineer or AWS Certified Machine Learning – Specialty certifications.
• 3+ year of experience with Machine Learning
• Experience with building a Data‑as‑a‑service platform
• Experience with building APIs

About NBME

NBME is a not‑for‑profit organization that specializes in the creation of assessments and learning tools for physicians and health professionals. Our mission is to advance assessment of these professionals to achieve optimal care for all, supporting the development of a highly effective, diverse and compassionate health care workforce.

Founded in 1915, NBME develops and manages the United States Medical Licensing Examination® with the Federation of State Medical Boards. We offer a comprehensive portfolio of assessment products for every stage of the medical school journey and provide assessment services to various health profession organizations.

We are dedicated to advancing innovative assessment approaches through research, collaboration and contributions to the medical education and assessment communities. In 2024, we expanded our assessment capabilities to include simulation through the acquisition of MedVR Education.

Learn more on NBME's website.

NBME's Community Collaborations and Contributions

NBME believes that a key path to meeting our mission is collaborating with and supporting our communities. Our Community Collaborations and Contributions programs provide resources and recognition to medical educators and researchers advancing the field of assessment throughout their careers. In addition, we invest in fee assistance, scholarships and pathway programs to aid learners and help support increased representation in health care. Headquartered in Philadelphia, NBME also gives to local organizations that advance health equity and access and positively impact social determinants of health.

Compensation

Compensation we are offering for this position is at $127,337- $160,000/year. Please note that the offered rate for this position typically aligns with the minimum to midrange of this grade, but it can vary based on the successful candidate's qualifications and experience, department budget, and an internal equity review.

Benefits

• Healthcare
• Dental
• Prescription
• Vision plans
• 401(k) w/match
• Tuition Reimbursement Plan
• Commuter Benefit: Public Transit or Parking options
• Remote Friendly Workplace

Equal Opportunity Employer

NBME is an Equal Opportunity Employer. We will consider all qualified applicants for employment without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.