Senior Data Engineer: Realtime Pipelines & ML Feature Store

INSPYR Solutions; Florida; On-site; USD 120,000 - 180,000; Full time

Job summary

INSPYR Solutions seeks a Senior Data Engineer to design and maintain scalable data pipelines and infrastructure powering analytics, ML, and BI initiatives. You will work with data scientists, analysts, and stakeholders to ensure high-quality data for pricing optimization and decision-making.

The role emphasizes accelerating feature store expansion through robust data modeling, pipeline development, and feature engineering, while contributing to pricing semantic frameworks and knowledge graphs.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 5+ years of experience in data engineering, ETL development, or a related role.
• Proficiency in SQL and Python, including Pandas, PySpark, and Python functions.
• Experience with cloud data platforms (e.g., Databricks Unity Catalog) and pipeline orchestration tools (e.g. dbt).
• Demonstrated experience with data modeling, feature engineering, and building transforms for ML feature stores.
• Strong understanding of data modeling, warehousing concepts, and relational databases.

Responsibilities

• Design, build, and maintain robust real-time and batch data pipelines to ingest, transform, and deliver structured, semi-structured, and unstructured data from multiple sources
• Develop and manage metadata-driven, real-time streaming pipelines and distributed data processing workflows at scale
• Build and expand the feature store through data modeling, pipeline development, feature building, and data transforms that accelerate ML model readiness
• Contribute to the design and build of pricing semantic layers, ontologies, and knowledge graphs to ensure consistent, reusable data definitions across the organization
• Support Data Marketplace development, enabling teams to discover, access, and share data products across the organization
• Develop cognitive search engine capabilities and support Data Syndication to distribute data across internal and external consumers
• Build and expand the feature store through data modeling, pipeline development, feature building, and data transforms
• Build data applications and support writeback capabilities that enable business users to interact with and act on data directly
• Optimize database performance and query efficiency to support large-scale analytics workloads
• Partner with cross-functional teams to understand data needs and translate them into scalable engineering solutions

Skills

SQL

Python

Pandas

PySpark

Data Modeling

Data Warehousing

Education

Bachelor's degree in Computer Science, Engineering, or related field

Tools

Databricks Unity Catalog

dbt

Databricks Delta Sharing

Great Expectations

Streamlit

Hex

Job description

INSPYR Solutions seeks a Senior Data Engineer to design and maintain scalable data pipelines and infrastructure powering analytics, ML, and BI initiatives. You will work with data scientists, analysts, and stakeholders to ensure high-quality data for pricing optimization and decision-making.

The role emphasizes accelerating feature store expansion through robust data modeling, pipeline development, and feature engineering, while contributing to pricing semantic frameworks and knowledge graphs.