Staff Data Engineer

Nayya; New York (NY); On-site; USD 185,000 - 225,000; Full time

Job summary

An innovative firm is seeking a Staff Data Engineer to lead the design and implementation of scalable data systems. This role involves developing data pipelines, enhancing data enrichment services, and collaborating across teams to drive data-driven decision-making. Join a dynamic environment where you can make an immediate impact on data infrastructure and strategy. If you thrive in fast-paced, high-growth settings and are passionate about data engineering, this opportunity is perfect for you. Be part of a mission to transform complex benefits experiences and help people thrive in their health and wealth.

Qualifications

• 7+ years in data engineering with strong Python and PySpark skills.
• Experience with data pipelines and AWS Glue for data integration.

Responsibilities

• Lead the design and implementation of scalable data systems.
• Develop a centralized data strategy and enhance data enrichment services.

Skills

Python

PySpark

RDBMS

SQL Optimization

Apache Spark

AWS Glue

Terraform

Workflow Orchestration

Monitoring and Observability

Data Quality Measures

Education

Bachelor's Degree in Computer Science or related field

Tools

Airflow

Apache Hudi

AWS Managed Services

Redshift

Job description

Founded in 2019, Nayya is on a mission to connect people’s most important information, so they can thrive in their health and wealth. Powered by AI and advanced analytics, Nayya’s platform transforms complex benefits experiences into intuitive, seamless, and ongoing interactions—meeting people meeting people's real world needs. As a trusted platform and partner to leading employers, benefits solutions, and HR tech providers, Nayya unlocks long-term value through helping employees live more resilient lives. Backed by strategic investors like ICONIQ, Felicis Ventures, SemperVirens, Workday Ventures, MetLife Nextgen Ventures, and ADP Ventures, Nayya is ushering in the future of health and wealth for all.

About the Role
We are seeking a highly skilled and motivated Staff Data Engineer to join our growing team at Nayya. In this role, you will lead the design and implementation of scalable data systems and pipelines that power our data extraction and integration services, while also developing a centralized data strategy. You will work on building batch, event processing, and stream processing infrastructure, enhancing our data enrichment services, developing a robust, de-identified analytics platform for our Data Science, BI, and Analytics teams to consume, and enabling our entire organization with data by developing easy access patterns. We are looking for a data expert who thrives in an environment that values impatience, excellence, resilience, and courage—a leader ready to make an immediate impact on our data infrastructure in a fast-paced, high-growth environment.

As a Staff Data Engineer, you will play a key role in shaping our data systems' architecture, reliability, and performance while fostering innovation and collaboration across teams. This position provides an exciting opportunity to drive technical strategy and lead efforts to solidify and scale our data infrastructure.

Objectives and Responsibilities

Technical Leadership & Data Infrastructure

• Centralized Data Strategy: Develop a single source of truth for organizational data, driving data validation, governance, and improved access for analytical and operational use.
• Build, Improve, and Maintain Data Systems: Lead the development of scalable data pipelines that handle high-volume batch and streaming data.
• Data API and Eventing Development: Enhance and maintain APIs and event driven architecture to provide efficient and reliable access to internal and external data consumers.

Anonymization and Tokenization Development

• Build utilities and workflows to de-identify data, link disparate sources, and build a holistic view of entities across data sources.
• Data Enrichment & Integration: Implement data enrichment solutions at scale that interface with third-party data sources to enhance product capabilities.
• Analytics & Reporting Platform: Improve our reporting and analytics platform while treating security and compliance as a top priority.
• Cross-Functional Collaboration: Work closely with product, engineering, business, and infrastructure teams to design solutions that meet evolving business and technical needs. Advocate for data-driven decision making.
• Mentor and Develop: Provide guidance and mentorship to engineers, fostering a culture of continuous learning and growth.
• Lead by Example: Identify and evaluate our current processes, documentation, workflows and governance and make recommendations and plans for improvements. Lead with documentation.
• Optimize Performance: Focus on tuning, performance testing, and optimization of the data platform.
• Innovate with Agility: Embrace a growth mindset, iterating on data infrastructure and processes to ensure scalability and reliability.
• Ensure Security and Scalability: Identify gaps and risk in current infrastructure to solidify the data platform.

Skills and Qualifications

• 7+ years of experience in data engineering, data infrastructure, or related roles.
• Strong experience with Python and PySpark.
• Strong experience with RDBMS.
• Proficiency with workflow orchestration tools (Airflow, Dagster, etc.).
• Experience implementing data pipelines using Apache Spark, AWS Glue, or EMR.
• Expertise in SQL optimization, query performance tuning, and data warehousing.
• Experience with infrastructure as code tools such as Terraform.
• Experience with AWS suite of data engineering managed services and OSS tools.
• Familiar with Domain Driven Design.
• Experience with monitoring and observability frameworks and tools.
• Familiarity with data quality measures, tools, and frameworks.
• Ability to identify tradeoffs for warehousing vs data lake infrastructure and applying solutions to the appropriate use case.
• Ability to communicate highly technical topics to non-technical stakeholders.
• Familiar with common pitfalls in high volume, partitioned data ingestion pipelines such as orphaned records and table locks.

Preferred Qualifications

• Experience with Apache Hudi or similar data lake platforms.
• Experience with provisioning and managing Redshift.
• Experience with federated query engines.
• Experience with data catalogues.
• Experience with claims data.
• Experience with MLOps engineering and best practices.
• Experience with data governance over PHI and other sensitive information.
• Experience in fast-paced startup environments or high-growth companies.

The salary range for New York based candidates for this role is $185,000 - $225,000. We use a location factor to adjust this range for candidates that are located outside of geographic region of our New York office. Placement within the salary band is determined based on experience.

Nayya is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, gender (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics.