Senior Data Engineer

Selby Jennings; New York (NY); On-site; USD 200,000 - 400,000; Full time

Job summary

Selby Jennings is seeking an experienced Data Engineer in New York to build and optimize scalable data pipelines and products for analytics, reporting, and AI applications.

You will establish engineering standards, improve data quality and observability, and integrate complex datasets while partnering with stakeholders to deliver impactful solutions. Leadership and mentoring are encouraged in this senior role.

Qualifications

• 6+ years of data engineering experience in production environments.
• Advanced Python and SQL skills.
• Strong experience with Databricks, Snowflake, or similar cloud data platforms.
• Hands-on experience with Spark/PySpark and large-scale data processing.
• Deep understanding of data modeling, ETL/ELT, and data warehousing concepts.
• Experience building data quality, validation, and monitoring solutions.
• Comfortable working with complex, imperfect datasets and solving data integrity challenges.
• Strong communication skills and ability to operate independently.
• Bachelor's degree in a technical or quantitative field.

Responsibilities

• Build and optimize scalable data pipelines and data products.
• Develop reliable datasets for analytics, reporting, and AI applications.
• Establish data engineering standards, frameworks, and best practices.
• Improve data quality, monitoring, testing, and platform observability.
• Integrate and reconcile complex datasets from multiple sources.
• Partner with business stakeholders and technical teams to deliver impactful solutions.
• Drive automation and operational excellence across the data platform.
• Mentor engineers and contribute to technical leadership within the team.

Skills

Advanced Python

SQL

Data pipelines

Data modeling

ETL/ELT

Data warehousing

Data quality/monitoring

Communication

Independent work

Education

Bachelor's degree in a technical or quantitative field

Tools

Databricks

Snowflake

Spark/PySpark

Job description

Salary: USD200000 - USD400000 per year

Responsibilities

• Build and optimize scalable data pipelines and data products.
• Develop reliable datasets for analytics, reporting, and AI applications.
• Establish data engineering standards, frameworks, and best practices.
• Improve data quality, monitoring, testing, and platform observability.
• Integrate and reconcile complex datasets from multiple sources.
• Partner with business stakeholders and technical teams to deliver impactful solutions.
• Drive automation and operational excellence across the data platform.
• Mentor engineers and contribute to technical leadership within the team.

Qualifications

• 6+ years of data engineering experience in production environments.
• Advanced Python and SQL skills.
• Strong experience with Databricks, Snowflake, or similar cloud data platforms.
• Hands-on experience with Spark/PySpark and large-scale data processing.
• Deep understanding of data modeling, ETL/ELT, and data warehousing concepts.
• Experience building data quality, validation, and monitoring solutions.
• Comfortable working with complex, imperfect datasets and solving data integrity challenges.
• Strong communication skills and ability to operate independently.
• Bachelor's degree in a technical or quantitative field.

Preferred

• Background working with financial, healthcare, life sciences, or alternative data.
• Experience with CI/CD, Terraform, and modern data engineering tooling.
• Exposure to AI/ML platforms and data products.
• Previous leadership or mentoring experience.