Lead Data Engineer

Selby Jennings; New York (NY); In loco; USD 175.000 - 250.000; Tempo pieno

Descrizione del lavoro

Selby Jennings in New York seeks an experienced data engineering leader to develop cloud-based data platforms and pipelines. You will drive design and production of scalable data solutions spanning market, alternative, and enterprise datasets.

The role requires strong Python/SQL, Snowflake expertise, and hands-on AWS deployment, with opportunities to mentor a growing team and shape platform strategy.

Competenze

• Experience building and supporting enterprise-scale data platforms and pipelines.
• Strong programming skills in Python and SQL.
• Expertise with Snowflake or a similar modern cloud data warehouse.
• Hands-on experience designing and deploying solutions within AWS.
• Experience with workflow orchestration tools such as Airflow.
• Knowledge of relational and NoSQL database technologies.
• Strong understanding of data modeling, data architecture, and large-scale data processing.
• Proven ability to lead technical initiatives and deliver complex projects.

Mansioni

• Lead the development of cloud-based data platforms, pipelines, and data products leveraging Python, SQL, Snowflake, and AWS.
• Build scalable solutions for a variety of data sources, including market, alternative, and enterprise datasets.
• Enhance the firm's data platform through APIs, automation, metadata management, and emerging AI-driven capabilities.
• Improve data reliability through quality controls, monitoring, testing, and operational best practices.
• Troubleshoot complex data issues and implement long-term solutions to improve platform stability.
• Design efficient data models and datasets that support analytics, reporting, and quantitative research.
• Collaborate with investment, research, and technology teams to deliver scalable data solutions.
• Provide technical leadership through architecture guidance, code reviews, mentorship, and engineering best practices.
• Drive projects from design through production while influencing the future direction of the data platform.
• Opportunity to mentor and potentially lead a growing team of engineers.

Conoscenze

Python

SQL

Data modeling

Leadership

Go

Strumenti

Snowflake

AWS

Airflow

Descrizione del lavoro

Salary: USD175000 - USD250000 per year

Responsibilities

• Lead the development of cloud-based data platforms, pipelines, and data products leveraging Python, SQL, Snowflake, and AWS.


• Build scalable solutions for a variety of data sources, including market, alternative, and enterprise datasets.


• Enhance the firm's data platform through APIs, automation, metadata management, and emerging AI-driven capabilities.


• Improve data reliability through quality controls, monitoring, testing, and operational best practices.


• Troubleshoot complex data issues and implement long-term solutions to improve platform stability.


• Design efficient data models and datasets that support analytics, reporting, and quantitative research.


• Collaborate with investment, research, and technology teams to deliver scalable data solutions.


• Provide technical leadership through architecture guidance, code reviews, mentorship, and engineering best practices.


• Drive projects from design through production while influencing the future direction of the data platform.


• Opportunity to mentor and potentially lead a growing team of engineers.

Qualifications

• Experience building and supporting enterprise-scale data platforms and pipelines.


• Strong programming skills in Python and SQL.


• Expertise with Snowflake or a similar modern cloud data warehouse.


• Hands-on experience designing and deploying solutions within AWS.


• Experience with workflow orchestration tools such as Airflow.


• Knowledge of relational and NoSQL database technologies.


• Strong understanding of data modeling, data architecture, and large-scale data processing.


• Proven ability to lead technical initiatives and deliver complex projects.

Preferred Qualifications

• Experience with Go or other backend development languages.


• Background in platform engineering or service-oriented architectures.


• Exposure to AI/LLM-powered data solutions and automation.


• Experience within financial services, investment management, trading, or market data environments.


• Track record of building highly reliable and scalable data systems.