Senior Data Engineer

Trepp; New York (NY); On-site; USD 167,000 - 204,000; Full time

Benefits offered by this job

Medical, Dental, Vision

401K with employer match

Life and disability insurance

Flexible PTO

Parental leave

Wellness subsidy

Pet insurance

Laptop + WFH equipment

Career progression

Pre-tax commuter benefit (NYC)

Company events

Job summary

Trepp, an NYC-based fintech data leader, seeks a Data Platform Pipeline Engineer to design and maintain scalable data pipelines powering CRE and banking analytics.

You will collaborate with multiple product and engineering teams to deliver robust pipelines, optimize performance, and embed data quality controls while advancing AI-driven workflows and LLM orchestration within our AWS cloud environment.

Qualifications

• Bachelor’s Degree required.
• 5+ years of application development experience.
• 3+ years in big data technologies.
• Dimensional data modeling and enterprise data warehouses/lakes experience.
• 3+ years of cloud computing (AWS, Azure, Google Cloud).
• Experience with Agile, SDLC, Git workflows, and CI/CD.
• Hands-on data pipelines on AWS core services (S3, EMR, Glue, Lambda, Step Functions).

Responsibilities

• Design, build, and maintain scalable batch and near real-time data pipelines powering Trepp’s data-driven products.
• Lead complex data engineering projects from planning through delivery with stakeholders.
• Translate business requirements into data models, designs, and production-ready implementations.
• Embed data quality, validation, monitoring, and reliability checks into pipelines.
• Develop domain expertise to shape roadmaps and prioritize initiatives.
• Break down ideas into actionable tasks, plans, and milestones.
• Evaluate options, justify tradeoffs, and propose business-impacting solutions.
• Champion best practices for scalability, maintainability, and performance.
• Design and integrate AI-driven workflows including LLM orchestration and vector search pipelines.

Skills

Python

SQL

Scala

Java

AWS

Big data technologies

Distributed data processing

Education

Bachelor's Degree

Tools

MapReduce

Hadoop

Hive

EMR

Kafka

Spark

Terraform

CloudFormation

CDK

S3

Lambda

Job description

(This role is based in Trepp’s NYC office and follows a hybrid work schedule)

The Data Platform Team powers all products at Trepp by using distributed data technologies built on our data lake. The Pipeline Team plays a crucial role in designing and maintaining scalable, performant data pipelines that support all product lines at Trepp.

About this role:

As a member of the Data Platform Pipeline Team, you’ll be using open-source and cloud technologies to build, optimize, and maintain robust data pipelines. You’ll collaborate with multiple product teams and engineering teams to solve complex problems in our financial and analytical products for CRE and Banking.

Responsibilities:

• Design, build, and maintain scalable batch and near real-time data pipelines that power Trepp’s data-driven products.

• Lead complex data engineering projects from planning through delivery, while managing stakeholder expectations and timelines.

• Translate high-level business and product requirements into data models, technical designs, and production-ready implementations.

• Embed data quality, validation, monitoring, and reliability checks into pipelines to ensure trusted production data.

• Develop deep subject matter expertise in internal frameworks and use that expertise to shape roadmaps and prioritize initiatives.

• Break down ambiguous ideas into actionable tasks, project plans, and clear execution milestones.

• Evaluate technical options through research, clearly articulate tradeoffs, and recommend solutions grounded in business and engineering impact.

• Champion data engineering best practices that improve scalability, maintainability, performance, and platform innovation.

• Design and integrate AI-driven workflows, including LLM orchestration, embedding generation, and vector search pipelines, into core data processing systems.

• Architect and operate reliable data ingestion and transformation pipelines using native AWS cloud infrastructure.

Basic Qualifications:

• Bachelor’s Degree
• At least 5 years of experience in application development
• At least 3 years of experience in big data technologies
• Demonstrated experience in dimensional data modeling and relational schema design for enterprise data warehouses and data lakes
• At least 3 years of experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
• Experience with Agile and SDLC, Git workflows, and CI/CD
• Hands-on experience building and deploying data pipelines on AWS core services (S3, EMR, Glue, Lambda, Step Functions)

Preferred Qualifications:

• 5 + years of experience in application development including Python, SQL, Scala, or Java
• 5+ years of experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark)
• 2+ years of experience working on real-time data and streaming applications
• 2+ years of experience working on ML ops or related
• 3+ years of data warehousing experience (Redshift or Snowflake)
• 2+ years of experience with UNIX/Linux including basic commands and shell scripting
• Experience integrating generative AI tooling, LLM APIs (e.g., Amazon Bedrock, SageMaker), or vector databases into production pipelines
• Experience with Infrastructure as Code (IaC) tools such as Terraform or AWS CloudFormation/CDK

Why Join Us:

• Work on a platform that powers data-driven products across Trepp’s CRE and Banking businesses.

• Solve meaningful engineering challenges using modern cloud, big data, and AI technologies at scale.

• Join a collaborative team where you can influence architecture, improve engineering standards, and help shape the future roadmap of the data platform.

Salary Range:

• Basesalarystartingfrom$185kplusbonuseligible

BenefitsandPerks:

• Base+targetbonuscompensationstructure
• Medical,Dental,Visioninsurance
• 401K(withemployermatch)
• Lifeinsurance,longtermdisability,shorttermdisabilityallcoveredbythecompany
• Flexiblepaidtimeoff(PTO)
• Sixteen(16)weekspaidprimarycaregiverleave(Biological,adoptive,andfosterparentsarealleligible)
• Four(4)weekspaidparentalleave
• Wellnesssubsidy
• Petinsurance
• Laptop+WFHequipment
• Careerprogressionplan
• Pre-taxcommuterbenefitwithcompanysubsidy_(ForNYC-officebasedemployeesonly)_
• Funcompanyeventsandvolunteeringopportunities

Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States and with Trepp (i.e., H1-B visa, F-1 visa (OPT), TN visa or any other non-immigrant status).

WorkplacePolicy:

NYC,PA,andLondonoffice-basedpositions: Trepp’sofficesfollowa3-2hybrid-workingpolicywiththeexpectationofin-officeworkonTuesday-ThursdayandtheoptiontoworkfromhomeonMondayandFriday.

About Trepp:

Trepp, Inc. founded in 1979, is the leading provider of information, analytics and technology to the CMBS, commercial real estate and banking markets. Trepp provides primary and secondary market participants with the web-based tools and insight they need to increase their operational efficiencies, information transparency and investment performance. Trepp serves its clients with products and services to support trading, research, risk management, surveillance and portfolio management. Trepp is wholly-owned by DMG Information, a division of the Daily Mail and General Trust (DMGT).