Staff Data Engineer

BlackLine; New York (NY); On-site; USD 193,000 - 242,000; Full time

Job summary

BlackLine is seeking a Lead Data Engineer to design, build, and optimize data pipelines powering AI-driven accounting agents. You will lead scalable data infrastructure, work with cross-functional teams, and drive data quality, reliability, and governance across diverse data sources.

The role emphasizes extracting, transforming, and loading data from multiple sources, implementing robust integrations, and collaborating with data scientists and analysts to turn data into actionable business

Qualifications

• 4+ years of experience as a data engineer or similar role
• Bachelor’s degree in Computer Science, Engineering or related field
• Expertise in PySpark and data transformation pipelines; familiarity with AI technologies and cloud platforms (AWS, Azure) is a plus

Responsibilities

• Lead data pipeline development: Build and maintain PySpark ETL pipelines with high data quality and performance
• Manage integrations: Establish robust connections to client data sources via APIs and tools like FiveTran, Plaid, and BlackLine’s internal connector ecosystem
• Ensure reliability: Monitor pipeline performance, automate testing, and validate data accuracy
• Optimize for scale: Implement performance improvements for large-scale datasets
• Collaborate with cross-functional teams to deliver valuable insights and design data security/governance protocols

Skills

PySpark

Data pipeline development

API integrations

Cloud platforms

Communication skills

Education

Bachelor’s degree in Computer Science or related field

Tools

FiveTran

Plaid

Internal connector ecosystem

Job description

It's fun to work in a company where people truly believe in what they're doing!

At BlackLine, we're committed to bringing passion and customer focus to the business of enterprise applications.

Since being founded in 2001, BlackLine has become a leading provider of cloud software that automates and controls the entire financial close process. Our vision is to modernize the finance and accounting function to enable greater operational effectiveness and agility, and we are committed to delivering innovative solutions and services to empower accounting and finance leaders around the world to achieve Modern Finance.

Being a best-in-class SaaS Company, we understand that bringing in new ideas and innovative technology is mission critical. At BlackLine we are always working with new, cutting edge technology that encourages our teams to learn something new and expand their creativity and technical skillset that will accelerate their careers.

Work, Play and Grow at BlackLine!

We’re looking for a Lead Data Engineer to design, build, and optimize data pipelines that power our next-generation AI-driven accounting agents. You’ll lead the development of scalable, high-performance data infrastructure while collaborating closely across teams.

Responsibilities:

• Lead data pipeline development: Build and maintain PySpark ETL pipelines with high data quality and performance
• Manage integrations: Establish robust connections to client data sources via APIs and tools like FiveTran, Plaid, and BlackLine’s own internal connector ecosystem
• Ensure reliability: Monitor pipeline performance, automate testing, and validate data accuracy
• Optimize for scale: Implement performance improvements (e.g., CDC mechanisms, indexing strategies) for large-scale datasets
• Collaborate & innovate: Work with business stakeholders to refine data requirements and integrate cutting-edge AI and big data technologies
• Design and build infrastructure for optimal extraction, transformation, and loading (ETL) of data from a wide variety of data sources, including data identification, mapping, aggregation, conditioning, cleansing, and analyzing.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver valuable insights.
• Effectively communicate complex technical concepts to non-technical audiences.
• Desire to automate everything.
• Maintain documentation and operational knowledge base.
• Coach and technically train junior staff on design and development standards and best practices.
• Design and implement data security and governance protocols to ensure the accuracy and reliability of data.

Qualifications:

Years of Experience in Related Field: 4+ years of experience as a Data Engineer or a similar role

Education: Bachelor’s degree in Computer Science, Engineering or a related field

Technical/Specialized Knowledge, Skills, and Abilities:

• Technical skills: Expertise in PySpark and data transformation pipelines. Familiarity with AI technologies and cloud platforms (AWS, Azure) is a plus
• Communication: Strong communication skills and a proactive, ownership-driven mindset
• Adaptability: Thrive in a fast-paced, in-office-first startup environment

Salary Range:

USD $193,000.00/Yr. - USD $242,000.00/Yr.

Pay Transparency Statement:

Placement within this range depends upon several factors, including the applicant's prior relevant job experience, skill set, and geographic location. In addition to base pay, BlackLine also offers short-term and long-term incentive programs, based on eligibility, along with a robust offering of benefit and wellness plans.

BlackLine is committed to creating an inclusive and accessible experience for all candidates. If you require a reasonable accommodation that would better enable your success during the application or interview process, please complete this form.

Accommodations:

BlackLine is committed to creating an inclusive and accessible experience for all candidates. If you require a reasonable accommodation that would better enable your success during the application or interview process, please complete this form.