Senior Data Pipeline Engineer

Candid; United States; Remote; USD 100,000 - 130,000; Full time

Benefits offered by this job

Health insurance

Retirement match

Life insurance

PTO

Disability

Transit benefits

Flexible spending accounts

PSLF eligibility

Job summary

Candid is a nonprofit focused on data and insights for the social sector. We seek a Senior Data Pipeline Engineer to lead day-to-day development and delivery of data pipelines, blending SQL/ETL expertise with cloud data platform experience to move data into our warehouse and lakehouse.

Reporting to the Data Operations Manager, this 35-hour-per-week role is remote with occasional in-person summits. You will architect, build, and tune ETL/ELT pipelines and mentor engineers while upholding data

Qualifications

• 5+ years of experience in database engineering, data engineering, or ETL development with production pipelines.
• Expert-level Transact-SQL skills (stored procedures, views, indexing) and hands-on C#/.NET development experience.
• Proven experience architecting ETL processes with SSIS; experience with orchestration platforms such as Apache Airflow preferred.

Responsibilities

• Pipeline development: Architect, develop, and maintain ETL/ELT and C#/.NET pipeline services that move data into the warehouse and lakehouse.
• Cloud platform & performance: Support evolving cloud data platform, including Iceberg ingestion and transformation pipelines, DynamoDB, and query tuning via Trino/Starburst.
• Data quality & incident resolution: Monitor data quality end-to-end and resolve production pipeline issues, documenting findings and driving fixes.

Skills

Transact-SQL

C#/.NET

ETL

Data engineering

AWS

Tools

SSIS

Airflow

AWS Lambda

DynamoDB

Iceberg

Parquet

Trino/Starburst

Job description

Candid is a nonprofit that provides the most comprehensive data and insights about the social sector. We get you the information you need to do good. Candid currently has an opportunity for a Senior Data Pipeline. The Senior Data Pipeline Engineer is a senior technical role responsible for the day-to-day development, and delivery of Candid's data pipelines. This role blends deep SQL and ETL engineering expertise with hands-on experience across modern cloud data platform technologies to design, build, and tune pipelines that move data reliably from source systems into Candid's data warehouse and lakehouse.

Reporting to:

Data Operations Manager

Schedule:

35-hour work week, Monday through Friday

Compensation

$100,000 - $130,000 (this range is for the NYC area and will be adjusted for other localities; additionally, factors like skills and experience will be considered).

Location:

Remote. In-person attendance is expected twice per year during our annual, weeklong all-staff summits. Additional in-person meeting participation is expected at least once per quarter for senior leaders and at least once per month for the executive team. Staff not located in the NYC area are expected to travel for these meetings.

Benefits

• Health insurance (medical, dental, vision)
• retirement contribution with additional option for a match
• paid life insurance and AD&D
• paid leave time (PTO, compassionate leave, volunteer, holiday, parental)
• short-term and long-term disability
• pre-tax transit
• flexible spending accounts
• supplemental insurance
• summer hours
• and Public Service Loan Forgiveness (PSLF) program eligible employer.

Applicantsfor this position must be legally authorized to work in the United States without employer sponsorship, now or in the future.

Responsibilities

• Pipeline development: Architect, develop, and maintain ETL/ELT and C#/.NET pipeline services (SSIS, Airflow, AWS Lambda, enhancer and message-processing services) that move data reliably from source systems into Candid's warehouse and lakehouse.
• Cloud platform & performance: Support Candid's evolving cloud data platform, including Iceberg ingestion and transformation pipelines, DynamoDB, and query performance tuning via Trino/Starburst.
• Data quality & incident resolution: Monitor data quality end-to-end and investigate production pipeline issues at the root cause, documenting findings and driving fixes through to reprocessing.
• Source onboarding & schema coordination: Scope and onboard new external and regulatory data sources end-to-end, and contribute to schema and architecture decisions, coordinating changes across ingestion and transformation layers.
• Documentation, mentorship & process: Maintain SOP and technical documentation, mentor and review the work of Associate-level engineers, and participate in the team's release and deployment process.
• Partnership & support: Partner with the Data Operations Manager on architecture and roadmap decisions, and provide escalation-level support for mission-critical pipeline issues, including after-hours as needed.

Requirements

• 5+ years of experience in database engineering, data engineering, or ETL development, with demonstrated ownership of production data pipelines.
• Expert-level Transact-SQL skills (stored procedures, views, indexing) and hands-on C#/.NET development experience.
• Proven experience architecting ETL processes with SSIS; experience with orchestration platforms such as Apache Airflow preferred.
• Experience with AWS and open table formats (Apache Iceberg, Parquet); AWS Lambda, DynamoDB, and Trino/Starburst a plus.
• Strong data quality and root-cause troubleshooting skills, with experience resolving production data incidents.
• Familiarity with source control, CI/CD, and release/deployment processes across Dev/QA/Prod environments.
• Ability to mentor and review the work of other engineers; excellent written and verbal communication.
• Good business judgment, independent prioritization, and the ability to work well in a distributed team.
• Willingness to perform other duties and special projects as needed/requested.
• Sensitivity and respect for racial, gender, sexual orientation, and cultural differences.
• Commitment to Candid's values: driven, direct, accessible, curious, and inclusive.

About Candid

Candid's mission is to get you the information you need to do good.

The world's problems are only growing, and change can't wait. Nonprofits are needed now more than ever, but all too often their work goes without adequate support.

Candid makes it easier and faster for nonprofits and funders to connect in pursuit of solutions to change the world. Candid is where nonprofits find grants, donors find nonprofits that inspire them, and all can gain insights about the work being done for good.

Candid is a qualifying nonprofit organization as defined by the Public Service Loan Forgiveness Program. As such, Candid employees may claim their employment time on their PSLF application. We offer a competitive salary and excellent benefits. Due to the high volume of applicants we typically receive, we regret that we can only contact candidates we would like to interview.

For more information on positions available at Candid, please visit our website: Work with us

Candid is an equal opportunity employer.

Candid provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.

This policy applies to all terms and conditions of employment, including recruiting, hiring, placement, promotion, termination, layoff, recall, transfer, leaves of absence, compensation, and training.

Notice:

We are aware of emails/communications impersonating Candid's recruitment staff. All legitimate recruiting communications come from a @candid.org email address. If you've received any communications claiming to be Candid that does not come from a @candid.org email address, please forward to [e-mail]. We appreciate your interest in Candid.

Equal Opportunity Employer/Protected Veterans/Individuals with Disabilities This employer is required to notify all applicants of their rights pursuant to federal employment laws. For further information, please review the Know Your Rights notice from the Department of Labor.