Senior Data Engineer

United Network for Organ Sharing (UNOS); Richmond (VA); On-site; USD 120,000 - 180,000; Full time

Job summary

United Network for Organ Sharing (UNOS) is seeking a Senior Data Engineer to design and implement end-to-end data solutions in a modern Azure-based cloud environment. You will build Data Lakehouse, scalable pipelines, and integrations while mentoring junior engineers and leading data initiatives.

You will partner with data scientists, analysts, and stakeholders to ensure data is accurate, accessible, and optimized for analytics, reporting, and operations, with a focus on reliability and

Qualifications

• 8+ years of hands-on experience in data engineering for enterprise-scale solutions.
• 2+ years of experience developing and operating Azure cloud-native data platforms.
• Strong expertise with Azure Data Factory, Databricks and related services.

Responsibilities

• Architect and implement secure, scalable data pipelines in Azure.
• Design and maintain ETL/ELT pipelines for large volumes of data.
• Lead data architecture discussions and performance tuning.
• Mentor junior engineers and lead code reviews.
• Collaborate with data scientists, analysts, and BI teams.
• Drive CI/CD adoption and automated data quality checks.
• Ensure data quality, monitoring, and availability in production.

Skills

MS SQL Server

Python (pandas, PySpark)

Azure Data Factory

Azure Functions

Azure Data Lake Storage

Azure Databricks

Spark

Delta Lake

Data Lakehouse

ETL/ELT pipelines

Git

CI/CD pipelines

REST APIs

Data warehousing

Education

4-year degree in computer science, engineering or other related IT field of study, or equivalent professional work experience

Tools

Databricks

Azure DevOps

Job description

Position Description

We are seeking a Senior Data Engineer to lead the design and implementation of end-to-end data solutions within a modern Azure-based cloud environment. This hands-on, technical role will partner closely with data scientists, analysts, and business stakeholders to ensure that data is accurate, accessible, and optimized for analytics, reporting, and operational needs. You will play a key role in building Data Lakehouse, scalable data pipelines and integrations while championing best practices, mentoring junior engineers, and leading strategic data initiatives across teams.

We are seeking a Senior Data Engineer to lead the design and implementation of end-to-end data solutions within a modern Azure-based cloud environment. This hands-on, technical role will partner closely with data scientists, analysts, and business stakeholders to ensure that data is accurate, accessible, and optimized for analytics, reporting, and operational needs. You will play a key role in building Data Lakehouse, scalable data pipelines and integrations while championing best practices, mentoring junior engineers, and leading strategic data initiatives across teams.

Key Responsibilities

• Architect and implement secure, scalable data pipelines using Azure Data Factory, Azure Functions, and Azure Data Lake Storage
• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks (PySpark/Spark SQL) to ingest, transform, and process large volumes of structured and unstructured data
• Build and manage data pipelines using Azure services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2), Azure Event Hubs, and Azure SQL Database
• Implement data quality, validation, and monitoring frameworks to ensure reliability and accuracy of data pipelines
• Collaborate cross-functionally with data scientists, analysts, and business stakeholders to understand data requirements and deliver fit-for-purpose data solutions
• Lead modernization and optimization efforts, improving pipeline performance, maintainability, and scalability
• Support CI/CD practices for data pipelines using tools such as Azure DevOps, Git, and Databricks Repos
• Detect and resolve data quality issues; implement automated audits and monitoring processes
• Troubleshoot and resolve production data pipeline issues, ensuring high availability and minimal downtime
• Act as a technical leader on schema design, performance tuning, and Azure data architecture
• Mentor and support junior engineers across data engineering, analytics, and BI teams
• Participate in and lead code reviews, promoting clean, well-documented, and testable code
• Stay current on trends in data engineering and cloud technologies, identifying opportunities to innovate

Minimum Requirements

• 8+ years of hands-on experience in data engineering, including designing and implementing enterprise-scale data solutions.
• 2+ years of experience developing and operating Azure cloud-native data platforms.

Critical Skills

• Expertise in MS SQL Server, Python (pandas, PySpark), Azure Data Factory, Azure Functions and Azure Data Lake Storage.
• Strong expertise with Azure Databricks, including Spark (PySpark/Scala), Delta Lake, and cluster/job optimization.
• Solid understanding and hands-on experience building Data Lakehouse architecture
• Experience working with a variety of file formats (e.g., CSV, JSON, XML, Parquet).
• Experience with version control (Git) and CI/CD pipelines for data engineering workflows
• Familiarity using REST APIs for data extraction and integration.
• Proven experience designing and implementing data solutions.
• Strong understanding of cloud architecture, data warehousing and modern data stack components.

Additional Skills & Qualifications

• Demonstrated ability to perform root cause analysis on data and processing issues
• Strong problem-solving skills with the ability to explain technical concepts to non-technical audiences
• A successful history of manipulating, processing and extracting value from large disparate datasets
• Experience with Big Data technologies such as Databricks, Spark, or Azure Synapse
• Knowledge of CI/CD workflows, version control, and agile development practices
• Familiarity with data governance, privacy, and compliance frameworks
• Experience with data warehousing, analytics tools, and BI platforms
• Familiarity with streaming data technologies (Azure Event Hubs, Kafka, Structured Streaming)

Education

• 4-year degree in computer science, engineering or other related IT field of study, or equivalent professional work experience

Physical Requirements

• General office demands
  • Prolonged periods of sitting at a desk and working on a computer.
  • Frequent reaching, handling, and fine manipulation for using office equipment, filing, and managing paperwork.
  • Manual dexterity sufficient to operate a keyboard, mouse, and other office tools.
  • Occasional standing, walking, and bending.
  • Ability to lift up to 10-20 pounds occasionally.
  • Vision abilities required include close vision for computer work and reading documents.
  • Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.