HPC Data Provenance - Postdoctoral Researcher

Lawrence Livermore National Laboratory

Livermore (CA)

On-site

USD 129,000 - 158,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Flexible schedules
Relocation assistance
Education reimbursement

Job summary

Lawrence Livermore National Laboratory (LLNL) is seeking a Postdoctoral Researcher to advance data provenance, multi-layer tracing, and I/O characterization for big HPC and AI workflows. You will develop low-overhead provenance, integrate with Slurm/Flux, and produce scalable data registries and benchmarks.

You will work with domain scientists on Livermore Computing systems, publish in journals, and contribute open‑source software within the CASC unit.

Qualifications

  • PhD in CS, CE, or related field.
  • Strong research background in HPC I/O, data management.
  • Experience with tracing and performance analysis of parallel apps.
  • Experience with large-scale DL or scientific workflows.
  • Proficient in C/C++ and Python in Linux.

Responsibilities

  • Develop low-overhead provenance and I/O capture for HPC workflows.
  • Integrate capture with resource managers like Slurm/Flux.
  • Design scalable registries for datasets and checkpoints.
  • Normalize and align trace data into a common schema.
  • Publish results and contribute open-source software.

Skills

C/C++
Python
Research
Communication
Publication

Education

PhD in Computer Science / Computer Engineering

Tools

Darshan
eBPF
DFTracer
Recorder
Linux kernel
Slurm
Flux

Job description

Company Description

Join us and make YOUR mark on the World!

Lawrence Livermore National Laboratory (LLNL) has turned bold ideas into world-changing impact advancing science and technology to strengthen U.S. security and promote global stability.

Our mission spans four critical national security areas nuclear deterrence, threat preparedness, energy security, and multi-domain defense empowering teams to take on the toughest challenges of today and tomorrow. With a culture built on innovation and operational excellence, LLNL is a place where your expertise can make a real impact.

Job Description

We are seeking a Postdoctoral Researcher to perform research in data provenance, multi-layer tracing, and I/O characterization for large-scale scientific and AI workflows on high-performance computing (HPC) systems. Scientists increasingly need to answer questions such as which data produced a result, what transformed it, and which downstream models and analyses are affected when that data changes. Today the evidence needed to answer these questions is fragmented across hardware, operating system, I/O, runtime, and workflow layers. You will research and develop low-overhead techniques to automatically capture provenance from heterogeneous sources (e.g., application- and workflow-level tracers, data registries, and kernel-level instrumentation such as eBPF), assign stable identities to data artifacts as they move through workflows, and normalize and temporally align these records into a common representation that supports trustworthy, auditable lineage at HPC scale. You will work with domain scientists to validate these capabilities on leading-edge AI-for-science workflows running on Livermore Computing production systems. This position is in the Center for Applied Scientific Computing (CASC) within the Computing Directorate.

In this role you will

  • Research and develop low-overhead, multi-layer provenance and I/O capture techniques for HPC workflows spanning the application, runtime, I/O library, file system, and operating system layers.
  • Research and develop methods to integrate capture with HPC resource managers (e.g., Flux, Slurm) so that provenance is collected automatically without requiring changes to user workflows.
  • Design and implement scalable registries that assign stable identities to datasets, model checkpoints, and workflow stages and reconcile the names used for them across different tools.
  • Research and develop techniques to normalize, compress, and temporally align high-volume heterogeneous trace and event streams into a common schema.
  • Characterize and minimize the performance overhead of tracing and provenance capture on large-scale deep learning and scientific workloads.
  • Develop benchmarks, ground-truth datasets, and validation methodologies to evaluate provenance correctness, coverage, and overhead.
  • Develop high-quality, open-source software that is extensible and maintainable by downstream users and developers.
  • Deploy and evaluate software on Laboratory production HPC systems and interact with application scientists to understand practical, real-world requirements and limitations.
  • Pursue independent (but complementary) research interests and interact with team members internally and externally.
  • Publish research results in external peer-reviewed scientific journals and participate in conferences and workshops.
  • Present formal and informal overview of research progress at group meetings.
  • Participate in existing division collaborations and alliances with academia and industry.
  • Perform other duties as assigned.
Qualifications
  • PhD in Computer Science, Computer Engineering, or related field.
  • Research experience in HPC I/O, storage systems, or scientific data management.
  • Research experience in tracing, profiling, or performance analysis of parallel or distributed applications.
  • Research experience with large-scale deep learning or scientific workflows, either as workloads to characterize or as tools applied to systems problems.
  • Ability to conduct high-quality research and to develop implementations of sophisticated algorithms and systems to evaluate the results.
  • Experience programming in C/C++ and Python in a Linux environment.
  • Proficient verbal and written communication skills necessary to interact with members of a research team, author technical and scientific reports and papers, and deliver technical presentations.
  • Demonstrated publication record in peer-reviewed journals and/or conferences.
  • Strong decision-making skills necessary to independently solve complex research problems.
Qualifications We Desire (optional)
  • Experience with data provenance, lineage tracking, or reproducibility of scientific or machine learning workflows.
  • Experience with operating system internals or kernel-level instrumentation (e.g., the Linux kernel, eBPF).
  • Experience with I/O tracing and characterization tools (e.g., Darshan, DFTracer, Recorder) or I/O benchmarks for deep learning.
  • Experience indexing, compressing, or analyzing large trace or event datasets.
  • Experience running workloads on leadership-class HPC systems, parallel file systems, and HPC resource managers.
  • Experience developing and maintaining open-source research software with an external user community.
Pay Range

$143,328 Annually

Additional Information

All your information will be kept confidential according to EEO guidelines.

Position Information

This is a Postdoctoral appointment with the possibility of extension to a maximum of three years. The postdoctoral appointment is a temporary, mentored training position intended for individuals in the early stages of their postdoctoral research careers. Consistent with the structure of the program, applicants must have received their Ph.D. (or equivalent doctorate) within five years of the anticipated start date.

Why Lawrence Livermore National Laboratory?
  • Included in 2026Best Places to Work by Glassdoor!
  • FlexibleBenefits Package
  • 401(k)
  • Relocation Assistance
  • Education Reimbursement Program
  • Flexible schedules (*depending on project needs)
  • Our values - visithttps://www.llnl.gov/inclusion/our-values
Security Clearance

None required.However, if your assignment is longer than 179 days cumulatively within a calendar year, you must go through the Personal Identity Verification process. This process includes completing an online background investigation form and receiving approval of the background check.

National Defense Authorization Act (NDAA)

The 2025 National Defense Authorization Act (NDAA), Section 3112, generally prohibits citizens of China, Russia, Iran and North Korea without dual US citizenship or legal permanent residence from accessing specific non-public areas of national security or nuclear weapons facilities. The restrictions of NDAA Section 3112 apply to this position. To be qualified for this position, Candidates must be eligible to access the Laboratory in compliance with Section 3112.

Equal Employment Opportunity

We are an equal opportunity employer that is committed to providing all with a work environment free of discrimination and harassment. All qualified applicants will receive consideration for employment without regard to race, color, religion, marital status, national origin, ancestry, sex, sexual orientation, gender identity, disability, medical condition, pregnancy, protected veteran status, age, citizenship, or any other characteristic protected by applicable laws.

Reasonable Accommodation

Our goal is to create an accessible and inclusive experience for all candidates applying and interviewing at the Laboratory. If you need a reasonable accommodation during the application or the recruiting process, please use our online form to submit a request.

CaliforniaPrivacy Notice

The California Consumer Privacy Act (CCPA) grants privacy rights to all California residents. The law also entitlesjob applicants, employees, and non-employee workers to be notified of what personal information LLNL collects and for what purpose. The Employee Privacy Notice can be accessed here.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

HPC Data Provenance – Postdoctoral Researcher
HPC Data Provenance – Postdoctoral Researcher

LLNL • Livermore (CA)

On-site
USD 122,000 - 165,000
HPC Data Provenance – Postdoctoral Researcher
HPC Data Provenance – Postdoctoral Researcher

Lawrence Livermore National Laboratory • Livermore (CA)

On-site
USD 129,000 - 158,000
Computational Engineering Division Graduate Intern - Summer 2027
Computational Engineering Division Graduate Intern - Summer 2027

LLNL • Livermore (CA)

Hybrid
USD 104,000 - 124,000
Holiday Pay
Sick leave accrual
Individual 401(k) contributions
Reduced Order Modeling - Postdoctoral Researcher
Reduced Order Modeling - Postdoctoral Researcher

Llnl • Livermore (CA)

On-site
USD 129,000 - 158,000
Flexible schedules
Numerical Program Analysis for Scientific Computing Postdoctoral Researcher
Numerical Program Analysis for Scientific Computing Postdoctoral Researcher

LLNL • Livermore (CA)

On-site
USD 129,000 - 158,000
Flexible schedules
Reduced Order Modeling - Postdoctoral Researcher
Reduced Order Modeling - Postdoctoral Researcher

Lawrence Livermore National Laboratory • Livermore (CA)

On-site
USD 129,000 - 158,000
Numerical Program Analysis for Scientific Computing Postdoctoral Researcher
Numerical Program Analysis for Scientific Computing Postdoctoral Researcher

Lawrence Livermore National Laboratory • Livermore (CA)

On-site
USD 129,000 - 158,000
SI/ML Computational Biologist - Postdoctoral Researcher
SI/ML Computational Biologist - Postdoctoral Researcher

Lawrence Livermore National Laboratory • Livermore (CA)

On-site
USD 111,000 - 135,000
Relocation Assistance
Education Reimbursement Program
Flexible schedules
+1
Computing Graduate Student Intern - Spring 2027
Computing Graduate Student Intern - Spring 2027

Lawrence Livermore National Laboratory • Livermore (CA)

Hybrid
USD 104,000 - 124,000
Holiday Pay
Sick leave accrual
Individual 401(k) contributions
Data Science Institute Graduate Student Intern - Summer 2027
Data Science Institute Graduate Student Intern - Summer 2027

Lawrence Livermore National Laboratory • Livermore (CA)

Hybrid
USD 97,000 - 131,000
Holiday Pay
Sick leave accrual
401(k) contributions