Senior Data Engineer

Node.Digital LLC

Washington (District of Columbia)

Remote

USD 140,000 - 190,000

Full time

6 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Medical
Dental
Vision
Basic Life
Health Saving Account
401K matching
Three weeks of PTO/Sick
11 Paid Holidays
Pre-Approved Online Training

Job summary

Node.Digital LLC in Herndon, VA (Remote Work) seeks a Senior Data Engineer to architect and optimize data pipelines and architectures for products and analysts. You will implement ELT/ETL in Azure Synapse and Azure ML, migrate data to Azure Data Lake, and ensure robust source control and QA across all pipelines.

The role requires a public trust clearance, a strong background in SQL, Python, and modern data stacks, and a track record of scalable, secure data solutions.

Qualifications

  • Bachelor’s degree in data engineering, CS, data science, ML, math, or related field.
  • Alternatively, five years of applied work experience in any of the same fields.
  • Experience with cloud-based ELT/ETL in data analytics environments.

Responsibilities

  • Provide authoritative expertise on data engineering methods and best practices.
  • Design, implement, and maintain data architecture with source control.
  • Develop ELT/ETL pipelines in Azure Synapse and Azure ML using SDKs.
  • Migrate data into Azure Data Lake Storage and normalize entity attributes.
  • Review and improve architecture, pipelines, and dependencies; address bottlenecks and drift.
  • Establish quality controls, logging, and validation checks across pipelines.
  • Ensure source control across pipelines and analytics codebases.
  • Optimize ingestion, processing and storage for diverse datasets including Parquet.
  • Develop self-service capabilities for analysts to query and export data.
  • Document data architecture, dictionaries, ER diagrams, and process maps.

Skills

Data engineering
Azure Synapse
Azure Machine Learning
ETL/ELT pipelines
Python
PySpark
Polars
Pandas
SQL

Education

Bachelor's degree in data engineering, computer science, data science, ML, mathematics, or related field
5 years of applied work experience in related fields

Tools

Azure Data Lake Storage
Azure Synapse
Azure Machine Learning
Python SDK
Git
Parquet

Job description

Location: Herndon, VA (Remote Work)
Must have an Public Trust Clearance
  • Provide authoritative expertise on data engineering methods and best practices, including code first development approaches and modern pipeline design patterns.
  • Design, implement, and maintain the data architecture that supports products and end users, with all assets managed under source control.
  • Design, implement, and maintain ELT and ETL pipelines for efficient processing of source data in Azure Synapse and Azure Machine Learning, using both SDK V1 and SDK V2.
  • Migrate source data identified by SBA OIG into Azure Data Lake Storage.
  • Normalize entity attributes such as addresses, phone numbers, and other common fields.
  • Review, maintain, and improve existing architecture and pipelines, including periodic audits addressing bottlenecks, deprecated dependencies, and architecture drift.
  • Establish quality controls across all pipelines and introduce error handling, logging mechanisms, and validation checks.
  • Incorporate source control across all pipelines and analytics codebases so code can evolve iteratively without destabilizing the architecture.
  • Optimize ingestion, processing, and storage across a wide variety of datasets and data types, including modern columnar formats such as Parquet.
  • Develop self service capabilities that let SBA OIG analysts query and export data for investigations and audits.
  • Author robust standard operating procedures governing the authoring, development, validation, publishing, execution, and monitoring of all data pipelines and assets in the Azure environment.
  • Produce detailed documentation of the data architecture, including data dictionaries, entity relationship diagrams, and pipeline process maps.
  • Maintain and expand the environment with additional datasets and services on request, following a defined intake and testing process before production deployment.
  • Stay current with emerging AI tooling relevant to data engineering and contribute to exploratory work evaluating automation and language model assisted capabilities.
Key Responsibilities
  • Provide authoritative expertise on data engineering methods and best practices, including code first development approaches and modern pipeline design patterns.
  • Design, implement, and maintain the data architecture that supports products and end users, with all assets managed under source control.
  • Design, implement, and maintain ELT and ETL pipelines for efficient processing of source data in Azure Synapse and Azure Machine Learning, using both SDK V1 and SDK V2.
  • Migrate source data identified by SBA OIG into Azure Data Lake Storage.
  • Normalize entity attributes such as addresses, phone numbers, and other common fields.
  • Review, maintain, and improve existing architecture and pipelines, including periodic audits addressing bottlenecks, deprecated dependencies, and architecture drift.
  • Establish quality controls across all pipelines and introduce error handling, logging mechanisms, and validation checks.
  • Incorporate source control across all pipelines and analytics codebases so code can evolve iteratively without destabilizing the architecture.
  • Optimize ingestion, processing, and storage across a wide variety of datasets and data types, including modern columnar formats such as Parquet.
  • Develop self service capabilities that let SBA OIG analysts query and export data for investigations and audits.
  • Author robust standard operating procedures governing the authoring, development, validation, publishing, execution, and monitoring of all data pipelines and assets in the Azure environment.
  • Produce detailed documentation of the data architecture, including data dictionaries, entity relationship diagrams, and pipeline process maps.
  • Maintain and expand the environment with additional datasets and services on request, following a defined intake and testing process before production deployment.
  • Stay current with emerging AI tooling relevant to data engineering and contribute to exploratory work evaluating automation and language model assisted capabilities.
Education

Bachelor's degree in data engineering, computer science, data science, machine learning, mathematics, or a related field. Alternatively, five years of applied work experience in any of the same fields.

  • 5 years - Maintaining SQL databases and conducting advanced operations in SQL and T-SQL.
  • 5 years - Designing, implementing, and maintaining ELT and ETL processes in cloud based data analytics environments.
  • 3 years - Working in Azure Synapse and Azure Machine Learning with the modern data stack. Certifications preferred, DP-203 or equivalent.
  • 3 years - Manipulating data in Python. Pandas is required. PySpark and Polars preferred. Experience developing reusable, modular code preferred.
Preferred Qualifications
  • DP-203, Microsoft Certified Azure Data Engineer Associate, or an equivalent current certification.
  • Implementing pipelines and infrastructure using code first approaches: Python SDK, CLI, REST APIs, or infrastructure as code tooling such as Terraform or Bicep.
  • Implementing source control and continuous integration and delivery workflows for data assets.
  • Demonstrated familiarity with AI coding assistants and large language model integration patterns.
  • PySpark or Polars at production scale.
  • Entity resolution and attribute normalization across records with inconsistent addresses, names, and identifiers.
  • Building self service analytic access for non engineering users.
Benefits

We are proud to offer competitive compensation and benefits packages to include

  • Medical
  • Dental
  • Vision
  • Basic Life
  • Health Saving Account
  • 401K matching
  • Three weeks of PTO/Sick
  • 11 Paid Holidays
  • Pre-Approved Online Training
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer - Azure ELT & ML Pipelines (Remote)
Senior Data Engineer - Azure ELT & ML Pipelines (Remote)

Node.Digital LLC • Herndon (VA)

Hybrid
USD 120,000 - 160,000
Medical
Dental
Vision
+6
Senior Data Engineer
Senior Data Engineer

Node.Digital LLC • Herndon (VA)

Remote
USD 120,000 - 160,000
Medical
Dental
Vision
+6
Azure Data Integration Engineer
Azure Data Integration Engineer

i4DM • Millersville (MD)

On-site
USD 120,000 - 160,000
Healthcare coverage
HSA Plan
Life and Disability Insurance
+1
Senior Data Architect & Engineer - Enterprise Pipelines
Senior Data Architect & Engineer - Enterprise Pipelines

Ares Enterprise • Washington

Hybrid
USD 120,000 - 180,000
Data Engineer
Data Engineer

TG Administration LLC • Scottsdale (AZ)

On-site
USD 95,000 - 130,000
Healthcare Benefits
Retirement Benefits
Time Off
Data Scientist
Data Scientist

PlanIT Group • Reston (VA)

On-site
USD 100,000 - 160,000
Senior Data Engineer / Data Architect | Hybrid | Camp Murray, WA | Contingent
Senior Data Engineer / Data Architect | Hybrid | Camp Murray, WA | Contingent

Ares Enterprise • Washington

Hybrid
USD 120,000 - 180,000
Data Engineer
Data Engineer

SteerBridge, Inc. • Vienna (VA)

On-site
USD 130,000 - 180,000
Data Operations and Development Lead
Data Operations and Development Lead

Catapult Solutions Group • Baltimore (MD)

Remote
USD 107,452
Senior Data Engineer
Senior Data Engineer

HumCap • Plano (TX)

Hybrid
USD 120,000 - 170,000