Data Engineer

VIVA

Denver (CO)

Remote

USD 120,000 - 160,000

Full time

19 hours ago
Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

VIVA USA Inc is seeking a Data Engineer in Colorado to implement data pipelines using Snowflake and Informatica. You will move data from source systems to analytical platforms, handle transformations, data cleansing, and provide structured data for analysts and scientists.

The role emphasizes onboarding into SLDS-like environments, production operations, and data governance, with a focus on performance and cost optimization in a cloud data warehouse.

Qualifications

  • 5+ years building complex Snowflake ETL/ELT pipelines.
  • Experience with Informatica CDGC for metadata and data quality.
  • IaC with Terraform and cloud resource provisioning.
  • CI/CD and DevOps practices using Azure DevOps.
  • Version control with Git and collaborative workflows.

Responsibilities

  • Design, develop, and optimize ETL/ELT data pipelines in Snowflake and Informatica.
  • Coordinate with Data Architect and Data Scientist to power dashboards and analytics.
  • Implement data quality, lineage, and governance checks across pipelines.
  • Maintain production data workflows, troubleshoot failures, and manage releases.
  • Support identity matching and record linkage across datasets.

Skills

Snowflake ETL
Data pipelines
Data governance
Identity matching
DevOps practices

Tools

Informatica CDGC
Terraform
Azure DevOps
Git

Job description

Remote / Telecommuting (Candidate must reside within the State of Colorado; strong preference for local / Denver Metro candidates)

The Data Engineer is responsible for realizing the vision of the Data Architect through the technical implementation of data pipelines using Snowflake and Informatica. Data Engineers are technical coding experts focused on moving data from line-of-business, transactional, or partner agency source systems into analytical data platforms, building transformations, managing data cleansing and quality, and presenting structured data for consumption by analysts and scientists.

In the context of the Statewide Longitudinal Data System an Data Engineer is expected to join the project team in January 2027 under a shared delivery model with a 3rd party implementor. The Data Engineer will participate in a structured knowledge-transfer progression (shadowing, co-running, and checking), leading specific new source ingestions, and progressively will take over production operations and upkeep of the SLDS data foundation and analytics layer.

1. Scope of Responsibilities
  • Data Ingestion & Pipeline Engineering: Design, develop, test, and deploy automated ETL/data pipelines in Informatica and Snowflake to ingest data from partner state agencies and third parties into the SLDS common data model.
  • Transformation & Analytics Layer Support: Implement transformation logic to convert raw agency feeds into harmonized, linked longitudinal records and analytics layer data models, adhering to core naming and modeling standards established with the Data Architect.
  • Record Linkage & Identity Matching: Execute the SLDS internal matching logic to link newly ingested agency records to master longitudinal student and workforce profiles, handling match exceptions and cross-agency reconciliations.
  • Production Pipeline Operations & Incident Response: Maintain operational ownership of scheduled feed runs, monitoring job execution, resolving run failures, reloading late or corrected files, reconciling row counts, and rebuilding downstream objects.
  • Data Quality & Governance Compliance: Implement L1 (structural/format), L2 (completeness/de-duplication), and L3 (referential integrity/logic) validation checks, layout-change detection, and audit trails; enforce data privacy, suppression rules, and compliance standards (FERPA, COPPA, ADA, and state statutes).
  • Performance Optimization & Platform Stewardship: Optimize query execution, warehouse compute, and pipeline performance for cost and latency; identify and tune heavily used models and performance hot spots in Snowflake.
  • Quality Assurance, Code Review & Promotion: Adhere to the unified release process across Development, Test, UAT, and Production environments using Azure DevOps and Azure Test Plans; author and execute test cases, conduct peer code reviews, and maintain release change logs.
2. Activities Performed
A. Ingestion & Pipeline Development (client-Led Feeds)
  1. CDHE Courses Data Ingestion:
    • Ingest and validate annual postsecondary course-level data files from CDHE per data sharing agreements.
    • Build ETL mapping processes from CDHE course files into the SLDS common data model based on design specifications provided by 3rd party implementation partner.
    • Pair with 3 party implementation team during ETL construction, validation, and promotion.
  2. Postsecondary SES & Disability Data Ingestion (Conditional):
    • Ingest and validate demographic, Pell Grant eligibility, and disability classification codes from postsecondary sources.
    • Apply mandated privacy protections, cell suppression rules, and permissible-use governance requirements.
  3. PSAT/SAT Assessment Data Ingestion (Conditional):
    • Ingest, validate, and map section/composite assessment scores and grade levels from the College Board or CDE.
    • Execute identity matching to connect assessment records to existing longitudinal student profiles.
B. Knowledge Transfer & Operational Transition (Jan–April 2027)
  • Shadowing & Co-running (Jan–Feb 2027): Shadow 3rd party implementation partner engineers on day-to-day operations of existing data feeds (e.g., NSC, K-3, CDEC, and Phase 1 feeds); Co-run live feed cycles, handle data loading, execute record matching, and troubleshoot exceptions alongside 3rd party implementation partner.
  • Readiness Gate Execution (Target: March 2027): Run two consecutive feed cycles completely unaided to achieve formal operational hand-over sign-off.
  • Full Production Operations (March 2027 Onward): Execute recurring feed schedules, rebuild downstream objects, reconcile row counts against source files, and triage failed jobs; Address ad-hoc data inquiries and export requests from client leadership and partner agencies.
C. Data Stabilization & Analytics Layer Upkeep
  • Stabilization Backlog Support: Resolve identified data reliability defects, run variations, and de-duplication issues transitioned from the Phase 1 stabilization backlog.
  • Analytics Layer Hardening & Maintenance (April 2027 Onward):
    • Apply object modeling standards, naming conventions, and layout-change detection into the RAW layer.
    • Perform ongoing reconciliation of analytics layer outputs across non-production and production environments.
    • Implement metric and calculation enhancements prioritizing downstream use cases (program participation, completion, equitable access, and wage outcomes).
    • Conduct query profiling and performance tuning on dashboard back-end models.
D. Cross-Functional Collaboration & Testing
  • Testing & Release Management: Author manual and automated test cases in Azure Test Plans; record execution evidence and resolve defect bugs before promoting code.
  • Cross-Role Support:
    • Partner with the Data Architect to review and adapt physical and logical data models and pipeline designs.
    • Support the Data Scientist / Visualization Designer by integrating new data sources to power Tableau dashboard refreshes and preparing source tables for de-identified Institution of Higher Education (IHE) outcome files.
    • Coordinate with Operations Engineers on Snowflake compute management, warehouse auto-shutdown, user role permissions, and CI/CD promotion pipelines.
3. Deliverables Created
Technical Code & Pipeline Artifacts
  • Production ETL/ELT Pipelines: Fully developed, scheduled Informatica workflows and Snowflake SQL/transformation pipelines for CDHE Courses and existing operational feeds.
  • Cleansing & Validation Modules: Automated L1, L2, and L3 quality-check rules and Informatica layout-change detection routines in the RAW layer.
  • Identity Matching & Linkage Routines: Documented internal matching scripts linking new partner agency student/workforce records into longitudinal IDs.
  • Analytics Layer Transformation Objects: Production views, tables, and metric models in Snowflake aligned with longitudinal reporting use cases.
  • De-Identified Data Extraction Scripts: Data extraction scripts that generate de-identified, suppressed outcome files for participating IHEs per specification.
Documentation & Engineering Specifications
  • ETL Pipeline Design Documents: Technical architecture and transformation specifications detailing source-to-target logic, schemas, and dependencies.
  • Mapping & Transformation Specifications: Contribution to enterprise data mapping documents aligning source fields to the SLDS common data model.
  • Lineage Documentation & Dependency Maps: As-built data lineage records and dependency diagrams detailing object-level upstream and downstream flows.
  • Operational Runbooks & Incident Procedures: Standard operating procedures for running feed cycles, handling late or malformed files, performing failover, and reconciling row counts.
Governance, Testing & Operational Reporting
  • Azure Test Plans & Execution Records: Fully documented test plans, test suites, manual test run results, and defect logs for all pipeline modifications prior to production promotion.
  • Release Change Logs: Detailed audit entries for all client-led production releases, detailing changes, rationales, linked Azure DevOps work items, test results, and deployment timestamps.
  • Feed Operation Run Logs & Match Summaries: Run completion records, reconciliation totals, exception logs, and data-quality summaries produced for agency loads.
Required Qualifications & Technical Experience
  • Snowflake ETL/ELT: Minimum 5 years of demonstrated experience building, orchestrating, and optimizing complex ETL/ELT data pipelines and transformation logic natively within the Snowflake platform.
  • Informatica CDGC: Minimum 1 year of hands-on experience working with Informatica Cloud Data Governance and Catalog (CDGC) for metadata management, data profiling, cataloging, and automated data quality validation.
  • Infrastructure as Code (IaC): Minimum 1 year of experience utilizing Terraform to provision, configure, and maintain cloud infrastructure and resources.
  • CI/CD & DevOps: Demonstrated proficiency implementing automated build, test, and release deployment pipelines using Azure DevOps.
  • Version Control: Expert proficiency with Git / version control workflows (branching strategies, pull requests, automated merge checks, code reviews).
Preferred Skills & Experience
  • Prior experience working with public sector, longitudinal education, and workforce data systems (e.g., CDE, CDHE, CDLE, CDEC, National Student Clearinghouse).
  • Experience with record linkage, entity resolution, and deterministic/probabilistic identity matching across disparate public records.
  • Familiarity with federal and state student/workforce privacy regulations, including FERPA, COPPA, and cell suppression standards.
  • Experience authoring and executing structured test plans in Azure Test Plans.
Notes

Work Arrangement: Remote / Telecommuting

VIVA is an equal opportunity employer. All qualified applicants have an equal opportunity for placement, and all employees have an equal opportunity to develop on the job. This means that VIVA will not discriminate against any employee or qualified applicant on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Application Developer - SLDS Data Engineer
Application Developer - SLDS Data Engineer

Mavensoft Technologies, LLC. • Denver (CO)

On-site
USD 83,000 - 117,000
Sr. Data Engineer – Snowflake, ETL, ADO
Sr. Data Engineer – Snowflake, ETL, ADO

Mavensoft Technologies, LLC. • Denver (CO)

On-site
USD 120,000 - 150,000
Application Developer
Application Developer

Apollo Technology Solutions LLC • Denver (CO)

Remote
USD 83,000 - 165,000
Senior Data Engineer
Senior Data Engineer

Quantum Technologies LLC • Boston (MA)

On-site
USD 86,000 - 123,000
Senior Data Engineer
Senior Data Engineer

Robotic Process Automation LLC • Boston (MA)

On-site
USD 96,000 - 124,000
Senior Data Engineer
Senior Data Engineer

Robotics Technologies LLC • Boston (MA)

On-site
USD 90,000 - 120,000
Senior Data Engineer
Senior Data Engineer

CloudBigD • Boston (MA)

On-site
USD 96,000 - 124,000
Data Engineer (Req. ID 14172610)
Data Engineer (Req. ID 14172610)

ZealHire Inc. • Denver (CO)

Remote
USD 90,000 - 135,000
SLDS Data Engineer
SLDS Data Engineer

Ampcus Inc • Denver (CO)

On-site
USD 120,000 - 170,000
Health, dental, vision insurance
Group term life insurance
401(k) retirement savings plan
Data Engineer & Insights Analyst
Data Engineer & Insights Analyst

VIVA • United States

Remote
USD 120,000 - 160,000