Senior Data Engineer

Robotic Process Automation LLC

Boston (MA)

On-site

USD 96,000 - 124,000

Full time

15 hours ago
Be an early applicant

Get a reply from this recruiter

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Robotic Technologies Automation in Boston, MA is seeking an experienced Senior Data Engineer to design, develop, and maintain scalable ETL/ELT pipelines using Snowflake, Informatica, Terraform, and Azure DevOps. You will integrate data across state agencies and partners, ensure data quality, and support longitudinal analytics for education and workforce data.

You will start with knowledge-transfer, shadowing, and gradually own production pipelines, performance tuning, and analytics support,

Qualifications

  • Minimum 5 years of Snowflake ETL/ELT development experience.
  • Experience with Informatica Cloud Data Governance and Catalog (CDGC).
  • Infrastructure as Code with Terraform.
  • CI/CD pipelines using Azure DevOps and Git workflows.
  • Strong SQL development and data transformation skills.

Responsibilities

  • Design, develop, test, and deploy automated ETL/ELT pipelines using Snowflake and Informatica.
  • Integrate data from multiple state agencies and partner systems.
  • Develop complex SQL queries and data processing workflows.
  • Maintain source-to-target mappings and data ingestion processes.
  • Support data quality validation, data lineage, and governance standards.

Skills

Snowflake
Informatica
Terraform
Azure DevOps
SQL
Git
Data ingestion
Data transformation

Tools

Informatica Cloud Data Governance
Azure DevOps
Tableau

Job description

Job Title: Senior Data Engineer
Location: Boston, MA
Duration: 8 Months + Extension
Bill Rate: $76/hour (C2C)
Job Type: W-2, 1099, C2C
Client: To Be Discussed Later
Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

We are seeking an experienced Senior Data Engineer with strong expertise in Snowflake, Informatica, Terraform, and Azure DevOps to support the Statewide Longitudinal Data System (SLDS / COdata) initiative for the State of Colorado.

The Data Engineer will be responsible for designing, developing, implementing, and maintaining scalable ETL/ELT data pipelines that integrate data from multiple state agencies, educational institutions, and third-party partner systems into a centralized Snowflake data platform.

This position will work closely with the Data Architect, Data Scientists, Operations Engineers, and a third-party implementation partner to build and maintain reliable data ingestion processes, implement data transformations, perform data quality validation, and support longitudinal student and workforce data analytics.

The selected candidate will initially participate in a structured knowledge-transfer process, including shadowing, co-running, and independently managing existing data pipelines. Over time, the engineer will assume full responsibility for production operations, pipeline maintenance, performance optimization, and analytics-layer support.

Key Responsibilities

Design, develop, test, and deploy automated ETL/ELT pipelines using Snowflake and Informatica.

Integrate data from state agencies, educational institutions, and third-party source systems.

Develop complex SQL queries, transformations, stored procedures, and data processing workflows.

Build and maintain source-to-target mappings and data integration processes.

Ingest and transform structured data into standardized longitudinal data models.

Develop Informatica workflows to automate data ingestion, cleansing, and transformation.

2. Snowflake Development & Optimization

Develop and maintain Snowflake tables, views, schemas, and transformation objects.

Optimize Snowflake queries, warehouse utilization, and pipeline performance.

Monitor data processing workloads and resolve performance bottlenecks.

Implement scalable and cost-efficiency data processing solutions.

Maintain RAW, transformation, and analytics data layers.

3. Data Quality, Governance & Identity Matching

Implement automated data quality checks covering:Structural and format validation.Data completeness and duplicate detection.

  • Referential integrity and business logic validation.

Perform data cleansing, reconciliation, and exception handling.

Implement record linkage and identity-matching processes across multiple agency datasets.

Maintain data lineage, metadata, audit trails, and data governance standards.

Ensure compliance with FERPA, COPPA, ADA, and applicable state data privacy regulations.

Implement data suppression and de-identification requirements.

4. Production Support & Pipeline Operations

Monitor scheduled data pipelines and ensure successful execution of recurring feed cycles.

Troubleshoot failed jobs, pipeline errors, and data ingestion issues.

Manage late, corrected, or malformed source files and perform data reloads.

Reconcile source and target record counts and investigate discrepancies.

Maintain operational runbooks, incident logs, and pipeline documentation.

Transition existing production pipeline responsibilities from the third-party implementation partner.

5. DevOps, Automation & Release Management

Develop and maintain CI/CD pipelines using Azure DevOps.

Utilize Git for source control, branching, pull requests, and code reviews.

Implement Infrastructure as Code (IaC) using Terraform.

Manage code promotion across Development, Test, UAT, and Production environments.

Develop and execute manual and automated test cases using Azure Test Plans.

Participate in peer reviews, defect resolution, and release documentation.

6. Collaboration & Knowledge Transfer

Collaborate with Data Architects to implement data models and architectural standards.

Work with Data Scientists and Visualization Designers to prepare datasets for Tableau dashboards and analytics.

Coordinate with Operations Engineers on Snowflake warehouse management, permissions, and deployment processes.

Participate in knowledge-transfer sessions with third-party implementation teams.

Gradually assume independent ownership of data pipelines and production operations.

Required Technical Qualifications

Snowflake: Minimum 5 years of experience developing, managing, and optimizing ETL/ELT pipelines and data transformations within Snowflake.

Informatica CDGC: Minimum 1 year of hands-on experience with Informatica Cloud Data Governance and Catalog (CDGC), including metadata management, data profiling, cataloging, and data quality validation.

Terraform: Minimum 1 year of experience with Infrastructure as Code (IaC), including provisioning and managing cloud infrastructure.

Azure DevOps: Strong experience developing and maintaining automated CI/CD pipelines for build, testing, and deployment.

Git: Strong proficiency with version control, branching strategies, pull requests, merge management, and code reviews.

Strong SQL development and data transformation skills.

Experience designing and implementing complex data ingestion and integration pipelines.

Experience with data quality, data validation, cleansing, and reconciliation processes.

Ability to troubleshoot production data pipelines and manage operational support activities.

Preferred Qualifications

Experience working with state government, public-sector, or education data systems.

Familiarity with Statewide Longitudinal Data Systems (SLDS) or longitudinal student and workforce data.

Experience integrating data from multiple government agencies and external partner systems.

Experience with record linkage, entity resolution, and deterministic or probabilistic identity matching.

Knowledge of FERPA, COPPA, student data privacy, and data suppression standards.

Experience with Informatica ETL workflows and metadata-driven data integration.

Experience with Azure Test Plans and structured software testing processes.

Familiarity with Tableau data integration and analytics-layer development.

Experience working with education and workforce datasets from organizations such as CDE, CDHE, CDLE, CDEC, and National Student Clearinghouse.

Project-Specific Responsibilities

Develop and maintain data ingestion pipelines for Colorado Department of Higher Education (CDHE) course-level data.

Support conditional ingestion of postsecondary demographic, Pell Grant eligibility, and disability-related data.

Integrate PSAT/SAT assessment data and perform student identity matching.

Support existing data feeds, including NSC, K-3, CDEC, and Phase 1 integrations.

Participate in knowledge transfer and operational transition from January through April 2027.

Support analytics-layer stabilization, data reliability improvements, and performance tuning.

Develop de-identified and privacy-compliant data extracts for participating higher education institutions.

Production-ready Informatica ETL workflows and Snowflake transformation pipelines.

Automated data cleansing and validation modules.

Identity matching and record-linkage routines.

Snowflake analytics-layer tables, views, and data models.

Data mapping, lineage, and dependency documentation.

Operational runbooks and incident management procedures.

Azure Test Plans, test execution records, and defect logs.

Release change logs and production feed reconciliation reports.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer
Senior Data Engineer

Robotics Technologies LLC • Boston (MA)

On-site
USD 90,000 - 120,000
Senior Data Engineer
Senior Data Engineer

Quantum Technologies LLC • Boston (MA)

On-site
USD 86,000 - 123,000
Senior Data Engineer
Senior Data Engineer

CloudBigD • Boston (MA)

On-site
USD 96,000 - 124,000
Application Developer - SLDS Data Engineer
Application Developer - SLDS Data Engineer

Mavensoft Technologies, LLC. • Denver (CO)

On-site
USD 83,000 - 117,000
Application Developer
Application Developer

Apollo Technology Solutions LLC • Denver (CO)

Remote
USD 83,000 - 165,000
Sr. Data Engineer – Snowflake, ETL, ADO
Sr. Data Engineer – Snowflake, ETL, ADO

Mavensoft Technologies, LLC. • Denver (CO)

On-site
USD 120,000 - 150,000
Data Engineer
Data Engineer

VIVA • Denver (CO)

Remote
USD 120,000 - 160,000
Data Engineer (Req. ID 14172610)
Data Engineer (Req. ID 14172610)

ZealHire Inc. • Denver (CO)

Remote
USD 90,000 - 135,000
Data Engineer
Data Engineer

Talent Groups • Colorado

Remote
USD 110,000 - 150,000
Data Engineer
Data Engineer

Ethereum Technologies LLC • Houston (TX)

On-site
USD 120,000 - 180,000