Data Integration Engineer

RSM Solutions, Inc; Irvine (CA); On-site; USD 90,000 - 120,000; Full time

Job summary

RSM Solutions, Inc is seeking a Data Integration Specialist to work onsite in Irvine, California. In this role, you will design, build, and operate data pipelines using MS SQL, SSIS, and Spark. You will collaborate with a team of data-centric professionals to ensure data accuracy and readiness for analytics.

The ideal candidate has 4+ years of experience in data integration, particularly in MS SQL and SSIS, with strong practical knowledge of machine learning clusters and Data Vault methodologies. This is an excellent opportunity for those who thrive in team environments and possess great problem-solving skills.

Qualifications

• 4+ years building data integration with MS SQL, SSIS, and Spark.
• At least 2 years of ML Cluster build experience.
• At least 2 years of experience with Data Vault.

Responsibilities

• Design and deploy pipelines using SSIS, Spark, Azure Data Factory.
• Create CI/CD pipelines for versioning ETL code.
• Benchmark and tune Spark and SQL performance.

Skills

Data integration

MS SQL

SSIS

Spark

PowerShell

Communication

Problem-solving

T SQL

Tools

Azure Data Factory

Git

MongoDB

Grafana

Job description

This role is being done onsite in Irvine, California. I prefer working with candidates that are already local to the area. If you need to relocate, that is fine, but there are no relocation dollars available.

I can only work with US Citizens or Green Card Holders for this role. I cannot work with H1, OPT, EAD, F1, H4, or anyone that is not already a US Citizen or Green Card Holder for this role.

For this role, you will be working with a team of about 6 other data centric individuals. That team is a mix of ML Cluster engineers, db engineers and a BA. You won't really be working that much on requirements gathering, as that is something that the BA on the team does. But, if you have worked on requirements gathering, documentation, process flow diagramming and so on, that would be great to see, as partnering with the BA would be a great thing to see in the candidate chosen for this role.

You will design, build and operate batch & streaming pipelines that move data from SQL Server, MongoDB, legacy files, and third-party APIs into this client's Data Vault warehouse and machine-learning (ML) cluster, ensuring that data is accurate, timely, and analytics-ready. This role blends hands-on ETL/ELT development in SSIS, Spark, Runbooks, and Azure Data Factory with data-modeling expertise (hubs, links, satellites) to support scalable reporting, predictive models, and AI agents. Working closely with development team and cross-functional product teams.

Here are some of those key responsibilities:

• Design, develop, and deploy incremental and full load pipelines using SSIS, Spark, Runbooks and Azure Data Factory to ingest data into landing, raw, and curated layers of the Data Vault.
• Build CDC (change data capture) solutions to minimize latency for downstream reporting and ML features.
• Automate schema evolution and metadata population for hubs, links, and satellites.
• Implement validation rules, unit tests, and data quality frameworks to enforce referential integrity and conformance to business rules.
• Maintain a requirements traceability matrix and publish data lineage documentation Metadata Management / SSAS models. This includes partnering with this team's BA to translate user stories into technical interfaces and mapping specs.
• Create CI/CD pipelines (Azure DevOps, Git) to version ETL code, infrastructure as code, and automated tests.
• Develop PowerShell/.NET utilities to orchestrate jobs, manage secrets, and push metrics to Grafana or Azure Monitor.
• Benchmark and tune Spark, SQL, and SSIS performance; recommend index strategies, partitioning, and cluster sizing strategies for cost/performance balance.
• Stay current with emerging integration patterns (e.g., event driven architectures, Delta Lake) and propose pilots for adoption.

Here is what we are seeking in terms of requirements for this role:

• 4+ years building data integration with MS SQL, SSIS, and Spark.
• At least 2 years of ML Cluster build experience.
• At least 2 years of experience with Data Vault.
• Strong T SQL, Python/Scala for Spark, PowerShell/.NET scripting; working knowledge of MongoDB aggregation, SSAS tabular models, and Git CI/CD.
• Excellent problem solving, communication, and stakeholder management abilities.