Data Warehouse Engineer

Staffing Spot, Inc.; New York (NY); On-site; USD 90,000 - 130,000; Full time

Job summary

Staffing Spot, Inc. is seeking a Data Warehouse Support specialist in New York to monitor and re-run failed data pipelines and workflows on SSIS and T-SQL. You will troubleshoot production issues, ensure data accuracy across pipelines, and document resolutions and runbooks.

Collaboration with onshore and offshore teams will be key to stabilizing cloud data operations. The role involves gathering business requirements for cloud datasets, designing and testing pipelines, and optimizing batch and

Qualifications

• Monitor and re-run failed data pipelines and workflows.
• Troubleshoot production issues focusing on bug fixes.
• Maintain data accuracy and integrity across all pipelines.
• Gather and analyze business requirements for cloud datasets.
• Design, develop, and test data pipelines for Cloud Data Warehouse initiatives.

Responsibilities

• Monitor and re-run failed data pipelines and workflows.
• Troubleshoot production issues, focusing on bug fixes.
• Maintain data accuracy and integrity across all pipelines.
• Collaborate with onshore/offshore teams to address production incidents.
• Design, develop, and test data pipelines for Cloud Data Warehouse initiatives.

Skills

Design

Operations

Data warehouse

Cloud

Data

Root cause analysis

Job description

Data Warehouse Support

• Monitor and re-run failed data pipelines and workflows to ensure seamless operations on SSIS & T-SQL.
• Troubleshoot and resolve production issues, focusing exclusively on bug fixes.
• Maintain data accuracy and integrity across all pipelines.
• Conduct root cause analysis for recurring failures and document resolutions.
• Provide daily status reports on pipeline execution, highlighting failures, root causes, and applied fixes.
• Collaborate with onshore teams to address critical production incidents.
• Implement temporary workarounds to minimize downtime while working toward permanent solutions.
• Gather and analyze business requirements for business datasets in the cloud.
• Design, develop, and test data pipelines to support Cloud Data Warehouse initiatives.
• Build and optimize batch processing and streaming jobs to ensure timely data refresh.
• Implement Medallion Architecture and design models for structured data processing across different layers.
• Collaborate with onshore teams to establish best practices and reduce technical debt.
• Define data models and ETL strategies in alignment with stakeholder requirements.
• Participate in regular stand-ups and discussions with onshore and offshore teams to ensure seamless collaboration.
• Ensure compliance with data governance policies, security standards, and best practices.
• Develop and maintain comprehensive runbooks to document cloud operations, ensuring clear, step-by-step guidance for troubleshooting and maintenance.
• Standardize and optimize cloud workflows by creating detailed documentation that enhances efficiency, reduces downtime, and supports seamless knowledge transfer across teams.

• Monitor and re-run failed data pipelines and workflows to ensure seamless operations on SSIS & T-SQL.
• Troubleshoot and resolve production issues, focusing exclusively on bug fixes.
• Maintain data accuracy and integrity across all pipelines.
• Conduct root cause analysis for recurring failures and document resolutions.
• Provide daily status reports on pipeline execution, highlighting failures, root causes, and applied fixes.
• Collaborate with onshore teams to address critical production incidents.
• Implement temporary workarounds to minimize downtime while working toward permanent solutions.
• Gather and analyze business requirements for business datasets in the cloud.
• Design, develop, and test data pipelines to support Cloud Data Warehouse initiatives.
• Build and optimize batch processing and streaming jobs to ensure timely data refresh.
• Implement Medallion Architecture and design models for structured data processing across different layers.
• Collaborate with onshore teams to establish best practices and reduce technical debt.
• Define data models and ETL strategies in alignment with stakeholder requirements.
• Participate in regular stand-ups and discussions with onshore and offshore teams to ensure seamless collaboration.
• Ensure compliance with data governance policies, security standards, and best practices.
• Develop and maintain comprehensive runbooks to document cloud operations, ensuring clear, step-by-step guidance for troubleshooting and maintenance.
• Standardize and optimize cloud workflows by creating detailed documentation that enhances efficiency, reduces downtime, and supports seamless knowledge transfer across teams.

Skills: design,operations,data warehouse,cloud,data,root