Cloud Data Engineer

Business Needs Inc; Quincy (MA); On-site; USD 130,000 - 170,000; Full time

Job summary

Commonwealth Of Massachusetts is seeking a Senior Cloud Data Engineer for the Data Modernization Initiative. You will design and implement cloud data pipelines, build centralized and decentralized architectures on AWS and Snowflake, and partner with epidemiologists, researchers, and policy teams to deliver scalable data solutions.

The role requires strong communication, governance, and security practices, experience with Informatica IICS, Airflow, SQL, and Python, and a proven ability to lead

Qualifications

• 5+ years designing and developing data pipelines and cloud data architectures.
• ,
• ],
• job_responsibilities_description
• ],
• CoT_job_summary_short
• job_summary_short
• contract_type
• location_type
• remote_scope
• perks
• stated_min
• stated_max
• stated_absolute
• salary_estimation_cot
• estimated_low
• estimated_medium
• estimated_high
• currency
• frequency
• bonus
• tax
• application_email
• job_end_date
• contact_person
• hiring_department_name

Responsibilities

• Design and develop cloud-based data management systems.
• Build data pipelines using centralized and decentralized architectures.
• Ensure data security and governance for cloud platforms.
• Collaborate with cross-functional teams and stakeholders.
• Evaluate new cloud products to improve data curation and processing.
• Monitor performance and optimize cloud data solutions.
• Lead design sessions and drive timely delivery of solutions.
• Document data engineering strategies and best practices.

Skills

Data pipeline design

Cloud data architecture

Data warehousing

ETL

Data governance

HIPAA compliance

Communication

Unix/Linux shell scripting

Tools

Informatica IICS

Snowflake

SQL

Python

Airflow

GitHub CI/CD

Tableau/Power BI

Job description

• Company: Commonwealth Of Massachusetts
• Location: Quincy,MA
• 2 times a week.
• Position: Cloud Data Engineer

Job Description: The Massachusetts Department of Public Health (DPH) is responsible for meeting the needs of residents across the Commonwealth by managing programs and services and supporting policies, legislation, and regulations that promote residents' health, safety, and well-being. DPH achieves these goals by ensuring access to high-quality public health and healthcare services and by focusing on prevention, wellness, and health equity for all people. The Department of Public Health (DPH) is looking for a highly skilled and experienced candidate to fill the Senior Cloud Data Engineer position for Data Modernization Initiative (DMI) Projects.

DPH is executing large-scale data modernization across the agency as part of DMI. This includes creating a consolidated Enterprise Data Platform on modern cloud tools (including AWS & Snowflake,) and re-platforming existing business data applications from solutions such as SAS and legacy databases to the new platform. DPH data systems enable epidemiologists, public health researchers, state and local public health officials, and business leaders to analyze public health trends and drive policy decisions.

The ideal candidate is an expert in Design and implementation of Data pipelines in both centralized and decentralized Data architectures. You can clearly articulate and implement different data ingestion patterns depending on use case. In this role, you are comfortable speaking to both business and technical stakeholders to ensure combined understanding and collaborative solutions. This role will be expected to own tasks and follow-up related to cloud data platform Design and development activities and establish well documented best practices and processes related to the Data Modernization efforts.

Duties and Responsibilities:

• Involve in Design and development of cloud-based data management systems.
• Proficiency and practical experience in Design and implementation of data pipelines inboth centralized and decentralized Data architectures.
• Ensures established data security policies and standards are followed while designing data solution for cloud-based platforms.
• Collaborates with cross-functional teams to architect highly scalable, highly available, and highly performant cloud data solutions.
• Evaluates and recommends new cloud products and services for efficient data curation, storage and processing.
• Monitors the performance of cloud data solutions and strategizes optimization measures.
• Ensures compliance with data governance policies and best practices in the cloud.
• Implements data management and data governance processes for cloud data.
• Conducts thorough analysis and documentation of cloud data engineering strategies and designs.
• Establish regular overall system performance assessment processes and action plans.
• Expertise in Data Integration leveraging Informatica (IICS), Snowflake, SQL and Python code to solve complex use cases
• Provide direction for the design and development of the data layers including review of ETL requirements.
• Capable of leading team design sessions and facilitating decisions.
• Ensure timely delivery of solutions meeting requirements and expectations

Qualifications:

Minimum experience/skills:

• 5+ years of experience in Design and development of Data pipelines, Cloud Data Architecture, Data warehousing, Data modeling, ETL, enterprise and business information system architectures, and system engineering
• Thorough knowledge and Implementation experience of data warehouse and Enterprise Data architecture principles, specifically Decentralized data architectures like Data Mesh.
• 3-5 years of experience developing Cloud-based (AWS) data solutions.
• 5+ years of Cloud database platform experience such as Snowflake.
• Experience in creating and maintaining end-to-end data pipelines.
• Strong communication skills comfortable explaining/proposing solutions to technical stakeholders, vendors, and business consumers.
• Proficiency with Unix, Linux, and shell scripting
• Experience working with a large, multi-terabyte enterprise data platform in Cloud.
• Clear understanding on system administration, database administration, storage architectures, backup & recovery, and networking.
• 3-5 years of experience in Informatica (IICS) ETL, GitHub(CI/CD) and Airflow (Orchestration) experience.
• Experience working with highly sensitive health care information and familiar with HIPAA and other data privacy controls.

Preferred experience/skills:

• Amazon Web Services (AWS) Certification or Cloud Data Engineer Certification
• Informatica (IICS) and Airflow (Orchestration) experience
• Snowflake certification
• Tableau and/or Power BI reporting experience