Sr. Data Engineer

ProArch; EUA; Teletrabalho; USD 140 000 - 190 000; Tempo integral

Resumo da oferta

ProArch is seeking an experienced Senior Data Engineer to architect and implement scalable data pipelines across structured and unstructured data.

You will design end-to-end data solutions, build data lakes and warehouses, and enable enterprise analytics through semantic models and dashboards using Power BI, Tableau, and other tools. Strong collaboration with business stakeholders is essential for translating requirements into scalable architectures.

Qualificações

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 8+ years of experience in data engineering.
• Experience with enterprise data modeling and dimensional modeling.
• Experience with cloud data platforms (AWS, Azure) and data warehousing.
• Strong programming/scripting skills (Python, SQL, PySpark).

Responsabilidades

• Architect and implement scalable data pipelines to process and integrate structured and unstructured data.
• Design end-to-end data solutions including Data Lake, Data Warehouse, and Data Mart.
• Leverage UDP framework to consolidate data pipelines across healthcare domains.
• Support integration of new data domains through standardized ingestion and transformation frameworks.
• Collaborate with stakeholders to translate business requirements into scalable data architectures.
• Develop reusable data assets to support self-service analytics across programs and business domains.
• Design and maintain enterprise dimensional data models including star/snowflake schemas and data marts.
• Design, develop, and maintain Power BI semantic models, datasets, dashboards, and reports.

Conhecimentos

Python

SQL

PySpark

Data modeling

Data governance

Communication

Stakeholder management

MLOps awareness

Formação académica

Bachelor's or Master's in Computer Science, Engineering, or related field

Ferramentas

AWS

Azure Synapse

Databricks

Power BI

Tableau

AWS Glue Data Catalog

Descrição da oferta de emprego

Key Responsibilities/ Accountabilities: Listing of key responsibilities / major activities necessary to fulfil the position's purpose. If possible, please include the percentage of time spent on each key responsibility.

Advanced Data Engineering and Solution Design (80%)

• Architect and implement scalable data pipelines to process and integrate structured and unstructured data.
• Design end-to-end data solutions, including Data Lake, Data Warehouse, and Data Mart, to support analytics and operational systems.
• Leverage UDP framework to consolidate data pipelines across healthcare domains.
• Support the integration of new data domains through standardized ingestion and transformation frameworks.
• Collaborate with stakeholders to translate business requirements into scalable, high-performing data architectures.
• Integrate and optimize data access across distributed systems using data federation and virtualization tools
• Develop reusable data assets to support self-service analytics across programs and business domains.
• Design and maintain enterprise dimensional data models including fact tables, conformed dimensions, star schemas, snowflake schemas, and analytical data marts.
• Translate business and reporting requirements into scalable analytical data structures and semantic data models.
• Develop and maintain semantic layers, curated datasets, and business views to support enterprise reporting and analytics.
• Design, develop, and maintain Power BI semantic models, datasets, dashboards, and reports for internal and external stakeholders.
• Create and optimize DAX measures, calculated columns, KPIs, and business metrics to support operational and strategic reporting.
• Implement Power BI best practices including Row-Level Security (RLS), deployment pipelines, performance optimization, and governance standards.
• Partner with business users and subject matter experts to gather reporting requirements and deliver actionable analytics solutions.
• Ensure consistency of business definitions, metrics, and calculations across enterprise reporting and analytics platforms.

Data Governance and Compliance (10%)

• Develop and enforce data governance standards, ensuring consistency, accuracy, and compliance with regulatory frameworks (e.g, HIPAA).
• Implement data lineage, metadata management, and auditability practices using tools like AWS Glue Data Catalog.
• Establish and manage data stewardship frameworks to improve data quality and trust across the organization.

Performance Optimization and Security (10%)

• Optimize system performance by designing and implementing data partitioning, indexing, and compression strategies.
• Ensure data security through access controls, encryption, and secure design practices.

Core Competencies

• Experience with enterprise data modeling tools (e.g., Erwin, SQL Data Modeler) and strong expertise in dimensional modeling methodologies including Star Schema, Snowflake Schema, Fact and Dimension design, and semantic modeling.
• Bachelor's or master's degree in computer science, Engineering, or related field.
• 8+ years of experience in data engineering, with a strong emphasis on data governance and solution design.
• Expertise in developing scalable data architectures for enterprise reporting
• Familiarity with MLOps and AI data pipelines leveraging cloud-native services such as AWS SageMaker, Glue ML, or Databricks for feature engineering and model deployment.
• Advanced knowledge of data governance tools and frameworks, including AWS Glue Data Catalog, to support enterprise-wide lineage, metadata, and compliance practices.
• Strong understanding of cloud data platforms and services - particularly AWS (Redshift, S3, EMR, Lambda) and hybrid integrations with Azure Synapse or equivalent modern data warehouse technologies.
• Proficiency in programming and scripting languages (Python, SQL, PySpark) for building testing and optimizing scalable data solutions.
• Advanced experience developing Power BI semantic models, datasets, dashboards, reports, DAX measures, Power Query transformations, Row-Level Security, and performance optimization. Experience with Tableau is a plus.

Additional Qualifications:

• Excellent analytical and troubleshooting skills with attention to detail.
• Strong communication skills to effectively articulate technical concepts to non-technical stakeholders.
• Ability to prioritize tasks in a dynamic environment and manage multiple initiatives simultaneously.
• Certifications in cloud, database, and programming are a plus.