Senior Data Integration Engineer

WEX Inc.; São Paulo; Teletrabalho; BRL 120.000 - 160.000; Tempo integral

Resumo da oferta

A leading financial services provider is looking for a Senior Data Integration Engineer in São Paulo, Brazil. You will design and implement scalable data pipelines, collaborate with teams on modernization initiatives, and support application deployments. Experience in cloud ecosystems and data engineering is essential for this role, along with proficiency in ETL processes and programming languages.

Qualificações

• Experience in data ingestion and transformation pipelines.
• Ability to collaborate with data analysts and stakeholders.
• Strong knowledge of data quality, consistency, and reliability.

Responsabilidades

• Support applications during and outside business hours.
• Resolve issues and support production deployments.
• Design application architecture and perform code reviews.

Conhecimentos

Design and implement robust data pipelines

Develop and maintain data flows

Optimize legacy data integration processes

Strong knowledge of ETL/ELT patterns

Proficiency in Python and PySpark

Ferramentas

Apache Kafka

Microsoft Azure

AWS

Databricks

Snowflake

Descrição da oferta de emprego

Senior Data Integration Engineer page is loaded

Senior Data Integration Engineer

Apply locations Brazil Sao Paulo - Remote Brazil Porto Alegre - Remote Brazil Salvador - Remote time type Full time posted on Posted 3 Days Ago job requisition id R18398 Job Description

About the Team/Role

Work as part of a team dedicated to delivering customizable reporting and processing solutions for external stakeholders.
Also, this role involves maintaining and supporting legacy code while contributing to the design and implementation of data platform modernization initiatives.

How you’ll make an impact

Team Collaboration

• Support applications both during and outside business hours.

• Resolve issues and Service Requests quickly and efficiently.

• Support production deployments, which occur on evenings and weekends. Contribute to post-deployment application validation. Quickly troubleshoot, evaluate, and resolve any issues that arise.

• Participate in and lead team conversations, meetings and feature discussions.

Software Design and Architecture

• Design application architecture. Others use your designs as examples to use from.

• Develop solutions that can be performant, scalable, consistent, and supportable that can be accomplished within the available scope, time and budget.

• Perform design and code reviews within the team and across other teams to ensure implementations adhere to goals, industry standards and best practices.

• Deliver software that meets requirements by applying knowledge of the product area to provide the best long-term solutions.

• Ensure code is testable at all levels (unit, integration, and end-to-end).

• Increase unit test coverage.

• Review requirements to provide meaningful, timely and critical feedbackto improve them.

• Ensure requirements arefeasible, implementation is efficient, and the design is aligned with the technical direction of WEX.

• Estimate and prioritize project requests.

• Assist with internal partner initiatives and projects.

• Work with Architects, Software Engineers, Quality Assurance Engineers and Business Analysts to collaborate on design and delivery decisions .

Project Implementation

• Identify and document performance improvements and technical debt.

• Reduce the risk of security threats to WEX Health through a strong understanding of security best practices and close collaboration with Security Architects.

• Identify new threats and vulnerabilities and work with Architects to evolve mitigation strategies.

• Efficiently and effectively implement solutions in all environments, resultingin an increased level of throughput. Others use your code as an example to learn from.

Product Proficiency

• Demonstrate expert level understanding of your team’s product domain and understands multiple product domains outside of your current team.

• Act as an internal and external subject matter expert on WEX systems.

• Able to describe application behavior and technology outside of the Development department in both a technical and business communication style.

Leadership

• Help others complete tasks to accomplish goals.

• Step into other roles to ensure project deadlines are met.

• Mentor and onboard new Software Engineers ensuring they have the proper tools and knowledge to become productive quickly.

• Participate in the interview process and provide valuable hiring recommendations.

• Provide technical and domain guidance to internal and external team members.

• Lead multiple features or components through the full software deployment lifecycle, including integration, customer feedback, release, and support phases.

• Recommend ways to distribute work across your team to increase development efficiency.

• Perform effectively in a cross-functional team environment to ensure successful execution and delivery of project commitments

• Demonstrate expertise in a domain as evidenced by others seeking your peer reviews, advice, and problem solving.

• Coordinate work efforts and communicate priorities across teams to ensure successful execution and delivery of commitments.

• These teams may include Business Analysts, Quality Assurance Engineers, Architects, Product Owners, Support, Management, Configuration Management, Database Administrators, etc.

• Facilitate team meetings as needed, including scrum master ceremonies, if applicable to the team.

Experience you’ll bring:

• Design and implement robust, scalable data ingestion and transformation pipelines using both batch and streaming architectures.
• Develop and maintain data flows across Data Lake, Data Warehouse, and Analytics platforms.
• Collaborate with data analysts, data scientists, and business stakeholders to understand data needs and deliver end-to-end solutions.
• Work with structured and semi-structured data from diverse sources, ensuring data quality, consistency, and reliability.
• Optimize and modernize legacy data integration processes and contribute to data platform migration and modernization initiatives.
• Modernize legacy ETL workflows to cloud-native pipelines
• Cloud & Big Data Ecosystems: Microsoft Azure (Azure Data Factory, Azure Data Lake Gen2, Azure Synapse Analytics, Azure Databricks, Azure Event Hub); AWS (Glue, S3, Redshift)
• Apache Kafka for real-time data streaming
• Databricks for advanced data engineering and analytics
• Snowflake for cloud-native data warehousing
• ThoughtSpot for self-service analytics
• DEHUB for enterprise data hub architecture
• ETL / ELT & Legacy Stack: SQL Server Integration Services (SSIS); SQL Server Reporting Services (SSRS); T-SQL for stored procedures, functions, and performance tuning
• Strong knowledge of ETL/ELT patterns and data modeling best practices
• Other Skills: Proficiency in Python, PySpark; Familiarity with CI/CD practices and version control (e.g., Github; Knowledge of data governance, lineage, and metadata management tools is a bonus