Hadoop Data Engineer (Big Data Developer)

Exxon Mobil; Curitiba; Presencial; BRL 80.000 - 150.000; Tempo integral

Resumo da oferta

Join a forward-thinking company as a Data Engineer, where your expertise in big data technologies will drive innovation and efficiency. In this role, you will design and maintain scalable data processing pipelines using Hadoop and Spark, collaborate with diverse teams to implement effective data integration solutions, and ensure data integrity and security. The ideal candidate will thrive in a dynamic environment, leveraging their strong analytical skills and problem-solving abilities to deliver impactful results. This is an exciting opportunity to be part of a global team committed to shaping the future of energy and technology, where your contributions will be valued and recognized.

Serviços

Health benefits

Professional development opportunities

Flexible working hours

Retirement savings plan

Qualificações

• 5+ years of experience in data engineering with strong skills in Hadoop and Spark.
• Proficient in programming languages such as Scala, Java, and Python.

Responsabilidades

• Design and maintain scalable data processing pipelines using Hadoop and Spark.
• Collaborate with teams to implement data integration and ETL processes.

Conhecimentos

Hadoop

Apache Spark

Scala

Java

Python

Data Integration

ETL Processes

Data Modeling

Problem-solving

Analytical Skills

Formação académica

Bachelor's degree in Computer Science

Ferramentas

GitHub

Cloudera Data Platform

Cloud Azure

Snowflake

Apache Kafka

Solr

Oozie

Airflow

Descrição da oferta de emprego

At ExxonMobil, our vision is to lead in energy innovations that advance modern living and a net-zero future. As one of the world’s largest publicly traded energy and chemical companies, we are powered by a unique and diverse workforce fueled by the pride in what we do and what we stand for.

The success of our Upstream, Product Solutions and Low Carbon Solutions businesses is the result of the talent, curiosity and drive of our people. They bring solutions every day to optimize our strategy in energy, chemicals, lubricants and lower-emissions technologies.

We invite you to bring your ideas to ExxonMobil to help create sustainable solutions that improve quality of life and meet society’s evolving needs.

More about us

Come be part of the EMIT Department of ExxonMobil Business Support Center Brasil Ltda. at our Global Business Center (GBC) in Curitiba, with more than 21 years of presence in the country and a workforce of ~1800 employees, that provides support to an extensive portfolio of businesses across the globe. Our offices are located in Brazil, USA, Europe and Asia.

What role you will play in our team

Work with business and technical teams to provide technical support and development expertise for overall data pipeline, Hadoop Spark workflows, big data solutions and implement data models in Big Data & Analytics portfolio.

What you will do

• Design, develop, and maintain scalable data processing pipelines using Hadoop and Spark.
• Implement data integration and ETL processes to ingest and transform large datasets.
• Collaborate with data scientists, analysts, business partners and other stakeholders to understand data requirements and deliver solutions.
• Optimize and tune Hadoop and Spark jobs for performance and efficiency.
• Manage and maintain data storage solutions, ensuring data integrity and security.
• Utilize GitHub for version control and collaboration on code development.
• Work with CDP (Cloudera Data Platform) to manage and deploy data applications.
• Integrate and manage data solutions on Cloud Azure, and Snowflake ensuring seamless data flow and accessibility.
• Monitor and troubleshoot data processing workflows, resolving issues promptly.
• Stay updated with the latest industry trends and technologies in big data and cloud computing.
• Explore and develop skills in newer technologies according to big data technology roadmap.

About you

We are looking for a Data Engineer with +5 years of experience who excels in verbal and written communication, and can work independently as a self-starter. The ideal candidate must demonstrate strong problem-solving and analytical skills and have the ability to develop seamless workflows with business partners and stakeholders. Additionally, they should be able to interface effectively with customers, technology partners, testing, architecture, and analysis groups. Technical requirements include:

Skills and Qualifications

• Minimum bachelor’s degree from a recognized university in Computer Science or equivalent; minimum GPA - 6.5 (out of 10.0)
• 3-5 Years of experience working with Hadoop
• Big Data Technologies: Proven experience with Apache Spark and the Hadoop Ecosystem.
• Programming Languages: Strong coding and programming skills in Scala, Java, and/or Python.
• Applied Experience with Hive, Iceberg and HBase.
• Applied Experience with Solr.
• Experience with Apache Kafka and Spark Streaming or Apache Flink.
• Experience with Oozie and good to have Airflow.
• Familiarity using Git/GitHub.
• Experience in UNIX or UNIX-like systems.
• Experience with SCRUM or another similar Agile framework.

Preferred Skills and Qualifications

• Familiarity with Azure DevOps
• Familiarity with SQL

An ExxonMobil career is one designed to last. Our commitment to you runs deep: our employees grow personally and professionally, with benefits built on our core categories of health, security, finance, and life.

We offer you:

Please note benefits may be changed from time to time without notice, subject to applicable law.

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, sexual orientation, gender identity, national origin, citizenship status, protected veteran status, genetic information, or physical or mental disability.