Staff Site Reliability Engineer - Hadoop (PRE)

Visa; Warszawa; Hybrid; PLN 180,000 - 240,000; Full time

Job summary

A global payments technology leader in Warsaw is seeking a Staff Site Reliability Engineer to maintain and support its Data Platform. The role involves ensuring the reliability of Big Data systems, collaborating with cross-functional teams, and automating processes using tools like Ansible and Python. Applicants should have a Master’s degree in a related field and at least five years of experience with Hadoop systems. This position offers a hybrid work arrangement.

Qualifications

• Proven experience in debugging issues on Hadoop platforms.
• Ability to automate manual tasks to enhance efficiency.
• Strong documentation and communication skills.

Responsibilities

• Maintain and support Visa’s Data Platform.
• Ensure the reliability and performance of critical Big Data systems.
• Collaborate with L-3 teams on platform enhancements.

Skills

Hadoop management

Collaboration and teamwork

Automation using Python or Ansible

Knowledge of observability tools

Linux understanding

Education

Master’s degree in relevant field

Bachelor’s degree and 5 years experience

Tools

AWS EMR

Ansible

Grafana

Splunk

Job description

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

Hadoop/Big-Data:

• Sound knowledge on managing large scale Hadoop platforms including monitoring the platform, debugging issues, and tuning the performance of the cluster.
• In-depth knowledge of the Hadoop ecosystem, including Zookeeper, HDFS, Yarn, HIVE, SPARK, Trino and Kafka.
• Proven experience in debugging issues on both Hadoop platform and applications.
• Familiarity with security tools such as Kerberos, Ranger, and active directory integrations.
• Experience on Cloud technologies preferably AWS EMR.
• Knowledge on Kubernetes, AI, MLOPS will be advantageous.

Collaboration and Teamwork:

• Collaborate closely with L-3 teams to review new use cases and implement cluster hardening techniques, ensuring the development of robust and reliable platforms.
• Foster cross-team collaboration, building and maintaining strong relationships with customer teams, user communities, architects, and engineering teams.
• Work jointly on key deliverables to ensure production scalability and stability.

Automation: Hands-on Experience with automations using Ansible, Shell, python, or any programming languages. The ability to automate the manual tasks is key in this role.

Observability: knowledge on observability tools like Grafana, opera, Prometheus and Splunk.

Linux: understanding of Linux, networking, CPU, memory, and storage.

Programming Languages: Knowledge of and ability to code or program in one of python, Java or a widely used coding language.

Communication: Excellent interpersonal skills, along with superior verbal and written communication abilities.

This position is not ideal for a Hadoop developer.

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.

Qualifications

Basic Qualifications:

• As a Staff Site Reliability Engineer, you will play a key role in maintaining and supporting Visa’s Data Platform, ensuring the reliability and performance of critical Big Data systems.
• You will drive innovation for our partners and clients globally by working on open-source Big Data clusters, optimizing their availability, efficiency, and scalability.

Education & Experience:

• Master’s degree in Math, Science, Engineering, Computer Science, Information Systems, or a related field.
• Bachelor’s degree in Math, Science, Engineering, Computer Science, Information Systems, or a related field, AND a minimum of five years of relevant experience.
• A minimum of five years of experience working with Hadoop systems.

Preferred Qualifications:

• Experience in Big Data SRE and Engineering across open-source platforms such as Hadoop, Kafka, HBase, and Spark, with strong troubleshooting and debugging skills.
• Proven ability to conduct effective root cause analysis of major production incidents, document findings, and implement high-availability solutions for critical services.
• Expertise in capacity planning, system expansions, and timely upgrades to mitigate scaling challenges, while automating repetitive tasks to reduce manual effort and prevent errors.
• Ability to fine-tune alerting and set up observability tools to proactively identify and resolve performance issues, collaborating with Level-3 teams on use case reviews and cluster hardening.
• Strong documentation skills to create standard operating procedures and platform utilization guidelines, ensuring consistency and efficiency in operations.
• Proficiency in leveraging DevOps tools and industry best practices, including incident, problem, and change management disciplines.
• Commitment to ensuring Hadoop platform performance meets service-level agreements, with experience in security remediation, automation, and self-healing implementations.
• Experience in developing automation tools and reports to streamline processes, using technologies such as Shell scripting, Ansible, Python, or other programming languages.

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.