Sr Big Data Engineer Airflow and Oozie (GCP) 1157 views

Open Data Science Conference India (ODSC); Town of Texas (WI); Remote; USD 116,000 - 199,000; Full time

Job summary

A leading company in data science is looking for a skilled Senior Big Data Engineer. This remote role requires expertise in Oozie and Airflow within the Apache Hadoop ecosystem, along with a significant understanding of GCP. The ideal candidate will develop robust batch processing systems and implement best practices in automation, playing a key role in transforming IT solutions. Join this dynamic team to drive innovation and impactful projects.

Qualifications

• 5+ years experience in software/technology or consulting.
• Experience with cloud migrations or IT transformations.
• Expertise in public cloud services, particularly GCP.

Responsibilities

• Develop scalable code for batch processing systems.
• Optimize data workflows using Oozie and Airflow.
• Implement automation best practices for CI/CD.

Skills

Oozie

Airflow

Java

Python

Map Reduce

Apache Hadoop

DevOps

Education

Bachelor’s degree in Computer Science

Tools

Terraform

Job description

Sr Big Data Engineer Airflow and Oozie (GCP) 1165 views

Join to apply for the Sr Big Data Engineer Airflow and Oozie (GCP) 1165 views role at Open Data Science Conference India (ODSC)

Sr Big Data Engineer Airflow and Oozie (GCP) 1165 views

2 weeks ago Be among the first 25 applicants

Join to apply for the Sr Big Data Engineer Airflow and Oozie (GCP) 1165 views role at Open Data Science Conference India (ODSC)

Get AI-powered advice on this job and more exclusive features.

About the Role:

We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in Oozie, the Apache Hadoop ecosystem, Airflow, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively.

Job Description

About the Role:

We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team. The ideal candidate will have a strong background in developing batch processing systems, with extensive experience in Oozie, the Apache Hadoop ecosystem, Airflow, and a solid understanding of public cloud technologies, especially GCP. This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively.

Work Location: US-Remote

Key Responsibilities

• Develop scalable and robust code for batch processing systems. This includes working with technologies like Hadoop, Oozie, Pig, Hive, Map Reduce, Spark (Java), Python, Hbase.
• Develop, Manage and optimize data workflows using Oozie and Airflow within the Apache Hadoop ecosystem.
• Leverage GCP for scalable big data processing and storage solutions.
• Implementing automation/DevOps best practices for CI/CD, IaC, etc.
  

Qualifications

• Bachelors’s degree in Computer Science, software engineering or related field of study.
• Experience with managed cloud services and understanding of cloud-based batch processing systems are critical.
• Proficiency in Oozie, Airflow, Map Reduce, Java.
• Strong programming skills with Java (specifically Spark), Python, Pig, and SQL.
• Expertise in public cloud services, particularly in GCP.
• Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, Map Reduce.
• Familiarity with BigTable and Redis.
• Experienced in Infrastructure and Applied DevOps principles in daily work. Utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes.
• Proven experience in engineering batch processing systems at scale.
  

Must Have

• 5+ years of experience in customer-facing software/technology or consulting.
• 5+ years of experience with “on-premises to cloud” migrations or IT transformations.
• 5+ years of experience building, and operating solutions built on GCP (ideally) or AWS/Azure.
• Proficiency in Oozie, Airflow, Map Reduce, Java.
  

The following information is required by pay transparency legislation in the following states: CA, CO, HI, NY, and WA. This information applies only to individuals working in these states.

• The anticipated starting pay range for Colorado is: $116,100 – $170,280.
• The anticipated starting pay range for the states of Hawaii and New York (not including NYC) is: $123,600 – $181,280.
• The anticipated starting pay range for California, New York City and Washington is: $135,300 – $198,440.
  
  

Unless already included in the posted pay range and based on eligibility, the role may include variable compensation in the form of bonus, commissions, or other discretionary payments. These discretionary payments are based on company and/or individual performance and may change at any time. Actual compensation is influenced by a wide array of factors including but not limited to skill set, level of experience, licenses and certifications, and specific work location. Information on benefits offered is here.

About Rackspace Technology

We are the multicloud solutions experts. We combine our expertise with the world’s leading technologies — across applications, data and security — to deliver end-to-end solutions. We have a proven record of advising customers based on their business challenges, designing solutions that scale, building and managing those solutions, and optimizing returns into the future. Named a best place to work, year after year according to Fortune, Forbes and Glassdoor, we attract and develop world-class talent. Join us on our mission to embrace technology, empower customers and deliver the future.

More on Rackspace Technology

Though we’re all different, Rackers thrive through our connection to a central goal: to be a valued member of a winning team on an inspiring mission. We bring our whole selves to work every day. And we embrace the notion that unique perspectives fuel innovation and enable us to best serve our customers and communities around the globe. We welcome you to apply today and want you to know that we are committed to offering equal employment opportunity without regard to age, color, disability, gender reassignment or identity or expression, genetic information, marital or civil partner status, pregnancy or maternity status, military or veteran status, nationality, ethnic or national origin, race, religion or belief, sexual orientation, or any legally protected characteristic. If you have a disability or special need that requires accommodation, please let us know.

More Information

• Salary Offer 0 ~ $3000
• Experience Level Senior
• Total Years Experience 0-5
• Dropdown field Option 1
  
  

Apply for this job

Related Jobs

Email Me Jobs Like These

New Job Alert

Never miss a chance!

Let us know your job expectations, so we can find you jobs better!

Email

Keywords

Email Frequency Daily Weekly Fortnightly Monthly

• Daily
• Weekly
• Fortnightly
• Monthly
  
  

Showing 1–5 of 1018 jobs

Principal MLOPs Engineer

Full TimeUnited StatesUnited StatesUnited StatesUnited States

Rackspace Technology Rackspace Technology Full Time Full Time United States

Send to friend Share

View more 5 months ago

Data Engineer (Remote)

RemoteSalt Lake CitySalt Lake CitySalt Lake CitySalt Lake CityUnited StatesUnited StatesUnited StatesUnited States

Sera Prognostics Sera Prognostics Remote Remote Salt Lake City, United States

Send to friend Share

View more 5 months ago

Data Scientist, Intermediate

Full TimeArizonaArizonaArizonaArizonaUnited StatesUnited StatesUnited StatesUnited States

Northern Arizona University Northern Arizona University Full Time Full Time Arizona, United States

Send to friend Share

View more 5 months ago

Senior Data Scientist

Full TimeNew YorkNew YorkNew YorkNew YorkUnited StatesUnited StatesUnited StatesUnited States

Ai+ Career Services Ai+ Career Services Full Time Full Time New York, United States

Send to friend Share

View more 5 months ago

Senior Data Scientist

Full TimeUnited StatesUnited StatesUnited StatesUnited States

Murmuration Murmuration Full Time Full Time United States

Send to friend Share

View more 5 months ago

Share this job

Rackspace Technology

Company Information

• Total Jobs 4 Jobs
• Category Information Technology & Services, Management Consulting
• Location Texas
  
  

Connect with us

Contact Us

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Engineering and Information Technology

• Industries

  Research

Referrals increase your chances of interviewing at Open Data Science Conference India (ODSC) by 2x

Get notified about new Big Data Developer jobs in Texas, United States.

Texas, United States $139,500.00-$271,566.94 1 week ago

Texas, United States $139,500.00-$271,566.94 1 week ago

Irving, TX $80,000.00-$100,000.00 1 week ago

Freelance Software Developer (Python) - AI Trainer

Dallas, TX $185,000.00-$200,000.00 8 months ago

Freelance Software Developer (Python) - AI Trainer

Freelance Software Developer (Python) - AI Trainer

AI Engineer - LLMOps/AIOps/MLOps - Software and Data Engineering

Austin, Texas Metropolitan Area $100,000.00-$220,000.00 3 days ago

Texas, United States $62,400.00-$112,400.00 4 days ago

Freelance GenAI Developer - Prompt Engineering & Data Workflows

Full-Stack Developer & Data Visualization Engineer

Freelance Software Developer (Python) - AI Trainer

Austin, Texas Metropolitan Area 5 days ago

Freelance Software Developer (Python-Rust) - AI Trainer

Freelance Software Developer (Python) - AI Trainer

Engineer 2 -Data and Analytics ( MicroStrategy)

Freelance Software Developer (Python-Rust) - AI Trainer

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.