Data Engineer - Human Behavior and Carbon Neutrality

HireArt; Los Altos (CA); Remote; Full time

Job summary

A leading company is seeking a Data Engineer focused on Human Behavior and Carbon Neutrality. This full-time role involves building and maintaining data pipelines in Databricks, collaborating with cross-functional teams, and ensuring data quality and performance. Ideal candidates will have strong cloud experience and a background in behavioral data analysis.

Benefits

PTO and paid holidays!

Pre-tax commuter benefits

Employer subsidized healthcare benefits

Flexible Spending Account for healthcare-related costs

Coverage for short- and long-term disability and life insurance

401k package

Qualifications

• 4+ years of experience designing, building, and maintaining data pipelines in a cloud environment.
• Experience with human behavior data is a plus.
• Expertise in AI/ML frameworks like TensorFlow or PyTorch is bonus.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines in Databricks.
• Collaborate closely with research scientists and other stakeholders to understand data requirements.
• Monitor and troubleshoot data pipelines, ensuring scalability and reliability.

Skills

SQL

Python

Cloud technologies

Data analysis

Problem-solving

Communication

Education

Bachelor's or Master's degree in Computer Science, Engineering, or related field

Tools

Databricks

AWS

Job description

Data Engineer - Human Behavior and Carbon Neutrality

Join to apply for the Data Engineer - Human Behavior and Carbon Neutrality role at HireArt

Data Engineer - Human Behavior and Carbon Neutrality

Join to apply for the Data Engineer - Human Behavior and Carbon Neutrality role at HireArt

Expected compensation: 81.75 - 93.75 USD Per Hour

HireArt is helping TRI hire a Data Engineer - Human Behavior and Carbon Neutrality who is interested in the overlap between technology and the behavioral sciences to architect and optimize cloud-based infrastructure to support research objectives in the Carbon Neutrality Department of the Human-Centered AI Division.

In this role, you will design, develop, and maintain end-to-end data pipelines in Databricks for processing/analyzing large-scale datasets, as well as build and optimize cloud-based infrastructure to support generative AI methods, ensuring scalability, reliability, and performance.

We're looking for someone with a Bachelor's degree or higher in Computer Science, Engineering, or a related field, and 4+ years of experience designing, building, and maintaining data pipelines in a cloud environment, preferably using Databricks.

As a Data Engineer - Human Behavior and Carbon Neutrality, you will:

• Design, develop, and maintain end-to-end data pipelines in Databricks for processing/analyzing large-scale datasets.
• Build and optimize cloud-based infrastructure to support generative AI methods, ensuring scalability, reliability, and performance.
• Collaborate closely with research scientists, software developers, machine learning engineers, and domain experts to understand data requirements and design effective solutions.
• Implement best practices for data quality, data integrity, and data security throughout the pipeline.
• Incorporate multi-model data (e.g., vision, language, and time series) techniques into data pipelines to enable AI model development.
• Monitor and troubleshoot data pipelines, identifying/resolving performance bottlenecks, and optimizing resource utilization.
• Stay up-to-date with the latest advancements in cloud technologies, generative AI frameworks, and data engineering techniques.
  
  

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field
• 4+ years of experience designing, building, and maintaining data pipelines in a cloud environment, preferably using Databricks
• Strong proficiency in cloud technologies, especially AWS
• A track record of working with human behavior data and the ability to derive actionable insights from such data is a plus
• Expertise in SQL and Python
• Experience building and maintaining live dashboards to monitor user behavior, product stability, and model performance
• Strong problem-solving skills, attention to detail, and the ability to work effectively in a collaborative, cross-functional team environment
• Excellent communication skills to convey complex technical concepts to both technical and non-technical stakeholders
  
  

Bonus Qualifications

• Expertise in AI/ML frameworks like TensorFlow or PyTorch, with hands-on experience in deploying/optimizing deep learning models
• Familiarity with AI model training and fine tuning techniques, including an understanding of the challenges associated with multi-modal data
• Proficiency in additional languages such as R, Scala, or Java
• Experience with containerization and orchestration tools like Docker and Kubernetes
• Experience with NoSQL, Change Data Capture Techniques, and BI Tools
  
  

Benefits

• PTO and paid holidays!
• Pre-tax commuter benefits
• Employer (HireArt) Subsidized healthcare benefits
• Flexible Spending Account for healthcare-related costs
• HireArt covers all costs for short- and long-term disability and life insurance
• 401k package
  
  

Commitment: This is a full-time, 9-month contract position staffed via HireArt. This position will be remote with preference given to candidates based in the Bay Area who are open to working onsite in Los Altos, CA.

HireArt values diversity and is an Equal Opportunity Employer. We are interested in every qualified candidate who is eligible to work in the United States. Unfortunately, we are not able to sponsor visas or employ corp-to-corp.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  Technology, Information and Internet

Referrals increase your chances of interviewing at HireArt by 2x

Get notified about new Data Engineer jobs in Los Altos, CA.

San Jose, CA $107,430.00-$176,490.00 5 days ago

Santa Clara, CA $80,000.00-$155,000.00 1 month ago

San Francisco Bay Area $200,000.00-$250,000.00 1 month ago

San Jose, CA $130,000.00-$160,000.00 4 days ago

Mountain View, CA $100,600.00-$215,400.00 1 week ago

Software Engineer - Windows Internals - Fully Remote

Python and Kubernetes Software Engineer - Data, AI/ML & Analytics

San Francisco Bay Area $200,000.00-$250,000.00 4 hours ago

Software Engineer, Full-Stack Opening, USA

Senior or Staff Software Engineer, Database Engineer

Python and Kubernetes Software Engineer - Data, Workflows, AI/ML & Analytics

Python and Kubernetes Software Engineer - Data, AI/ML & Analytics

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.