Senior Data Engineer — AI Evaluation Pipelines

LMArena; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive compensation

Comprehensive health and wellness benefits

Opportunity to work on cutting-edge AI

Culture that values transparency and community impact

Job summary

A cutting-edge AI company in San Francisco is seeking a Senior Software Engineer focused on data engineering. In this role, you will design and build robust data pipelines to process user vote data, collaborating with researchers and product leadership. The ideal candidate has 5+ years of software engineering experience, proficiency in SQL and Python, and hands-on experience with data processing frameworks. Attractive compensation, comprehensive benefits, and a commitment to diversity are part of this opportunity.

Qualifications

• 5+ years of experience in software engineering focused on data engineering and big data technologies.
• Proficiency in SQL and at least one programming language for data analysis (Python preferred, Scala, R).
• Hands-on experience with data processing frameworks (Apache Spark, Ray Data) and big data platforms (Databricks, Snowflake).
• Experience in designing and optimizing production data pipelines.

Responsibilities

• Design and build data pipelines to process and transform user vote data.
• Collaborate with researchers to understand product goals and necessary data.
• Implement solutions to generate dashboards and reports.
• Ensure data quality and reliability of pipelines.
• Scale data infrastructure to accommodate increasing data volumes.

Job description

A cutting-edge AI company in San Francisco is seeking a Senior Software Engineer focused on data engineering. In this role, you will design and build robust data pipelines to process user vote data, collaborating with researchers and product leadership. The ideal candidate has 5+ years of software engineering experience, proficiency in SQL and Python, and hands-on experience with data processing frameworks. Attractive compensation, comprehensive benefits, and a commitment to diversity are part of this opportunity.