Data Engineer – Real-Time Pipelines & Platforms

Stanford Climate Ventures; Houston (TX); Hybrid; USD 100,000 - 140,000; Full time

Benefits offered by this job

Medical, dental, and vision benefits

Flexible paid time off

Incentive stock options program

401(k) plan with employer match

Job summary

Stanford Climate Ventures is seeking a Data Engineer to join their Data & AI team in Houston, TX. In this role, you will design, build, and operate scalable data pipelines to provide reliable data for analysis. Your work will involve collaboration with various teams to develop data products from ingestion to monitoring.

The ideal candidate holds a Bachelor’s or Master’s degree in a related field, has strong programming skills in Python and SQL, and experience with Apache Spark and Databricks. The position offers some hybrid work flexibility and a robust benefits package.

Qualifications

• 2+ years of hands-on experience building and operating production data pipelines.
• Deep understanding of Apache Spark framework and data processing.
• Experience with cloud data warehousing on Snowflake or equivalent.

Responsibilities

• Design, build, and operate scalable data pipelines on Databricks and Azure.
• Implement automated data quality frameworks with alerting.
• Support production incidents and develop runbooks to reduce toil.

Skills

Apache Spark

Python (PySpark)

SQL

Azure Data Factory

Data Governance

Data Quality Frameworks

Data Lake Storage

Delta Lake

Data Modeling

CI/CD

Education

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field

Tools

Databricks

Snowflake

Power BI

Event Hubs

Kafka

Job description

Stanford Climate Ventures is seeking a Data Engineer to join their Data & AI team in Houston, TX. In this role, you will design, build, and operate scalable data pipelines to provide reliable data for analysis. Your work will involve collaboration with various teams to develop data products from ingestion to monitoring.

The ideal candidate holds a Bachelor’s or Master’s degree in a related field, has strong programming skills in Python and SQL, and experience with Apache Spark and Databricks. The position offers some hybrid work flexibility and a robust benefits package.