Senior Data Engineer: Spark, Snowflake & Cortex AI

Ethereum Technologies; Atlanta (GA); Vor Ort; USD 85.000 - 116.000; Vollzeit

Zusammenfassung

Ethereum Technologies is seeking a Senior Data Engineer in Atlanta for an 8‑month engagement. You will design production-grade PySpark pipelines, manage Snowflake environments, and enable Cortex AI capabilities to deliver scalable data products.

Join a team that works with Snowflake, AWS, and modern data lakehouse patterns to optimize performance, cost, and reliability while collaborating with architects, analysts, TPMs and business stakeholders.

Qualifikationen

• 6+ years of enterprise data engineering experience with PySpark, Spark SQL, Python and advanced SQL.
• Production experience reading large S3 Parquet/Delta datasets and optimizing PySpark processing.
• Deep Snowflake experience: data modeling, SnowSQL, Snowpipe, Streams, Tasks, RBAC, and Iceberg Tables.
• Exposure to Snowflake Cortex AI, semantic analytics, and Snowpark Python; understanding GenAI/LLMs/RAG.
• Hands-on AWS services: S3, Glue, EMR, Lambda, IAM, Step Functions, CloudWatch, Redshift; orchestration tools like Airflow.
• Knowledge of data warehousing, ETL/ELT, data lakes and lakehouse patterns; strong communication and collaboration skills.

Aufgaben

• Design production-grade PySpark batch/streaming pipelines reading S3 Parquet/Delta data.
• Tune Spark performance, partitions, caching, joins, and cost optimization.
• Engineer Snowflake warehouses, schemas, SnowSQL, Snowpipe, Streams, Tasks, and Materialized Views.
• Create and manage Iceberg tables; implement incremental ingestion and CDC patterns.
• Optimize Snowflake with profiling, clustering, and micro-partition pruning.
• Develop Snowpark Python DataFrame transformations, UDFs, and stored procedures.
• Integrate Cortex Analyst/Search/AI Functions and Cortex AI workflows.
• Build AWS-native solutions using S3, Glue, EMR, Lambda, and Redshift; ensure security and monitoring.
• Modernize ETL tools; ensure data quality, lineage, and runbook documentation.
• Collaborate with architects, analysts, TPMs and business teams to deliver secure, cost-effective data products.

Kenntnisse

PySpark

Python

Spark SQL

SQL

ETL/ELT

Data warehousing

Tools

Snowflake

SnowSQL

Snowpipe

Iceberg Tables

RBAC

AWS

S3

Glue

EMR

Airflow

Jobbeschreibung

Ethereum Technologies is seeking a Senior Data Engineer in Atlanta for an 8‑month engagement. You will design production-grade PySpark pipelines, manage Snowflake environments, and enable Cortex AI capabilities to deliver scalable data products.

Join a team that works with Snowflake, AWS, and modern data lakehouse patterns to optimize performance, cost, and reliability while collaborating with architects, analysts, TPMs and business stakeholders.