Senior Data Engineer: Cloud Lakehouse & AI Analytics

Robotic Technologies Automation; Atlanta (GA); On-site; USD 83,000 - 124,000; Full time

Job summary

Robotic Technologies Automation in Atlanta, GA seeks a Senior Data Engineer to design production-grade PySpark pipelines, read S3 Parquet/Delta, and implement complex transformations for high-volume data. You will optimize Spark performance and manage Snowflake warehouses, schemas, and dynamic tables to support lakehouse workloads.

The role requires deep Snowflake expertise, Cortex AI features, and AWS data services; you will collaborate with architects, TPMs and business teams to deliver

Qualifications

• 6+ years of enterprise data engineering experience with PySpark, Spark SQL, Python and advanced SQL.
• Experience reading large S3 Parquet/Delta datasets and optimizing PySpark processing.
• Deep Snowflake experience: data modeling, SnowSQL, Snowpipe, Streams, Tasks, Iceberg Tables, Materialized Views.
• Familiarity with Snowflake Cortex AI features, semantic analytics, and Snowpark Python.
• Hands-on AWS data services: S3, Glue, EMR, Lambda, IAM, Step Functions, Redshift; orchestration tools like Airflow.

Responsibilities

• Design production-grade PySpark batch/streaming pipelines reading S3 Parquet/Delta datasets.
• Tune Spark performance with partitioning, caching, joins, and optimization techniques.
• Engineer Snowflake warehouses, schemas and models; implement SnowSQL, Snowpipe, and Materialized Views.
• Create and manage Iceberg tables; enable incremental ingestion and CDC patterns.
• Optimize Snowflake with profiling, clustering and micro-partition pruning.
• Build Snowpark Python transformations, UDFs and stored procedures; explore Cortex AI and Cortex Agents.
• Integrate AWS-native solutions (S3, Glue, EMR, Lambda, Redshift) and monitoring workflows.
• Modernize DataStage/Informatica ETL; ensure data quality, lineage, security and runbooks.

Skills

PySpark

Spark SQL

Python

Advanced SQL

Data engineering

ETL/ELT

Tools

Snowflake

SnowSQL

Snowpipe

Iceberg

AWS

Airflow

Job description

Robotic Technologies Automation in Atlanta, GA seeks a Senior Data Engineer to design production-grade PySpark pipelines, read S3 Parquet/Delta, and implement complex transformations for high-volume data. You will optimize Spark performance and manage Snowflake warehouses, schemas, and dynamic tables to support lakehouse workloads.

The role requires deep Snowflake expertise, Cortex AI features, and AWS data services; you will collaborate with architects, TPMs and business teams to deliver