Senior Data Engineer (Hybrid) - Cloud Data & AI

Creative Solutions Services, LLC; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Creative Solutions Services, LLC is seeking a Senior Data Engineer to design scalable data platforms powering analytics and AI initiatives. You will build reliable, high-performance data ecosystems on AWS using Databricks, Delta Lake, Snowflake, Kafka, and Spark, enabling Generative AI capabilities.

Based in New York, NY, the role is hybrid with in-person interview and onboarding requirements. Collaborate with architects, analysts, and data scientists to deliver enterprise-grade data solutions

Qualifications

• Bachelor's or Master's in CS/Engineering/IS or related field.
• 8+ years in software/data engineering or big data.
• Enterprise-scale data pipelines design/implementation experience.
• Hands-on with Python, PySpark, Spark SQL, SQL, Kafka, Databricks, Delta Lake, Snowflake, Hive.
• Experience building on AWS cloud platforms.
• Understanding of CI/CD and Git workflows.
• Strong analytical and problem-solving abilities.

Responsibilities

• Design, build, and maintain scalable data pipelines for batch and real-time data.
• Implement Lakehouse architectures using Databricks, Delta Lake, Bronze/Silver/Gold.
• Develop processing with PySpark, Spark SQL, Kafka, AWS Kinesis.
• Architect AWS-based data solutions (S3, EMR, EC2, Redshift, Lambda, IAM, SQS).
• Optimize performance and ensure data quality, governance, and security.
• Develop orchestration with Airflow/Databricks Workflows; enable CI/CD automation.
• Assist migrations to cloud-native platforms and support Generative AI initiatives.

Skills

Analytical skills

Troubleshooting

Problem-solving

Stakeholder communication

Education

Bachelor's or Master's in CS/Engineering/IS

Tools

Python

PySpark

Spark SQL

SQL

Kafka

Databricks

Delta Lake

Snowflake

Hive

AWS

Git

CI/CD

Job description

Creative Solutions Services, LLC is seeking a Senior Data Engineer to design scalable data platforms powering analytics and AI initiatives. You will build reliable, high-performance data ecosystems on AWS using Databricks, Delta Lake, Snowflake, Kafka, and Spark, enabling Generative AI capabilities.

Based in New York, NY, the role is hybrid with in-person interview and onboarding requirements. Collaborate with architects, analysts, and data scientists to deliver enterprise-grade data solutions