Senior Data Engineer - ETL, Cloud DW & Spark

Cloud Analytics Technologies, LLC; Jersey City (NJ); On-site; USD 100,000 - 150,000; Full time

Job summary

Cloud Analytics Technologies, LLC seeks an experienced Data Engineer to build and maintain scalable data pipelines, data warehouses, and data integration solutions. The role transforms raw data into analytics-ready datasets to support business intelligence, reporting, and data science initiatives.

You will design ETL/ELT processes using Python, SQL, and Spark, work with Snowflake, Databricks, and cloud platforms, and collaborate with analysts, architects, and stakeholders in an Agile environment.

Qualifications

• Bachelor's degree or equivalent experience in computing fields.
• 5+ years in data engineering, data integration, or ETL development.
• Strong Python, SQL, and data transformation skills.
• Experience with cloud data platforms and distributed processing.
• Understanding of data warehousing and data lake architectures.
• Proven debugging, performance tuning, and production support abilities.

Responsibilities

• Design, develop, and maintain robust ETL/ELT pipelines for structured and semi-structured data.
• Ingest data from APIs, databases, files, and streaming sources.
• Build scalable data transformation workflows using Python, SQL, and Spark.
• Develop and optimize data warehouse tables, data models, and analytical datasets.
• Implement batch and real-time data processing solutions.
• Improve query performance, pipeline reliability, and data processing efficiency.
• Implement data quality checks, validation rules, reconciliation, and error-handling mechanisms.
• Monitor production pipelines and troubleshoot data failures.
• Apply data security, access controls, and governance standards.
• Collaborate with data analysts, data scientists, architects, and business stakeholders.
• Automate deployments and support CI/CD practices.
• Maintain technical documentation and participate in Agile development activities.

Skills

Python

SQL

PySpark

ETL/ELT design

Data modeling

CI/CD practices

Data governance

Education

Bachelor's degree in Computer Science / IT / Engineering or related field

Tools

Informatica

Azure Data Factory

AWS Glue

Snowflake

Databricks

Amazon Redshift

Google BigQuery

Apache Spark

Hadoop

Kafka

PostgreSQL

SQL Server

Oracle

MySQL

AWS

Azure

GCP

Star schema

Snowflake schema

Dimensional modeling

Data warehouse design

Git

CI/CD

Docker

Job description

Cloud Analytics Technologies, LLC seeks an experienced Data Engineer to build and maintain scalable data pipelines, data warehouses, and data integration solutions. The role transforms raw data into analytics-ready datasets to support business intelligence, reporting, and data science initiatives.

You will design ETL/ELT processes using Python, SQL, and Spark, work with Snowflake, Databricks, and cloud platforms, and collaborate with analysts, architects, and stakeholders in an Agile environment.