Data Engineer: Cloud Pipelines & Lakes – Onsite Edison

Brilliant Infotech Inc.; Edison (NJ); Vor Ort; USD 83.000 - 124.000; Teilzeit

Zusammenfassung

Brilliant Infotech Inc. is seeking a skilled Data Engineer in Edison, NJ to design, develop, and maintain scalable data pipelines and data solutions.

The role requires strong experience with ETL/ELT, cloud data platforms, databases, and analytics technologies. The ideal candidate will build and optimize data integration for structured and unstructured data, work with AWS/Azure/GCP, and collaborate with Data Scientists and engineers to meet data requirements while ensuring governance and security

Qualifikationen

• Strong proficiency in SQL and relational databases.
• Hands-on experience with Python/PySpark and data pipelines.
• Experience with ETL/ELT tooling and cloud data platforms.
• Knowledge of data warehousing concepts and technologies.
• Experience with Spark, Databricks, or Hadoop.

Aufgaben

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Develop and optimize data integration solutions for structured and unstructured data.
• Work with cloud data platforms such as AWS, Azure, or GCP.
• Build and maintain data warehouses, data lakes, and lakehouse solutions.
• Develop complex SQL queries, stored procedures, and data transformation processes.
• Work with big data technologies such as Spark, Databricks, or Hadoop.
• Implement data quality, validation, governance, and security processes.
• Integrate data from APIs, databases, applications, and other enterprise data sources.
• Optimize data pipelines and queries for performance, scalability, and reliability.
• Collaborate with Data Scientists, Analysts, Engineers, and business teams to understand data requirements.
• Monitor data pipelines and troubleshoot data processing and integration issues.
• Maintain technical documentation for data architecture, pipelines, workflows, and processes.

Kenntnisse

SQL proficiency

Python

PySpark

ETL/ELT

Data modeling

Data governance

Git & CI/CD

Cloud data platforms

Data warehousing

Big data concepts

Ausbildung

Bachelor's degree in CS/IT/Engineering/Data Science

Tools

Snowflake

Databricks

Azure Data Factory

AWS Glue

Redshift

Jobbeschreibung

Brilliant Infotech Inc. is seeking a skilled Data Engineer in Edison, NJ to design, develop, and maintain scalable data pipelines and data solutions.

The role requires strong experience with ETL/ELT, cloud data platforms, databases, and analytics technologies. The ideal candidate will build and optimize data integration for structured and unstructured data, work with AWS/Azure/GCP, and collaborate with Data Scientists and engineers to meet data requirements while ensuring governance and security