Data Engineer

Brilliant Infotech Inc.; Edison (NJ); On-site; USD 83,000 - 124,000; Part time

Job summary

Brilliant Infotech Inc. is seeking a skilled Data Engineer in Edison, NJ to design, develop, and maintain scalable data pipelines and data solutions.

The role requires strong experience with ETL/ELT, cloud data platforms, databases, and analytics technologies. The ideal candidate will build and optimize data integration for structured and unstructured data, work with AWS/Azure/GCP, and collaborate with Data Scientists and engineers to meet data requirements while ensuring governance and security

Qualifications

• Strong proficiency in SQL and relational databases.
• Hands-on experience with Python/PySpark and data pipelines.
• Experience with ETL/ELT tooling and cloud data platforms.
• Knowledge of data warehousing concepts and technologies.
• Experience with Spark, Databricks, or Hadoop.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Develop and optimize data integration solutions for structured and unstructured data.
• Work with cloud data platforms such as AWS, Azure, or GCP.
• Build and maintain data warehouses, data lakes, and lakehouse solutions.
• Develop complex SQL queries, stored procedures, and data transformation processes.
• Work with big data technologies such as Spark, Databricks, or Hadoop.
• Implement data quality, validation, governance, and security processes.
• Integrate data from APIs, databases, applications, and other enterprise data sources.
• Optimize data pipelines and queries for performance, scalability, and reliability.
• Collaborate with Data Scientists, Analysts, Engineers, and business teams to understand data requirements.
• Monitor data pipelines and troubleshoot data processing and integration issues.
• Maintain technical documentation for data architecture, pipelines, workflows, and processes.

Skills

SQL proficiency

Python

PySpark

ETL/ELT

Data modeling

Data governance

Git & CI/CD

Cloud data platforms

Data warehousing

Big data concepts

Education

Bachelor's degree in CS/IT/Engineering/Data Science

Tools

Snowflake

Databricks

Azure Data Factory

AWS Glue

Redshift

Job description

Postion: Data Engineer Location: Edison, NJ (Onsite) Type of Employment: W2 Contract Duration: 06+ Months

Job Summary

We are looking for a skilled Data Engineer to design, develop, and maintain scalable data pipelines and data solutions. The ideal candidate should have strong experience with data integration, ETL/ELT processes, cloud data platforms, databases, and data analytics technologies.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Develop and optimize data integration solutions for structured and unstructured data.
• Work with cloud data platforms such as AWS, Azure, or Google Cloud Platform (GCP).
• Build and maintain data warehouses, data lakes, and lakehouse solutions.
• Develop complex SQL queries, stored procedures, and data transformation processes.
• Work with big data technologies such as Spark, Databricks, or Hadoop.
• Implement data quality, validation, governance, and security processes.
• Integrate data from APIs, databases, applications, and other enterprise data sources.
• Optimize data pipelines and queries for performance, scalability, and reliability.
• Collaborate with Data Scientists, Data Analysts, Software Engineers, and business teams to understand data requirements.
• Monitor data pipelines and troubleshoot data processing and integration issues.
• Maintain technical documentation for data architecture, pipelines, workflows, and processes.

Required Skills

• Strong proficiency in SQL and relational databases.
• Hands-on experience with Python, PySpark, or similar programming languages.
• Experience with ETL/ELT tools and data pipeline development.
• Experience with cloud platforms such as AWS, Azure, or GCP.
• Knowledge of data warehousing concepts and technologies.
• Experience with technologies such as Snowflake, Databricks, Azure Data Factory, AWS Glue, or Redshift.
• Experience working with Apache Spark/PySpark.
• Strong understanding of data modeling, data integration, and database concepts.
• Experience with Git and CI/CD practices.
• Good understanding of data security, governance, and data quality.

Preferred Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
• Experience with Kafka or other real-time data streaming technologies.
• Knowledge of dbt, Airflow, or similar data orchestration tools.
• Experience with data visualization tools such as Power BI or Tableau.
• Experience working with modern cloud data architectures and lakehouse platforms.
• Relevant certifications in AWS, Azure, GCP, Databricks, or Snowflake.