Big Data Engineer: Cloud, Spark & Real-Time Pipelines

EdgeQ Technologies Private Limited

Chicago (IL)

Hybrid

USD 85,000 - 160,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Health insurance
Paid time off
Professional development
Opportunities to work on large-scale数据

Job summary

EdgeQ Technologies Private Limited is seeking a skilled Big Data Developer in the United States. The role requires 2–8 years of experience and focuses on designing, developing, and maintaining scalable data processing systems.

Candidates should have expertise in Spark, Hadoop, and related technologies and be comfortable working across cloud platforms. You will build ETL/ELT pipelines, write optimized SQL, and collaborate with data engineers, scientists, and analysts.

Qualifications

  • 2–8 years of relevant experience in big data development.
  • Strong programming skills with big data technologies.
  • Experience with distributed computing and data pipelines.
  • Familiarity with cloud data platforms and data warehouses.
  • Ability to write SQL and data transformation scripts.

Responsibilities

  • Design, develop, and maintain scalable big data applications and pipelines.
  • Build ETL/ELT workflows for structured and unstructured data.
  • Develop data processing with Spark, Hadoop, Hive, and Kafka.
  • Write optimized SQL queries and data transformations.
  • Integrate data from databases, APIs, and external sources.
  • Develop real-time streaming solutions using Kafka or similar.
  • Work with AWS, Azure, or GCP to deploy data platforms.
  • Optimize performance, scalability, and reliability of data jobs.
  • Collaborate with data engineers, scientists, analysts, and developers.
  • Implement data quality, security controls, and monitoring.
  • Troubleshoot data issues and maintain documentation.

Skills

Python
Java
Scala
SQL
Apache Spark
Hadoop
Hive
Kafka
Distributed computing
ETL/ELT pipelines
Data integration
Relational databases
NoSQL databases
Cloud data platforms
Data warehousing
Data modeling
Performance tuning
Data quality
Analytical skills
Communication skills

Education

Bachelor's degree in Computer Science / IT / Data Engineering

Tools

Git
Docker
CI/CD
Apache Airflow
Delta Lake
Dataproc / Databricks / EMR

Job description

EdgeQ Technologies Private Limited is seeking a skilled Big Data Developer in the United States. The role requires 2–8 years of experience and focuses on designing, developing, and maintaining scalable data processing systems.

Candidates should have expertise in Spark, Hadoop, and related technologies and be comfortable working across cloud platforms. You will build ETL/ELT pipelines, write optimized SQL, and collaborate with data engineers, scientists, and analysts.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer: Build Scalable Pipelines & Cloud Data
Data Engineer: Build Scalable Pipelines & Cloud Data

EdgeQ Technologies Private Limited • Dallas (TX)

On-site
USD 100,000 - 160,000
Remote Cloud Data Engineer – ETL & Data Warehousing
Remote Cloud Data Engineer – ETL & Data Warehousing

EdgeQ Technologies Private Limited • New York (NY)

Hybrid
USD 85,000 - 160,000
Health insurance
Paid time off
Professional development
+1
Senior Data Engineer: Real-Time Pipelines & Cloud Analytics
Senior Data Engineer: Real-Time Pipelines & Cloud Analytics

Cloudndata • United States

Hybrid
USD 140,000 - 180,000
Big Data Engineer: Spark, ETL & Data Quality
Big Data Engineer: Spark, ETL & Data Quality

eNcloud Services LLC • Katy (TX)

On-site
USD 100,000 - 170,000
Cloud Data Engineer: Spark, Snowflake & Azure Pipelines
Cloud Data Engineer: Spark, Snowflake & Azure Pipelines

Quantum Technologies. LLC • San Jose (CA)

On-site
USD 120,000 - 180,000
Data Engineer: Spark, Hadoop & ETL Pipelines
Data Engineer: Spark, Hadoop & ETL Pipelines

Quantum Technologies. LLC • Alpharetta (GA)

On-site
USD 120,000 - 180,000
Big Data Engineer: ETL, Spark & Data Pipelines
Big Data Engineer: ETL, Spark & Data Pipelines

Synechron • Charlotte (NC)

On-site
USD 100,000 - 110,000
Highly competitive compensation and benefits package
10 days of paid annual leave
Comprehensive insurance plan
+2
Analytics Engineer: Build Scalable Data Pipelines
Analytics Engineer: Build Scalable Data Pipelines

EdgeQ Technologies Private Limited • San Francisco (CA)

On-site
USD 120,000 - 180,000
Remote Junior Data Engineer (Python, SQL, Spark)
Remote Junior Data Engineer (Python, SQL, Spark)

EdgeQ Technologies Private Limited • United States

Hybrid
USD 75,000 - 115,000
Health insurance
Retirement plans
Paid time off
+1
Senior Data Engineer: Scalable Spark Pipelines & Real-Time
Senior Data Engineer: Scalable Spark Pipelines & Real-Time

3M Consultancy • Washington

On-site
USD 100,000 - 130,000