Big Data Developer

EdgeQ Technologies Private Limited

Chicago (IL)

Hybrid

USD 85,000 - 160,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Health insurance
Paid time off
Professional development
Opportunities to work on large-scale数据

Job summary

EdgeQ Technologies Private Limited is seeking a skilled Big Data Developer in the United States. The role requires 2–8 years of experience and focuses on designing, developing, and maintaining scalable data processing systems.

Candidates should have expertise in Spark, Hadoop, and related technologies and be comfortable working across cloud platforms. You will build ETL/ELT pipelines, write optimized SQL, and collaborate with data engineers, scientists, and analysts.

Qualifications

  • 2–8 years of relevant experience in big data development.
  • Strong programming skills with big data technologies.
  • Experience with distributed computing and data pipelines.
  • Familiarity with cloud data platforms and data warehouses.
  • Ability to write SQL and data transformation scripts.

Responsibilities

  • Design, develop, and maintain scalable big data applications and pipelines.
  • Build ETL/ELT workflows for structured and unstructured data.
  • Develop data processing with Spark, Hadoop, Hive, and Kafka.
  • Write optimized SQL queries and data transformations.
  • Integrate data from databases, APIs, and external sources.
  • Develop real-time streaming solutions using Kafka or similar.
  • Work with AWS, Azure, or GCP to deploy data platforms.
  • Optimize performance, scalability, and reliability of data jobs.
  • Collaborate with data engineers, scientists, analysts, and developers.
  • Implement data quality, security controls, and monitoring.
  • Troubleshoot data issues and maintain documentation.

Skills

Python
Java
Scala
SQL
Apache Spark
Hadoop
Hive
Kafka
Distributed computing
ETL/ELT pipelines
Data integration
Relational databases
NoSQL databases
Cloud data platforms
Data warehousing
Data modeling
Performance tuning
Data quality
Analytical skills
Communication skills

Education

Bachelor's degree in Computer Science / IT / Data Engineering

Tools

Git
Docker
CI/CD
Apache Airflow
Delta Lake
Dataproc / Databricks / EMR

Job description

Big Data Developer Job Title: Big Data Developer Job Locations: Austin, TX; Dallas, TX; Seattle, WA; Chicago, IL; New York, NY Job Type: Full-Time Experience Required: 2–8 Years Work Mode: Remote / Hybrid / On-site, depending on employer Estimated Annual Salary: $85,000–$160,000

Job Overview

We are seeking a skilled Big Data Developer with 2–8 years of experience to design, develop, and maintain scalable data processing systems. The ideal candidate will have strong programming skills and hands-on experience with big data technologies, distributed computing, cloud platforms, and data pipeline development.

Key Responsibilities
  • Design, develop, and maintain scalable big data applications and processing pipelines.
  • Build ETL/ELT workflows to process structured and unstructured data.
  • Develop data processing solutions using Apache Spark, Hadoop, and related technologies.
  • Write optimized SQL queries and data transformation scripts.
  • Integrate data from databases, APIs, and external sources.
  • Develop real-time data streaming solutions using Apache Kafka or similar technologies.
  • Work with AWS, Microsoft Azure, or Google Cloud Platform.
  • Optimize data processing performance, scalability, and reliability.
  • Collaborate with data engineers, data scientists, analysts, and software developers.
  • Implement data quality checks, security controls, and monitoring.
  • Troubleshoot data processing issues and maintain technical documentation.
Required Skills
  • Proficiency in Python, Java, Scala, or SQL.
  • Experience with Apache Spark, Hadoop, Hive, or Kafka.
  • Understanding of distributed computing and big data architecture.
  • Hands-on experience with ETL/ELT pipelines and data integration.
  • Knowledge of relational and NoSQL databases.
  • Familiarity with cloud data platforms and data warehouses.
  • Experience with data modeling, performance tuning, and data quality.
  • Strong analytical, problem-solving, and communication skills.
Experience Levels & Estimated Annual Salary

ExperienceEstimated Annual Salary (USD)2–3 Years$85,000–$105,0003–5 Years$100,000–$130,0005–8 Years$125,000–$160,000 Salary estimates are indicative and vary by employer, location, industry, and technical expertise.

Educational Qualifications

Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field preferred. Equivalent professional experience, relevant certifications, and demonstrated technical skills may be considered.

Preferred Qualifications
  • Experience with AWS EMR, Azure Databricks, or Google Cloud Dataproc.
  • Familiarity with Apache Airflow, Delta Lake, or cloud data warehouses.
  • Knowledge of real-time streaming, data lakes, and distributed storage.
  • Experience with Git, Docker, CI/CD, and automated testing.
  • Understanding of data governance, security, and privacy standards.
Benefits
  • Competitive salary and performance-based incentives.
  • Health, dental, and vision insurance, where offered.
  • Paid time off and company holidays.
  • Professional development and certification opportunities.
  • Opportunities to work on large-scale data engineering projects.

Equal Opportunity Statement: We welcome qualified applicants from diverse backgrounds.

Candidates must meet the applicable US work authorization requirements for the position.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

EdgeQ Technologies Private Limited • New York (NY)

Hybrid
USD 85,000 - 160,000
Health insurance
Paid time off
Professional development
+1
Data Developer
Data Developer

SUNRISE ENERGY • United States

Hybrid
USD 90,000 - 140,000
Flexible working arrangements
Hybrid work model
Career development opportunities
Data Developer
Data Developer

Vista Empleos Argentina • United States

Hybrid
USD 90,000 - 130,000
Competitive salary + bonuses
Flexible or hybrid work
Big Data Engineer
Big Data Engineer

Remotebridge • United States

Remote
USD 120,000 - 180,000
Big Data Engineer
Big Data Engineer

Compunnel Inc. • Costa Mesa (CA)

Remote
USD 120,000 - 160,000
Software Engineer, Big Data
Software Engineer, Big Data

ziprecruiter • United States

Hybrid
USD 130,000 - 175,000
Hybrid work
Competitive compensation
Extensive benefits
+1
Big Data Engineer
Big Data Engineer

Compunnel Inc. • Tysons (VA)

On-site
USD 150,000 - 190,000
Senior Data Engineer
Senior Data Engineer

Unisys • McLean (VA)

On-site
USD 120,000 - 180,000
Big Data Application Developer
Big Data Application Developer

Bright Vision Technologies • United States

Remote
USD 100,000 - 150,000
Remote work
SOFTWARE DEVELOPER - HADOOP OR BIGDATA
SOFTWARE DEVELOPER - HADOOP OR BIGDATA

IntelliGroup USA LTD • Irving (TX)

On-site
USD 134,802 - 164,759