Sr Data Engineer

Spectraforce Technologies, Inc.; McLean (VA); Hybrid; USD 96,000 - 138,000; Full time

Job summary

Spectraforce Technologies, Inc. is seeking a Sr Data Engineer for a 7-month contract in McLean, VA (Hybrid).

You will design, build, and maintain scalable data pipelines, work with large-scale data sets, and implement real-time streaming solutions using Kafka and AWS services. Responsibilities include implementing ETL/ELT processes, developing REST APIs, and optimizing SQL queries while collaborating with cross-functional teams to deliver reliable data platforms and AI-powered tooling

Qualifications

• Strong hands-on experience with Python and/or Scala.
• Extensive experience with AWS cloud services.
• Hands-on experience with AWS EMR and AWS Glue.
• Experience with Apache Kafka and real-time streaming.
• Strong SQL development and query optimization skills.
• Experience designing and developing REST APIs.
• Strong understanding of data engineering concepts, ETL/ELT pipelines, and distributed data processing.

Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Python and Scala.
• Build and optimize data processing solutions leveraging AWS services, including EMR and AWS Glue.
• Develop and maintain ETL/ELT workflows for large-scale data processing.
• Implement real-time data streaming and event-driven architectures using Apache Kafka.
• Develop, integrate, and maintain REST APIs for data exchange and application integration.
• Write complex SQL queries and optimize database performance.
• Work with distributed computing technologies to process and transform large volumes of structured and unstructured data.
• Collaborate with cross-functional teams to deliver high-quality data engineering solutions.
• Explore and leverage AI-powered tools and technologies to improve data engineering efficiency.

Skills

Python

Scala

REST APIs

SQL

ETL/ELT

Data pipelines

Real-time streaming

Tools

AWS

AWS EMR

AWS Glue

Apache Kafka

Job description

Role: Sr Data Engineer Duration: 7 Months Location: Mclean, VA (Hybrid)

Job Description

The candidate should have experience building robust data pipelines, working with large-scale datasets, implementing real-time streaming solutions, and developing REST APIs.

Key Responsibilities

• Design, develop, and maintain scalable and reliable data pipelines using Python and Scala.
• Build and optimize data processing solutions leveraging AWS services, including EMR and AWS Glue.
• Develop and maintain ETL/ELT workflows for large-scale data processing.
• Implement real-time data streaming and event-driven architectures using Apache Kafka.
• Develop, integrate, and maintain REST APIs for data exchange and application integration.
• Write complex SQL queries and optimize database performance.
• Work with distributed computing technologies to process and transform large volumes of structured and unstructured data.
• Collaborate with cross-functional teams to deliver high-quality data engineering solutions.
• Explore and leverage AI-powered tools and technologies to improve data engineering efficiency.

Required Skills

• Strong hands-on experience with Python and/or Scala.
• Extensive experience with AWS cloud services.
• Hands-on experience with AWS EMR and AWS Glue.
• Experience with Apache Kafka and real-time streaming.
• Strong SQL development and query optimization skills.
• Experience designing and developing REST APIs.
• Strong understanding of data engineering concepts, ETL/ELT pipelines, and distributed data processing.

Preferred Qualifications

• Experience working with AI/ML technologies or AI-powered development tools.
• Experience building scalable cloud-based data platforms.