Data Engineer with Kafka

Techstra Solutions; Pittsburgh (Allegheny County); On-site; USD 55,000 - 140,000; Full time

Job summary

An innovative firm is seeking a talented Data Engineer to join their dynamic team in Pittsburgh. This role focuses on building and scaling real-time data pipelines using Apache Kafka, ensuring high availability and performance. You will work closely with software engineers and data scientists to integrate diverse data sources and optimize Kafka topics for scalability. If you are passionate about data engineering and eager to make a significant impact in a collaborative environment, this opportunity is perfect for you. Join a company that values transformation and innovation in the tech space.

Qualifications

• 3+ years of experience in data engineering or backend development.
• Strong hands-on experience with Apache Kafka and related technologies.
• Proficiency in Python, Java, or Scala for data integration.

Responsibilities

• Design and maintain robust data pipelines using Apache Kafka.
• Collaborate with teams to build streaming solutions.
• Monitor and troubleshoot Kafka cluster performance.

Skills

Apache Kafka

Python

Java

Scala

Apache Flink

Spark Streaming

Data Structures

Distributed Systems

Event-Driven Architecture

Cloud Platforms (AWS, GCP, Azure)

Education

Bachelor's Degree in Computer Science or related field

Tools

Kafka Streams

Kafka Connect

ksqlDB

Confluent Cloud

Managed Kafka Services

Job description

3 weeks ago Be among the first 25 applicants

Techstra is seeking an experienced Data Engineer with strong experience in Apache Kafka to join our growing data team. You’ll help build and scale our real-time data pipelines, ensuring high availability, reliability, and performance across systems. This role is crucial in transforming how we ingest, process, and distribute data across the organization.

From the day you join, you will hit the ground running surrounded by amazing people. Here is a look of some of the expectations and responsibilities for this position:

Responsibilities

Design, implement, and maintain robust data pipelines using Apache Kafka and related streaming technologies.

Work with Kafka producers and consumers to enable real-time data ingestion and processing.

Optimize Kafka topics, partitions, and consumer groups for performance and scalability.

Integrate data from multiple sources (databases, APIs, cloud services) into the data platform.

Collaborate with software engineers, data scientists, and analysts to build streaming solutions.

Monitor and troubleshoot Kafka cluster performance, ensuring data integrity and reliability.

Document architecture, workflows, and best practices.

Requirements

3+ years of experience in data engineering or backend development.

Strong hands-on experience with Apache Kafka (e.g., Kafka Streams, Kafka Connect, ksqlDB).

Proficiency in Python, Java, or Scala for integrating Kafka producers/consumers.

Experience with stream processing tools like Apache Flink or Spark Streaming

Solid understanding of data structures, distributed systems, and event-driven architecture.

Familiarity with cloud platforms (AWS, GCP, or Azure) and managed Kafka services (like Confluent Cloud or MSK).

Experience with schema management

At Techstra Solutions, we help top companies and brands achieve the business value of Digital and Talent Transformation. We believe there are three components in successful business transformation: Business Strategy, Technology and Talent. It is the coming together of these three disciplines that enable companies to take full advantage of opportunities. It differentiates us. Our approach is holistic and all encompassing. We consider the full picture as we guide our clients on this journey.

We are experts in transformation, business strategy, technology, innovation, and human capital management. We deliver our expertise through client consulting, innovative staffing solutions and software development. From strategy through implementation, we are dedicated to bringing our clients world-class business and talent solutions that fit strategic requirements and most importantly, deliver results.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  Financial Services, Software Development, and Technology, Information and Internet

Referrals increase your chances of interviewing at Techstra Solutions by 2x

Get notified about new Data Engineer jobs in Pittsburgh, PA.

Coraopolis, PA $83,000.00-$138,200.00 1 week ago

Data Engineer - Local to Pittsburgh, PA, Dallas, TX or Cleveland, OH

Pittsburgh, PA $120,000.00-$140,000.00 2 weeks ago

Data Engineer Lead_Pittsburgh, PA_Only on W2_No 1099/C2C

Pittsburgh, PA $75,000.00-$120,000.00 2 months ago

Pittsburgh, PA $55,000.00-$70,000.00 1 week ago

Data Analyst - Local to Pittsburgh or Cleveland

Pittsburgh, PA $85.10-$251,000.00 2 weeks ago

Data Analyst - Local to Pittsburgh, PA or Cleveland, OH

Pittsburgh, PA $115,000.00-$135,000.00 3 weeks ago

Software Engineer, Site Reliability Engineering, Campus

Software Engineer II (Data Services Team) - IDELIC

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.