Enable job alerts via email!

DevOps Lead

JR United Kingdom

Leeds

On-site

GBP 60,000 - 100,000

Full time

4 days ago
Be an early applicant

Boost your interview chances

Create a job specific, tailored resume for higher success rate.

Job summary

An established industry player is looking for a Kafka Real-Time Architect to design and implement high-performance real-time data processing systems. This role involves architecting robust Kafka clusters, integrating with various data processing platforms, and ensuring optimal system performance. You will collaborate closely with cross-functional teams to meet data requirements while implementing security measures and compliance standards. If you are passionate about real-time data processing and want to be at the forefront of technology, this is an exciting opportunity to make a significant impact.

Qualifications

  • Deep understanding of Apache Kafka and related technologies.
  • Experience in architecting scalable, real-time data processing systems.

Responsibilities

  • Architect and design scalable, real-time streaming systems using Apache Kafka.
  • Integrate Kafka with key data processing tools and platforms.
  • Monitor Kafka cluster performance and resolve issues for high availability.

Skills

Apache Kafka
Kafka Streams
Kafka Connect
Apache Spark Streaming
Apache Flink
Apache Beam
Schema Registry
Data Processing
System Performance Optimization
Security Measures

Job description

Social network you want to login/join with:

We are seeking an experienced Kafka Real-Time Architect to design and implement scalable, high-performance real-time data processing systems leveraging Apache Kafka. In this role, you will be responsible for architecting and managing Kafka clusters, ensuring system scalability and availability, and integrating Kafka with various data processing platforms. This position requires a deep understanding of Kafka and related technologies, along with a proactive approach to addressing business data needs and ensuring optimal system performance.

Key Responsibilities:

  1. Architect and design scalable, real-time streaming systems using Apache Kafka, ensuring they are robust, highly available, and meet business requirements for data ingestion, processing, and real-time analytics.
  2. Configure, deploy, and maintain highly available Kafka clusters, ensuring scalability and fault tolerance across distributed environments. Manage configurations, optimize cluster performance, and troubleshoot issues to maintain smooth operations.
  3. Integration & Data Processing:
  4. Integrate Kafka with key data processing tools and platforms, including Kafka Streams, Kafka Connect, Apache Spark Streaming, Apache Flink, Apache Beam, and Schema Registry. This integration will facilitate data stream processing, event-driven architectures, and continuous data pipelines.
  5. Cross-functional Collaboration:
  6. Work closely with product and engineering teams to understand data requirements, design and implement solutions, and ensure seamless integration with existing infrastructure and tools. Provide strategic guidance on best practices for real-time data processing.
  7. Security & Compliance:
  8. Implement and enforce security measures to protect Kafka clusters and data streams, including encryption, access control, and data integrity. Ensure compliance with organizational and regulatory standards.
  9. Performance Monitoring & Optimization:
  10. Proactively monitor Kafka cluster performance, analyze logs, identify bottlenecks, and resolve issues to ensure high availability and low-latency data streaming. Leverage monitoring tools and metrics to enhance decision-making and operational efficiency.
  11. Technical Leadership & Support:
  12. Provide technical leadership and mentorship to development and operations teams. Help troubleshoot and resolve Kafka-related issues in production environments. Drive the continuous improvement of Kafka usage, providing insights into new tools, technologies, and practices.
  13. Staying Current with Industry Trends:
  14. Keep up to date with the latest Apache Kafka features, updates, and emerging industry best practices. Participate in Kafka-related forums, conferences, and workshops to maintain a cutting-edge understanding of the technology.
Get your free, confidential resume review.
or drag and drop a PDF, DOC, DOCX, ODT, or PAGES file up to 5MB.

Similar jobs

Lead DevOps Engineer - Fully Remote Outside IR35

JR United Kingdom

Leeds

Remote

GBP 60,000 - 100,000

6 days ago
Be an early applicant

DevOps Lead

Pyramid Consulting, Inc

Leeds

On-site

USD 70,000 - 120,000

6 days ago
Be an early applicant

Lead DevOps Engineer - Fully Remote Outside IR35

JR United Kingdom

Greater Manchester

Remote

GBP 60,000 - 90,000

3 days ago
Be an early applicant

Lead DevOps Engineer - Fully Remote Outside IR35

JR United Kingdom

Bolton

Remote

GBP 60,000 - 100,000

6 days ago
Be an early applicant

Lead DevOps Engineer - Fully Remote Outside IR35

JR United Kingdom

Ashton-under-Lyne

Remote

GBP 60,000 - 100,000

6 days ago
Be an early applicant

Lead DevOps Engineer - Fully Remote Outside IR35

JR United Kingdom

Leigh

Remote

GBP 60,000 - 90,000

6 days ago
Be an early applicant

Devops Lead

TN United Kingdom

Leeds

Hybrid

GBP 60,000 - 100,000

12 days ago

Lead DevOps Engineer - SaaS Product (MS Azure)

ZigZag Global

Remote

GBP 60,000 - 100,000

6 days ago
Be an early applicant

Lead DevOps Engineer - SaaS Product (MS Azure)

ZigZag Global

London

Remote

GBP 75,000 - 85,000

6 days ago
Be an early applicant