Data streaming architect

Data Streaming Architect

We are seeking a highly skilled Data Streaming Architect with expertise in GCP to design and implement real-time data streaming solutions. The ideal candidate will have a deep understanding of event-driven architectures, streaming technologies, and cloud-based data platforms, ensuring optimal performance, scalability, and reliability of enterprise data streaming solutions.

• Design and implement real-time data streaming architectures using GCP services and others.
  
• Develop solutions leveraging Apache Kafka, Cassandra, GCP and other real-time data streaming technologies.
  
• Real-Time Data Processing: Experience in handling real-time streaming data pipelines for network performance and customer insights.
  
• Cloud Data Warehousing (DWH): Advanced knowledge of designing and implementing DWH architecture for large-scale data ingestion, with a preference for GCP Big Query.
  
• Experience with GCP Big Query and other cloud data warehousing solutions.
  
• Architect, deploy and optimize event-driven and microservices-based architectures.
  
• Define and enforce best practices for data ingestion, transformation, and streaming pipeline optimization.
  
• Ensure high availability, fault tolerance, and security of data streaming solutions.
  
• Collaborate with data engineers, cloud architects, DevOps teams, and business stakeholders to align technical strategies with business needs.
  
• Implement monitoring and alerting mechanisms to proactively detect and address issues in streaming pipelines.
  
• Evaluate and recommend emerging technologies to enhance data streaming capabilities.
  

Required Qualifications:

• 12+ years of experience in data architecture, cloud computing and real-time data processing.
  
• Hands-on experience with Apache Kafka (Confluent), Cassandra etc and related technologies.
  
• Strong expertise in GCP.
  
• Realtime services experience using GCP services like Pub/Sub, Cloud Functions, Datastore, and Cloud Spanner.
  
• Experience with message queues (e.g., RabbitMQ) and event-driven patterns.
  
• Hands-on experience with data serialization formats (e.g., Avro, Parquet, JSON) and schema registries.
  
• Strong understanding of DevOps and CI/CD pipelines for data streaming solutions.
  
• Familiarity with containerization and orchestration tools
  
• Excellent communication and leadership skills, with experience collaborating across technical and business teams.
  
• Good Knowledge of programming languages such as Python, Java, or Scala for real-time data processing