Data Engineer

SIMARN Solutions; Charlotte (NC); On-site; USD 140,000 - 190,000; Full time

Job summary

Simarn Solutions in Charlotte, NC seeks an experienced Lead Data Engineer with strong GCP and Big Data expertise to design scalable data solutions for banking, financial crimes, and AML platforms. You will design and optimize large-scale data pipelines, build event processing with Kafka or Pub/Sub, and lead data modeling, orchestration with Airflow, and CI/CD practices to deliver reliable systems.

The role requires 5+ years in data engineering, hands-on experience with Spark, Dataproc, and

Qualifications

• 5+ years of hands-on experience with Big Data and Analytics solutions.
• Strong experience with GCP services, including BigQuery, Dataproc, and Pub/Sub.
• Hands-on experience with Airflow, Hadoop, Hive, Spark, Python, and shell scripting.
• Strong SQL skills and experience designing scalable data models.
• Experience with Kafka or Pub/Sub for event-driven data processing.
• Understanding of distributed systems, relational databases, object-oriented programming, and design patterns.
• Experience with Git/GitHub, Jenkins, CI/CD pipelines, and automated testing frameworks.
• Familiarity with Agile/Scrum and software development lifecycle practices.
• Strong problem-solving, communication, and collaboration skills.

Responsibilities

• Design, develop, and optimize large-scale data pipelines for ingestion, transformation, and analytics.
• Build event processing pipelines using Kafka or GCP Pub/Sub.
• Develop and optimize SQL queries and data models for performance and scalability.
• Orchestrate data workflows using Airflow and implement reliable processing solutions using Spark and Dataproc.
• Improve engineering practices through automation, testing, CI/CD, and resilient system design.
• Collaborate with product and cross-functional teams to deliver financial crimes compliance solutions.
• Identify opportunities to improve pipeline reliability, efficiency, and maintainability.

Skills

Problem solving

Communication

Collaborative

Agile/Scrum familiarity

Tools

GCP (BigQuery, Dataproc, Pub/Sub)

Airflow

Hadoop

Hive

Spark

Python

SQL

Kafka

CI/CD (Jenkins) / Git/GitHub

Shell scripting

Job description

Work Authorization: Must be authorized to work in the United States.

We are seeking an experienced Lead Data Engineer with strong GCP and Big Data expertise to design and deliver scalable data solutions supporting banking, financial crimes, and Anti-Money Laundering (AML) platforms.

Key Responsibilities

• Design, develop, and optimize large-scale data pipelines for ingestion, transformation, and analytics.
• Build event processing pipelines using Kafka or GCP Pub/Sub.
• Develop and optimize SQL queries and data models for performance and scalability.
• Orchestrate data workflows using Airflow and implement reliable processing solutions using Spark and Dataproc.
• Improve engineering practices through automation, testing, CI/CD, and resilient system design.
• Collaborate with product and cross-functional teams to deliver financial crimes compliance solutions.
• Identify opportunities to improve pipeline reliability, efficiency, and maintainability.

Required Qualifications

• 5+ years of hands-on software development experience with Big Data and Analytics solutions.
• Strong experience with GCP services, including BigQuery, Dataproc, and Pub/Sub.
• Hands-on experience with Airflow, Hadoop, Hive, Spark, Python, and shell scripting.
• Strong SQL skills and experience designing scalable data models.
• Experience with Kafka or Pub/Sub for event-driven data processing.
• Understanding of distributed systems, relational databases, object-oriented programming, and design patterns.
• Experience with Git/GitHub, Jenkins, CI/CD pipelines, and automated testing frameworks.
• Familiarity with Agile/Scrum and software development lifecycle practices.
• Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

• GCP Professional Data Engineer certification.
• Experience supporting banking, financial crimes, or AML platforms.
• Familiarity with enterprise Big Data frameworks and release automation tools such as XLR.