GCP Data Lead

TechDigital Group; Charlotte (NC); On-site; USD 110,000 - 150,000; Full time

Job summary

TechDigital Group in Charlotte, NC is seeking a senior data software engineer to design and optimize large-scale data pipelines using GCP tools like BigQuery, Airflow, Pub/Sub, and Spark. You will collaborate with product teams to turn complex data into actionable insights.

The role requires 5+ years of hands-on development, strong OO programming and SQL, plus experience with CI/CD, Git, and Jenkins. A passion for learning cloud tech and continuous improvement is essential.

Qualifications

• 5+ years of hands-on software development experience with Big Data & Analytics solutions.
• GCP Cloud - Big Query, Airflow, DataProc, PubSub, Hadoop Hive, Spark, Python, shell scripting
• Strong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.
• Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.
• Experience of building event processing pipelines with Kafka or GCP PubSub.
• Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.
• Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.
• Strong Object-Oriented Programming skills and design patterns.
• Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).
• Good knowledge and experience with configuration management tools like GitHub
• Ability to analyze complex data engineering problems, propose effective solutions, and implement them effectively.
• Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big Data
• Looks proactively beyond the obvious for continuous improvement opportunities.
• Communicates effectively with product and cross functional team.
• Willingness to learn new technologies and leverage them to their optimal potential.
• Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.
• Certifications in cloud platform (GCP Professional Data Engineer) is a plus.
• Provide engineering expertise to the team for developing features for Banking including Financial Crimes, Anti-money Laundering platforms
• Improve engineering practices, including driving resilience, predictability, and end efficiency through automation
• Partner with product teams to envision and deliver solutions which transform Financial Crimes Compliance using technolo

Responsibilities

• Develop features for Banking including Financial Crimes, Anti-money Laundering platforms.
• Improve engineering practices, including driving resilience, predictability, and end efficiency through automation.
• Partner with product teams to envision and deliver solutions which transform Financial Crimes Compliance using technology.

Job description

• 5+ years of hands-on software development experience with Big Data & Analytics solutions
• GCP Cloud - Big Query, Airflow, DataProc, PubSub, Hadoop Hive, Spark, Python, shell scripting
• Strong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.
• Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.
• Experience of building event processing pipelines with Kafka or GCP PubSub.
• Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.
• Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.
• Strong Object-Oriented Programming skills and design patterns.
• Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).
• Good knowledge and experience with configuration management tools like GitHub
• Ability to analyze complex data engineering problems, propose effective solutions, and implement them effectively.
• Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big Data
• Looks proactively beyond the obvious for continuous improvement opportunities.
• Communicates effectively with product and cross functional team.
• Willingness to learn new technologies and leverage them to their optimal potential.
• Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.
• Certifications in cloud platform (GCP Professional Data Engineer) is a plus.
• Provide engineering expertise to the team for developing features for Banking including Financial Crimes, Anti-money Laundering platforms
• Improve engineering practices, including driving resilience, predictability, and end efficiency through automation
• Partner with product teams to envision and deliver solutions which transform Financial Crimes Compliance using technolo