Lead - GCP Data Engineer

TechDigital Group; Nashville (TN); On-site; USD 80,000 - 120,000; Full time

Job summary

An innovative firm is seeking a skilled GCP Data Engineer to enhance data integration processes with Lawson ERP. In this role, you will build and optimize data pipelines, ensuring seamless data flow between GCP and enterprise systems. Your expertise in cloud architecture and data security will be crucial in delivering robust ETL solutions. Collaborating with cross-functional teams, you will document workflows and implement automation to improve efficiency. This position offers a dynamic environment where you can stay ahead of emerging trends in data engineering and make a significant impact on data-driven decision-making.

Qualifications

• 4+ years in data engineering with strong focus on GCP.
• Proven experience integrating ERP systems with cloud platforms.

Responsibilities

• Design and implement ETL processes for data integration.
• Collaborate with teams to ensure smooth data flow and optimization.

Skills

Data Integration

ETL Development

GCP (Google Cloud Platform)

SQL

Data Warehousing

Cloud-based ETL tools

BigQuery

Apache Airflow

Tools

Apache Beam

Dataflow

Cloud Functions

Cloud Pub/Sub

Job description

Mandatory skill:
1. Data Integration (Lawson to GCP)
a. GCP data flow,
b. Data Proc in Python
c. Big Query

Job Description: We are seeking a skilled GCP Data Engineer with expertise in integrating data solutions into the Lawson ERP system. The successful candidate will be responsible for building, maintaining, and optimizing data pipelines that integrate data between Google Cloud Platform (GCP) and the Lawson ERP system, ensuring that all data flow is smooth, reliable, and accurate. As a GCP Data Engineer, you will collaborate with cross-functional teams, including business analysts, IT, and ERP specialists, to design and implement scalable and efficient data solutions. The ideal candidate will have a strong background in cloud-based data engineering, specifically on GCP, and experience working with Lawson and other enterprise systems.

Key Responsibilities:

1. Data Integration & ETL Development: Design and implement robust ETL (Extract, Transform, Load) processes to integrate data from the Lawson ERP system into GCP services (BigQuery, Dataflow, Dataproc, etc.) and other cloud data storage solutions.
2. Cloud Architecture Design: Work closely with cloud architects to design and deploy data pipelines, ensuring optimal performance, scalability, and cost-efficiency within GCP's cloud ecosystem.
3. Lawson Integration: Develop and maintain seamless integration between GCP and Lawson ERP, ensuring smooth data transfer, transformation, and synchronization between the systems.
4. Data Pipeline Optimization: Ensure data pipelines are optimized for high availability, fault tolerance, and scalability. Monitor data pipelines for performance, troubleshoot and resolve issues promptly.
5. Collaboration & Documentation: Collaborate with business stakeholders and project teams to understand data requirements and provide technical recommendations. Document data engineering processes, integration workflows, and troubleshooting guides.
6. Data Security & Governance: Implement data security best practices and ensure compliance with relevant industry standards, including data privacy and governance policies when integrating and handling sensitive business data.
7. Automation: Automate routine data processes to improve efficiency and reduce manual errors, leveraging tools and technologies such as Apache Airflow, Cloud Functions, and other GCP automation services.
8. Reporting & Monitoring: Set up data reporting and monitoring systems to track the performance of integrations, identify bottlenecks, and proactively improve data flows.
9. Continuous Improvement: Stay up-to-date with emerging data engineering trends, GCP tools, and best practices to continuously enhance the quality and efficiency of data engineering processes.

Qualifications:

1. 4+ years of experience in data engineering or cloud data engineering, with a strong focus on GCP (Google Cloud Platform).
2. Proven experience in integrating data systems (especially ERP systems like Lawson) with cloud platforms.
3. Proficiency in SQL, data warehousing, and cloud-based ETL tools (such as Apache Beam, Dataflow, or similar).
4. Strong knowledge of BigQuery, Cloud Storage, Cloud Functions, Cloud Pub/Sub, and other GCP services.
5. Familiarity with Lawson ERP architecture, data models, and integration methods.
6. Experience with data pipeline orchestration tools (e.g., Apache Airflow, Cloud Composer).