Data Engineer

Insight Global; Richmond (VA); On-site; USD 82,656 - 110,208; Full time

Job summary

Insight Global is seeking experienced Data Engineers in Richmond, Virginia to support a high-impact data platform initiative. In this role, you will design and maintain scalable data pipelines leveraging Databricks and AWS technologies, while collaborating with cross-functional teams.

Candidates should have a minimum of 3 years of Databricks experience, a strong background in Python and data engineering, and an active Databricks certification. This position offers flexible pay rates between $60 and $80 per hour depending on experience.

Qualifications

• 3+ years of hands-on Databricks experience.
• Active Databricks Certification (Data Engineer Associate minimum required).
• Experience building and maintaining ETL/ELT pipelines.

Responsibilities

• Design and maintain scalable data pipelines.
• Build and optimize data workflows within Databricks.
• Support data migration into AWS-based CDP environments.

Skills

Databricks

Python

PySpark

ETL/ELT pipelines

AWS environments

Tools

Delta Lake

Starburst

Immuta

Collibra

Job description

Job Description

We are seeking experienced Data Engineers to support a high-impact data platform initiative. This team will be responsible for building and migrating data assets into a Common Data Platform (CDP) leveraging a modern AWS-based data stack, including Databricks, Starburst, Immuta, and Collibra.

This role requires strong hands-on experience in Databricks, modern data architecture, and cloud-based data engineering.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines to ingest, transform, and deliver high-quality data from multiple sources
• Build and optimize data workflows within a Databricks-based architecture
• Support data migration efforts into AWS-based CDP environments
• Implement monitoring, alerting, and auto-remediation to ensure pipeline reliability
• Ensure adherence to data quality, governance, and security-first practices
• Collaborate with cross-functional teams (product, analytics, data science, and business stakeholders)
• Participate in Agile ceremonies and follow SAFe Agile delivery practices
• Deliver production-ready data solutions aligned with business needs

Pay rate between 60 - 80/hr based on experience.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [e-mail]. To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

Skills and Requirements

• Minimum 3+ years of hands-on Databricks experience
• Active Databricks Certification (Data Engineer Associate minimum required)
• Strong experience with: Delta Lake architecture and performance optimization, Unity Catalog for governance
• Databricks workflows and job orchestration, Auto Loader for incremental ingestion, Medallion architecture (Bronze/Silver/Gold)
• 3+ years of Python and PySpark experience
• Experience building and maintaining ETL/ELT pipelines
• Strong experience with relational and NoSQL databases
• Knowledge of data modeling techniques (e.g., STAR schema, dimensional modeling)
• Experience working in AWS environments
• Familiarity with modern data tools and frameworks such as: S3 (object storage), Spark, Airflow or workflow orchestration tools, Redshift or Snowflake
• Experience using Jupyter notebooks (development, testing, and debugging) Databricks Data Engineer Professional Certification
• Experience with: Starburst, Immuta, Collibra, or AI-assisted data migration tools (e.g., Lakebridge AI or similar)
• Experience working within Common Data Platform (CDP) environments
• Exposure to real-time data processing / streaming pipelines
• Previous experience supporting federal or regulated environments
• Prior security clearance