Data Engineer

Illumination-Works-LL; United States; Remote; USD 110,000 - 160,000; Full time

Benefits offered by this job

Competitive salary

PTO

Medical, dental, vision insurance

401K

Training opportunities

Job summary

Illumination Works is seeking a Data Engineer to design, build, and optimize data pipelines and data models supporting analytics and data science initiatives. You will work with data scientists and engineers to ensure reliable, high-quality data delivery across multiple data sources.

The role emphasizes hands-on development with Python, SQL, and Databricks, plus Delta Lake and data ingestion strategies. A Secret clearance is required for our government-related projects, with strong communication

Qualifications

• 4+ years of experience as a Data Engineer or in a similar data-focused role.
• Advanced SQL skills with complex joins, aggregations and window functions.
• Python proficiency for data engineering and transformation.
• Experience with Databricks notebooks and data pipelines.
• Experience with Delta Lake and data modeling.

Responsibilities

• Develop, maintain, and enhance data pipelines using Python, SQL, and Databricks.
• Develop and maintain Databricks notebooks supporting data ingestion, transformation, and delivery.
• Design and maintain data tables, schemas, and Delta Lake structures for analytics use.
• Collaborate with data scientists to evolve pipelines and data structures.
• Create complex SQL queries with joins, aggregations, and window functions.
• Integrate and transform data from multiple sources including structured and semi-structured data.
• Support data quality through validation, troubleshooting, and issue resolution.
• Optimize pipelines and queries for performance and scalability.
• Document data sources, transformations, and pipeline configurations.
• Collaborate with stakeholders to deliver data solutions meeting mission requirements.

Skills

SQL

Python

Databricks

ETL/ELT

Delta Lake

Data modeling

Data integration

Data quality

Communication

Education

Bachelor's degree in CS/Math/DS/Engineering

Tools

Databricks

Job description

If you are unable to complete this application due to a disability, contact this employer to ask for an accommodation or an alternative application process.

US

16 days ago Requisition ID: 1221

Location:Remote

We are seeking an experienced Data Engineerto support challenging data engineering, data management, and analytics initiatives. In this role, you will develop and maintain data pipelines and data structures that enable advanced analytics and data science capabilities. You will work closely with data scientists and other technical stakeholders to understand evolving data needs, incorporate new data elements, and ensure reliable, high-quality data is available for analysis.

Key Responsibilities

• Develop, maintain, and enhance data pipelines using Python, SQL, and Databricks
• Develop and maintain Databricks notebooks supporting data ingestion, transformation, and delivery
• Design and maintain data tables, schemas, and Delta Lake structures to support analytics and data science requirements
• Work closely with data scientists to understand data requirements and modify pipelines, tables, and data structures as needs evolve
• Develop complex SQL queries using joins, aggregations, window functions, and other advanced techniques to transform and prepare data
• Integrate and transform data from multiple structured and semi-structured data sources
• Support data quality through validation, troubleshooting, and resolution of data issues
• Optimize data pipelines, SQL queries, and data structures for performance
• Document data sources, transformations, data flows, schemas, and pipeline configurations
• Collaborate with data scientists, engineers, analysts, and business stakeholders to deliver data solutions that support mission requirements

Do you have what it takes?

We value passion, curiosity, and perseverance, along with the ability to communicate ideas and results to diverse audiences. We look for people who thrive in both collaborative and independent assignments, quickly learn new data and business domains, and enjoy solving complex data challenges.

Required Qualifications

• 4+ years of experience as a Data Engineer or in a similar data-focused role
• Possess advanced SQL skills (i.e., complex joins, aggregations, window functions, transformations)
• Proficiency with Python for data engineering and transformation
• Hands-on experience with Databricks, including development and maintenance of notebooks and data pipelines
• Experience developing and maintaining ETL/ELT pipelines and data integration workflows
• Experience with Delta Lake
• Experience with database design, data modeling, schema development, and data-centric documentation
• Experience integrating and transforming data from multiple data source types
• Experience troubleshooting data issues and optimizing queries and pipelines
• Strong analytical and problem-solving skills
• Excellent verbal and written communication skills and ability to collaborate effectively with technical and business stakeholders
• Bachelor's degree in Computer Science, Mathematics, Data Science, Engineering, or a comparable academic discipline
• Must hold an active Secret clearance with eligibility for Top Secret (U.S. Citizenship required)
• Must successfully pass required background screening and drug testing

Preferred Qualifications

• Experience supporting data scientists and advanced analytics or machine learning use cases
• Functional knowledge of Department of Waror U.S. Air Force data sources and systems
• Experience working with logistics, supply chain, maintenance, or other operational data
• CompTIA Security+ certification

About ILW

At Illumination Works, we know data, and we should, we’ve been doing it since we started in 2006! We specialize in everything data from big data to data science, data engineering, software engineering, and cloud design. We are a trusted technology partner in user-centered digital transformation-delivering impactful business results to clients. We partner with customers to solve their unique technology and data challenges and stay on top of modern technologies and advancements leveraging our Innovation Lab. Check out our website to learn more at www.ilwllc.com .

Why choose us?

We invest in our employees in all aspects of their life and we value family. We offer market-competitive salary, a generous PTO package, and comprehensive medical, dental, vision and life insurance plans. We also offer 401K, short/long-term disability insurance, a fun and engaging culture, and training opportunities to keep you up to speed on the latest technologies.

Commitment to Non-Discrimination

All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran, or any other status protected by applicable federal, state, local, or international law.

Accommodations

If you are an individual with a disability and would like to request a reasonable workplace accommodation for any part of our employment process, please contact the Human Resources Staff by sending an email to [e-mail] . This option is reserved only for individuals who are requesting a reasonable workplace accommodation. It is not intended for other purposes or inquiries.

Notification for current or previously cleared professionals:

Official U.S. Government information appearing in the public domain shall not automatically be considered UNCLASSIFIED or approved for public release. ILW recognizes that information contained in resumes of current or previously cleared professionals may be sensitive, containing potentially proprietary and/or protected information. Protected Information is considered classified, in the process of a classification determination, or unclassified, but protected by statute. Therefore, all resumes should be approved for public release by a U.S. Government Official with Original Classification Authority, prior to posting the resume to ILW’s applicant tracking system.

By submitting my resume, I understand that I am NOT authorized to upload content with Official U.S. Government information that is considered, sensitive, proprietary, or protected.