Data Engineer + Python + SQL + Databricks

Improving; Ciudad de México; Remote; MXN 600,000 - 1,000,000; Full time

Job summary

Improving is a modern digital services company focused on positively changing the IT professional landscape. We seek a data or software engineer to migrate SAS code to Python/SQL on Databricks, validate outputs, and own deployment through DABs and GitLab CI/CD.

You will work with Unity Catalog, Workflows, and Asset Bundles, tuning Spark/Delta Lake performance and ensuring data quality while collaborating with client stakeholders. 4+ years of experience and strong English are required.

Qualifications

• 4+ years of professional data or software engineering experience.
• Strong Python and SQL skills.
• Spark and Delta Lake performance tuning experience.
• Data validation and reconciliation experience.
• Hands-on Databricks experience with Unity Catalog, Workflows, and Asset Bundles.
• Infrastructure as code or DAB-based deployment experience.
• GitLab CI/CD experience, including merge request workflows and automated testing
• Strong Git branching and code review discipline
• Clear written and spoken English for client-facing collaboration
• A strong sense of ownership: you scope, deliver, and flag risks without being prompted
• Nice to have: ability to read SAS code, exposure to healthcare data, and Azure experience

Responsibilities

• Migrate SAS code to Python and SQL on Databricks, focusing on pipeline reliability, validation, and delivery.
• Build and run pipelines that process SAS inventories and converted outputs, using existing accelerators.
• Validate converted code for parity against SAS outputs (row counts, checksums, schemas, data types).
• Own deployment through Databricks Asset Bundles (DABs) and GitLab CI/CD.
• Manage Unity Catalog objects, permissions, and environment promotion.
• Troubleshoot job failures and performance issues.
• Communicate directly with client and partner stakeholders.

Skills

Python

SQL

Spark

Delta Lake

Data validation

Databricks Asset Bundles

Unity Catalog

GitLab CI/CD

Azure

Tools

Databricks

Azure

GitLab CI/CD

Terraform

Job description

Improving is committed to building a great place to work by cultivating an environment that fosters professional and personal relationships. We value open communication, personal growth, and shared rewards, which result in sustainable success.

Voted “best place to work” numerous times, Improving strives to create and maintain a culture that exemplifies teamwork, excellence, and fun! We believe this kind of culture encourages both the inspiration and the motivation to achieve amazing things.

• Join a team converting SAS code to Python and SQL on Databricks using generative AI, focusing on pipeline reliability, validation, and delivery
• Build and run the pipelines that process SAS inventories and converted outputs, working with existing conversion accelerators
• Validate converted code for parity against SAS outputs, including row counts, checksums, schemas, and data types
• Own deployment through Databricks Asset Bundles (DABs) and GitLab CI/CD
• Manage Unity Catalog objects, permissions, and environment promotion
• Troubleshoot job failures and performance issues
• Communicate directly with client and partner stakeholders

What You’ll Bring

• 4+ years of professional data or software engineering experience
• Strong Python and SQL skills
• Spark and Delta Lake performance tuning experience
• Data validation and reconciliation experience
• Hands‑on Databricks experience with Unity Catalog, Workflows, and Asset Bundles
• Infrastructure as code or DAB-based deployment experience
• GitLab CI/CD experience, including merge request workflows and automated testing
• Strong Git branching and code review discipline
• Clear written and spoken English for client‑facing collaboration
• A strong sense of ownership: you scope, deliver, and flag risks without being prompted
• Nice to have: ability to read SAS code, exposure to healthcare data, and Azure experience

Improving welcomes people from all backgrounds and walks of life, and this is reflected in our largely diverse community of Improvers. Improving is proud to be an equal opportunity employer and is committed to providing equal opportunities for all employees and applicants. Improving recruits, hires, trains, promotes, compensates, and administers all personnel actions without regard to race, color, religion, sex, sex stereotyping, pregnancy (which includes pregnancy, childbirth, and medical conditions related to pregnancy, childbirth, or breastfeeding), gender, gender identity, gender expression, national origin, age, mental or physical disability, ancestry, medical condition, marital status, military or veteran status, citizenship status, sexual orientation, genetic information, or any other status protected by applicable law.

About Improving

Improving is a modern digital services company dedicated to positively changing the perception of the IT professional. We offer innovative solutions through consulting, software development, and training to help thousands of our clients achieve new heights in a competitive and ever-changing market.

As our company continues to grow, we are looking for enthusiastic thought leaders to join our team. Improving has a unique mix of passionate professionals who strive to grow and thrive in new ways. We are committed to establishing and maintaining an inclusive culture that allows all Improvers to bring their authentic selves to work each day. This is why we work hard to build inclusion and diversity in our workplace, so we can all do amazing things and succeed together.