BGS news

Update released for BGS open-source database software, ETL Helper

The software tool helps simplify the process of data transfer into and out of databases.

09/07/2024 By BGS Press
GitHub logo
Through the BGS GitHub page, projects from across the organisation have provided access to innovative tools, documentation, and even training materials. Source: GitHub

BGS’s ETL Helper software tool is a package for the Python programming language. Version 1 includes driver updates and usability improvements; however, the most significant change is the addition of a documentation website. This will make the software more accessible and expand the pool of users and potential contributors.

Since ETL Helper was released as open-source software in 2019, it has picked up a global user base, with users from Columbia to Kenya, India to China as well as North America and Europe. It is downloaded over 25 000 times per month and has received code contributions from as far away as Australia.

Speech marks icon

ETL Helper was created because common proprietary data-transfer tools were difficult to use in a team setting. I always thought that the data-transfer tasks we were doing should be simple to write in Python. However, once you add in database connection setup, error handling and memory management, the reality was more complicated. ETL Helper takes care of this and allows us to transfer the data using only a few lines of code.

Dr John Stevenson, BGS Senior Software Developer and lead developer of ETL Helper.

Example code on ETL helper
Information icon

Code on ETL Helper. BGS © UKRI.

Expand icon

Open-source software allows users to inspect how the software works and modify it to suit their needs. In a scientific context, using open-source software allows collaborative software development with partners and the wider community. ETL Helper is one of the many software outputs that are released by BGS under open-source licences. Others include utilities for plotting maps of volcanic of ash clouds, validating borehole log data and selecting target points on samples for microanalysis.

ETL Helper and other BGS open-source software can be explored and downloaded on the GitHub platform.

Was this page helpful?

  • How can we make this section better?*

  • Please select a reason*

  • How can we make this section better?*