Update released for BGS open-source database software, ETL Helper
The software tool helps simplify the process of data transfer into and out of databases.
09/07/2024 By BGS Press
BGS’s ETL Helper software tool is a package for the Python programming language. Version 1 includes driver updates and usability improvements; however, the most significant change is the addition of a documentation website. This will make the software more accessible and expand the pool of users and potential contributors.
Since ETL Helper was released as open-source software in 2019, it has picked up a global user base, with users from Columbia to Kenya, India to China as well as North America and Europe. It is downloaded over 25 000 times per month and has received code contributions from as far away as Australia.
ETL Helper was created because common proprietary data-transfer tools were difficult to use in a team setting. I always thought that the data-transfer tasks we were doing should be simple to write in Python. However, once you add in database connection setup, error handling and memory management, the reality was more complicated. ETL Helper takes care of this and allows us to transfer the data using only a few lines of code.
Dr John Stevenson, BGS Senior Software Developer and lead developer of ETL Helper.
Code on ETL Helper. BGS © UKRI.
Open-source software allows users to inspect how the software works and modify it to suit their needs. In a scientific context, using open-source software allows collaborative software development with partners and the wider community. ETL Helper is one of the many software outputs that are released by BGS under open-source licences. Others include utilities for plotting maps of volcanic of ash clouds, validating borehole log data and selecting target points on samples for microanalysis.
ETL Helper and other BGS open-source software can be explored and downloaded on the GitHub platform.
Relative topics
Related news
What lies beneath Liverpool?
11/10/2024
The geological secrets lying under the surface of Liverpool and Warrington have been unveiled for the first time on a new 3D interactive tool.
Seabed geology data: stakeholder consultation
25/09/2024
BGS is seeking stakeholder feedback as part of a new Crown Estate-led initiative to improve understanding of national-scale seabed geology requirements
BGS Groundwater Flooding Susceptibility: helping mitigate one of the UK’s most costly hazards
25/09/2024
Groundwater flooding accounts for an estimated £530 million in damages per year; geoscientific data can help to minimise its impact.
National Hydrocarbons Data Archive files now accessible through the National Data Repository
15/07/2024
Offshore well and survey information held within BGS’s archive have now been transferred to the North Sea Transition Authority’s National Data Repository to enable greater access.
Update released for BGS open-source database software, ETL Helper
09/07/2024
The software tool helps simplify the process of data transfer into and out of databases.
Update to BGS’s AGS file utilities tool released
19/06/2024
BGS has released significant updates to its online tool for validating geotechnical data held as AGS files. The update includes improvements to the tool’s user interface and the ability to convert AGS data into MS Excel files.
BGS’s National Geoscience Data Centre releases over 8000 technical reports
05/06/2024
The technical reports, covering the full spectrum of BGS activities and subjects, were produced between 1950 and 2000.
BGS announces collaboration with Ordnance Survey
29/05/2024
Ordnance Survey and BGS have teamed up to add a range of geological data products to the OS Data Hub, making it easier for users to access and use geospatial data.
BGS data product licence fees to rise
01/03/2024
New direct licence fees to be introduced from 1 April 2024.
Evolving policies in the face of climate change: BGS data can help
25/08/2023
Climate change is increasingly recognised as a major challenge for organisations, with the need for adaptation driving a wave of policy updates and reforms across multiple sectors. BGS data holdings are relevant to many of these changes and the following examples outline how these data packages can be applied.
A new open dataset to benefit onshore geoscience research
15/06/2023
Data from deep onshore hydrocarbon wells is being released on an open access basis to help meet the UK’s net zero targets.
New report on the National Land Data Programme published
06/06/2023
The report, based on pilot projects across the UK, explains how spatial data and modelling can support better decisions on land use.