A new suite of AGS file utilities and public API
BGS has released an online AGS file utilities tool validator.
24/11/2021BGS has released an online AGS file utilities tool validator, offering the ability to validate AGS files against both the official AGS data format rules (v4.x) and BGS data submission requirements. It also to converts AGS data format files to and from MS Excel.
The AGS file utilities tool also has a public API for programmatic access. The tool accepts multiple AGS files, up to a 50MB total size limit.
AGS
AGS is a data format for sharing information about geotechnical projects, which includes boreholes, geological information, samples and test results. They are particularly useful for preliminary desk-based studies in the early stages of a project.
It’s a key data type that the industry is required to submit to the National Geoscience Data Centre (NGDC), which is managed by BGS to collect and preserve key geoscientific data, making it available in the long-term to a wide range of users and for the benefit of future geotechnical projects.
Previous tool
BGS had an existing AGS validation tool, which performed data checking against rules set out in the NGDC submission guidelines. However, we recognised there were some limitations and features we would like to add, such as AGS schema validation, and further data checks.
AGS committee and Python library
The AGS Data Management Working Group recently released a AGS v4.x Python library, which offers AGS data format validation and file conversion to or from .ags or .xlsx. This is a fantastic resource for the AGS community and the first official AGS file validator.
On top of this library, the working group has also released a MS Windows validation tool. The AGS Validator (BETA) standalone software program was launched earlier this year by the Association of Geotechnical & Geoenvironmental Specialists, to provide a single source for validating electronic transfer of AGS4’ (AGS data format) files. This is available as a free download.
New tool
Recognising the potential of the Python library to benefit the geotechnical community and as part of our commitment to the AGS data format, wewanted to make a web-accessible tool that included an API for programmatic access. This came to be the AGS file utilities tool, which complements the release of AGS Validator (BETA). The BGS tool supports wider efforts by the industry to standardise the validation process and reduce additional work for users, contractors and software providers.
The AGS file utilities tool validator provides the capability to validate AGS files against both the official AGS data format rules (v4.x) and BGS data submission requirements. It also converts AGS data format files both to and from MS Excel format.
Development was undertaken over 20 days, with three of BGS’s software developers working on the project. A key feature for us was to make a public API available to allow others in the community to build their own applications or web pages that could use our tool to perform AGS validation or conversion, rather than having to host a server themselves, hopefully facilitating further use and uptake of AGS data format.
Open source
Another requirement we had was for the application to be free, open source and released under an open license. This aligns with the BGS Digital Strategy and the wider Government digital service standard, the value of which we recognise for the wider community. The source code for the application can be found on the BGS GitHub site.
Future work
If there is sufficient interest from the AGS community, we hope to continue to develop this tool and add new features in line with further development of the underlying AGS Python library.
About the author
Edd Lewis
Data Standards Lead
Edd Lewis is the Standards Lead at the British Geological Survey, with responsibility for improving the organisation’s data provision through using geospatial data standards to better align with FAIR data principles (and hopefully driving economic growth through improved data accessibility). He is a member of the British Standard Institute IST/36 Geographic Information committee, MEDIN standards committee, Association of Geotechnical and Geoenvironmental Specialists (AGS) AGS Data Format committee and contributor to ISO/TC 211 & OGC standards.
Find out moreRelative topics
Related news
BGS data product licence fees to rise
01/03/2024
New direct licence fees to be introduced from 1 April 2024.
Evolving policies in the face of climate change: BGS data can help
25/08/2023
Climate change is increasingly recognised as a major challenge for organisations, with the need for adaptation driving a wave of policy updates and reforms across multiple sectors. BGS data holdings are relevant to many of these changes and the following examples outline how these data packages can be applied.
A new open dataset to benefit onshore geoscience research
15/06/2023
Data from deep onshore hydrocarbon wells is being released on an open access basis to help meet the UK’s net zero targets.
New report on the National Land Data Programme published
06/06/2023
The report, based on pilot projects across the UK, explains how spatial data and modelling can support better decisions on land use.
AGS file utilities tool: production release
16/05/2023
BGS’s online tool for validating AGS files against the official AGS data format rules and its own NGDC data submission requirements goes live.
BGS releases 10K maps through updated maps portal
16/05/2023
BGS releases an updated maps portal, which allows users to view all publicly available maps, and includes 10 000 and 1:10 560 scale maps for the first time.
Improvements to borehole delivery
25/04/2023
BGS releases improvements to its borehole layer on the GeoIndex, which makes access faster and improves quality.
Six BGS datasets for assessing shrink–swell subsidence hazards
17/02/2023
Shrink–swell subsidence is one of the most significant geological hazards affecting the UK. BGS has six datasets to help assess the problem.
Introducing the BGS Debris Flow Susceptibility Model for Great Britain
21/11/2022
Debris flows are a landslide hazard of particular concern to transport infrastructure managers and local authorities.
BGS GeoScour: bridging the river erosion data gap
31/08/2022
The geological properties of bedrock and superficial deposits that make up riverbeds and banks are fundamental controls on the susceptibility of any given river reach to scouring.
BGS joins the UK Centre for Seabed Mapping to raise data and mapping standards
25/08/2022
BGS is the latest UK Government organisation to join the newly unveiled UK Centre for Seabed Mapping, helping to boost marine data and mapping standards.
River erosion: the forgotten hazard of flooding
03/08/2022
Impacts from flood events can be widespread, long-lasting and extremely costly. The UK Government and environmental protection agencies continue to invest heavily in mitigation measures, as well as trying to predict which areas are most at risk.