A new suite of AGS file utilities and public API

BGS has released an online AGS file utilities tool validator, offering the ability to validate AGS files against both the official AGS data format rules (v4.x) and BGS data submission requirements. It also to converts AGS data format files to and from MS Excel.

The AGS file utilities tool also has a public API for programmatic access. The tool accepts multiple AGS files, up to a 50MB total size limit.

AGS

AGS is a data format for sharing information about geotechnical projects, which includes boreholes, geological information, samples and test results. They are particularly useful for preliminary desk-based studies in the early stages of a project.

It’s a key data type that the industry is required to submit to the National Geoscience Data Centre (NGDC), which is managed by BGS to collect and preserve key geoscientific data, making it available in the long-term to a wide range of users and for the benefit of future geotechnical projects.

Previous tool

BGS had an existing AGS validation tool, which performed data checking against rules set out in the NGDC submission guidelines. However, we recognised there were some limitations and features we would like to add, such as AGS schema validation, and further data checks. 

AGS committee and Python library

The AGS Data Management Working Group recently released a AGS v4.x Python library, which offers AGS data format validation and file conversion to or from .ags or .xlsx. This is a fantastic resource for the AGS community and the first official AGS file validator.

On top of this library, the working group has also released a MS Windows validation tool. The AGS Validator (BETA) standalone software program was launched earlier this year by the Association of Geotechnical & Geoenvironmental Specialists, to provide a single source for validating electronic transfer of AGS4’ (AGS data format) files. This is available as a free download.

New tool

Recognising the potential of the Python library to benefit the geotechnical community and as part of our commitment to the AGS data format, wewanted to make a web-accessible tool that included an API for programmatic access. This came to be the AGS file utilities tool, which complements the release of AGS Validator (BETA). The BGS tool supports wider efforts by the industry to standardise the validation process and reduce additional work for users, contractors and software providers.

The AGS file utilities tool validator provides the capability to validate AGS files against both the official AGS data format rules (v4.x) and BGS data submission requirements. It also converts AGS data format files both to and from MS Excel format.

Development was undertaken over 20 days, with three of BGS’s software developers working on the project.  A key feature for us was to make a public API available to allow others in the community to build their own applications or web pages that could use our tool to perform AGS validation or conversion, rather than having to host a server themselves, hopefully facilitating further use and uptake of AGS data format. 

Open source

Another requirement we had was for the application to be free, open source and released under an open license. This aligns with the BGS Digital Strategy and the wider Government digital service standard, the value of which we recognise for the wider community. The source code for the application can be found on the BGS GitHub site.

Future work

If there is sufficient interest from the AGS community, we hope to continue to develop this tool and add new features in line with further development of the underlying AGS Python library.

About the author

Edd Lewis

Data standards lead

BGS Keyworth

Edd Lewis is the Standards Lead at the British Geological Survey, with responsibility for improving the organisation’s data provision through using geospatial data standards to better align with FAIR data principles (and hopefully driving economic growth through improved data accessibility). He is a member of the British Standard Institute IST/36 Geographic Information committee, MEDIN standards committee, Association of Geotechnical and Geoenvironmental Specialists (AGS) AGS Data Format committee and contributor to ISO/TC 211 & OGC standards.

Find out more