AGS file utilities tool: production release

In late 2021, BGS released the beta service of the AGS file utilities tool. We are pleased to announce the application has been updated and is now ready for production use. The online tool is easy and free to use, offering the ability to validate AGS files against both the official AGS data format rules (v4.x) and the National Geoscience Data Centre (NGDC) data submission requirements. It also converts AGS data format files to and from MS Excel format.

This work has been made possible by the efforts of the AGS Data Format Working Group, who have updated the underlying official AGS Python library. As we use the same library, the AGS rules are checked and the response is identical to the official AGS validator.

How to use the AGS file utilities tool

The application offers two ways to interact with the available tools: a web-based graphical user interface (GUI) and an application programming interface (API) for programmatic access. Files are not saved or stored by the AGS file utilities tool.

Web GUI

To validate an AGS file you need to specify three parameters:

• which AGS version to use or the version specified in the file
• which validation to run: AGS rules, BGS submission requirements or both
• how you would prefer the results: HTML, plain text or JSON

You can then select and upload one or more AGS files (50 Mb maximum) for validation (Figure 1). To convert files, select and upload either AGS files (.ags) or MS Excel files (.xlsx) files and the other file type will be returned. There is an option to sort the output worksheets alphabetically, but doing this means the original group order will be lost.

API

We have also chosen to make an API available, which offers the same validation and conversion functions for users who would like to integrate the tools into their data science workflows or other applications. Swagger UI documentation is available (Figure 2).

AGS v4.x data

The AGS data format standard is the file format used for reliable data transfer (project metadata, boreholes, geological information, samples and test results) between stakeholders in the geotechnical domain. Geotechnical data is of vital importance not only to BGS but also to UK PLC, to gain a greater understanding of the subsurface for planning, engineering and modelling. It is the recommended file type for submission of geotechnical data to the NGDC, which is managed by BGS, to collect and preserve key geoscientific data, and make it available in the long-term to a wide range of users and for the benefit of future geotechnical projects.

AGS data validation

In 2021, the AGS Data Management Working Group released an AGS v4.x Python library, which offers AGS data format validation for AGS version 4.x and file conversion between .ags and .xlsx. This is a fantastic resource for the AGS community and offered the first official AGS file validation capability.

This library is released under an open license allowing others, including BGS, to incorporate the features offered in new applications.

AGS v3.x

For users who are creating or discovering AGS v3 files that they would like to submit to the NGDC, we continue to host our legacy AGS validation tool. This performs data checking against rules set out in the NGDC submission guidelines.

Open source

In line with UKRI, the BGS Digital Strategy and the wider Government Technology Code of Practice and Digital Service Standard, the source code for the AGS file utilities tool application is available under open license and hosted on the BGS GitHub.

Future work

We hope to continue to develop this tool and add new features in line with further development of the underlying AGS Python library.

We welcome feedback from the user community. Please raise any feature requests or issues on GitHub repository issues board.

We’ve also been making improvements to the borehole records data delivery and AGS graphical logs in line with this work.

About the author

Edd Lewis

Data Standards Lead

BGS Keyworth

Edd Lewis is the Standards Lead at the British Geological Survey, with responsibility for improving the organisation’s data provision through using geospatial data standards to better align with FAIR data principles (and hopefully driving economic growth through improved data accessibility). He is a member of the British Standard Institute IST/36 Geographic Information committee, MEDIN standards committee, Association of Geotechnical and Geoenvironmental Specialists (AGS) AGS Data Format committee and contributor to ISO/TC 211 & OGC standards.

Find out more