A new suite of AGS file utilities and public API
BGS has released an online AGS file utilities tool validator.
24/11/2021![Edd Lewis presenting at the Annual AGS Conference](https://www.bgs.ac.uk/wp-content/uploads/2021/11/ags_Edd_Lewis.jpg)
BGS has released an online AGS file utilities tool validator, offering the ability to validate AGS files against both the official AGS data format rules (v4.x) and BGS data submission requirements. It also to converts AGS data format files to and from MS Excel.
The AGS file utilities tool also has a public API for programmatic access. The tool accepts multiple AGS files, up to a 50MB total size limit.
AGS
AGS is a data format for sharing information about geotechnical projects, which includes boreholes, geological information, samples and test results. They are particularly useful for preliminary desk-based studies in the early stages of a project.
It’s a key data type that the industry is required to submit to the National Geoscience Data Centre (NGDC), which is managed by BGS to collect and preserve key geoscientific data, making it available in the long-term to a wide range of users and for the benefit of future geotechnical projects.
Previous tool
BGS had an existing AGS validation tool, which performed data checking against rules set out in the NGDC submission guidelines. However, we recognised there were some limitations and features we would like to add, such as AGS schema validation, and further data checks.
AGS committee and Python library
The AGS Data Management Working Group recently released a AGS v4.x Python library, which offers AGS data format validation and file conversion to or from .ags or .xlsx. This is a fantastic resource for the AGS community and the first official AGS file validator.
On top of this library, the working group has also released a MS Windows validation tool. The AGS Validator (BETA) standalone software program was launched earlier this year by the Association of Geotechnical & Geoenvironmental Specialists, to provide a single source for validating electronic transfer of AGS4’ (AGS data format) files. This is available as a free download.
New tool
Recognising the potential of the Python library to benefit the geotechnical community and as part of our commitment to the AGS data format, wewanted to make a web-accessible tool that included an API for programmatic access. This came to be the AGS file utilities tool, which complements the release of AGS Validator (BETA). The BGS tool supports wider efforts by the industry to standardise the validation process and reduce additional work for users, contractors and software providers.
The AGS file utilities tool validator provides the capability to validate AGS files against both the official AGS data format rules (v4.x) and BGS data submission requirements. It also converts AGS data format files both to and from MS Excel format.
Development was undertaken over 20 days, with three of BGS’s software developers working on the project. A key feature for us was to make a public API available to allow others in the community to build their own applications or web pages that could use our tool to perform AGS validation or conversion, rather than having to host a server themselves, hopefully facilitating further use and uptake of AGS data format.
Open source
Another requirement we had was for the application to be free, open source and released under an open license. This aligns with the BGS Digital Strategy and the wider Government digital service standard, the value of which we recognise for the wider community. The source code for the application can be found on the BGS GitHub site.
Future work
If there is sufficient interest from the AGS community, we hope to continue to develop this tool and add new features in line with further development of the underlying AGS Python library.
About the author
![Edd Lewis](https://www.bgs.ac.uk/wp-content/uploads/2023/02/Edd_Lewis_Colour.jpg)
Edd Lewis
Data Standards Lead
Edd Lewis is the Standards Lead at the British Geological Survey, with responsibility for improving the organisation’s data provision through using geospatial data standards to better align with FAIR data principles (and hopefully driving economic growth through improved data accessibility). He is a member of the British Standard Institute IST/36 Geographic Information committee, MEDIN standards committee, Association of Geotechnical and Geoenvironmental Specialists (AGS) AGS Data Format committee and contributor to ISO/TC 211 & OGC standards.
Find out moreRelative topics
Related news
![North Sea Transition Authority](https://www.bgs.ac.uk/wp-content/uploads/2024/07/NSTA_image.jpg)
National Hydrocarbons Data Archive files now accessible through the National Data Repository
15/07/2024
Offshore well and survey information held within BGS’s archive have now been transferred to the North Sea Transition Authority’s National Data Repository to enable greater access.
![GitHub logo](https://www.bgs.ac.uk/wp-content/uploads/2021/12/GitHub_logo.jpg)
Update released for BGS open-source database software, ETL Helper
09/07/2024
The software tool helps simplify the process of data transfer into and out of databases.
![Screenshot of the AGS4 file utilities tool and API with map markers over the UK.](https://www.bgs.ac.uk/wp-content/uploads/2024/05/ags4-file-utilities-and-api.webp)
Update to BGS’s AGS file utilities tool released
19/06/2024
BGS has released significant updates to its online tool for validating geotechnical data held as AGS files. The update includes improvements to the tool’s user interface and the ability to convert AGS data into MS Excel files.
![Technical reports released by BGS. BGS © UKRI](https://www.bgs.ac.uk/wp-content/uploads/2024/06/DSSS8523-resized.webp)
BGS’s National Geoscience Data Centre releases over 8000 technical reports
05/06/2024
The technical reports, covering the full spectrum of BGS activities and subjects, were produced between 1950 and 2000.
![Map showing rock type and lithology](https://www.bgs.ac.uk/wp-content/uploads/2024/05/data-from-bgs-geology-625k.webp)
BGS announces collaboration with Ordnance Survey
29/05/2024
Ordnance Survey and BGS have teamed up to add a range of geological data products to the OS Data Hub, making it easier for users to access and use geospatial data.
![Debris flow susceptibility model GlenCoe. Contains Ordnance Survey data © Crown Copyright and database rights 2022.](https://www.bgs.ac.uk/wp-content/uploads/2022/07/Debris-Flow-Susceptibility-Model_GlenCoe.jpg)
BGS data product licence fees to rise
01/03/2024
New direct licence fees to be introduced from 1 April 2024.
![Subsidence warning notice, Arvalee © Kenneth Allen cc-by-sa/2.0. Geograph Britain and Ireland](https://www.bgs.ac.uk/wp-content/uploads/2023/08/geograph-6428042-by-Kenneth-Allen.jpg)
Evolving policies in the face of climate change: BGS data can help
25/08/2023
Climate change is increasingly recognised as a major challenge for organisations, with the need for adaptation driving a wave of policy updates and reforms across multiple sectors. BGS data holdings are relevant to many of these changes and the following examples outline how these data packages can be applied.
![Contains OS data © Crown Copyright and database right 2020](https://www.bgs.ac.uk/wp-content/uploads/2023/06/GeoIndex_NSTA.jpg)
A new open dataset to benefit onshore geoscience research
15/06/2023
Data from deep onshore hydrocarbon wells is being released on an open access basis to help meet the UK’s net zero targets.
![landscape-devon](https://www.bgs.ac.uk/wp-content/uploads/2022/11/landscape-devon-960x645.jpg)
New report on the National Land Data Programme published
06/06/2023
The report, based on pilot projects across the UK, explains how spatial data and modelling can support better decisions on land use.
![A woman carrying a notebook examines a piece of rock core](https://www.bgs.ac.uk/wp-content/uploads/2023/05/P994447-1-960x640.jpg)
AGS file utilities tool: production release
16/05/2023
BGS’s online tool for validating AGS files against the official AGS data format rules and its own NGDC data submission requirements goes live.
![Argyllshire sheet 84, solid and drift, 1:10 560, 1914. BGS © UKRI](https://www.bgs.ac.uk/wp-content/uploads/2023/05/13599901_web-1.jpg)
BGS releases 10K maps through updated maps portal
16/05/2023
BGS releases an updated maps portal, which allows users to view all publicly available maps, and includes 10 000 and 1:10 560 scale maps for the first time.
![BGS' Edd Lewis, presenting BGS' new API release at the ASGS Geotech conference, April 2023. Photo: Ricky Terrington BGS](https://www.bgs.ac.uk/wp-content/uploads/2023/04/API_image.jpg)
Improvements to borehole delivery
25/04/2023
BGS releases improvements to its borehole layer on the GeoIndex, which makes access faster and improves quality.