Overview From 31 December 2022 both the Museums of History NSW and State Records NSW are administering the State Records Act 1998 in partnership and implementing a range of improvements incorporated into the Act. This session is designed to provide an understanding of the recent changes in NSW and how the two organisations are implementing the legislation together. State Records Act 1998 The amended State Records Act aims to ensure that important government records are kept safe and accessible now and into the future. The changes introduce stronger regulatory powers and change access requirements. These include the authority to issue written notices requiring public offices to assess and report on their recordkeeping, with increased penalties for non-compliance. In addition, from 1 January 2024 State records will be available to the public sooner, defaulting to open access after 20 years. The open access changes apply not only to State archives that have been transferred into the control of Museums of History NSW but to all records currently held by NSW public offices. Public office stakeholders, which include all NSW Government departments, local government, and universities, are being widely engaged to increase awareness of the changes and promote best practice in recordkeeping.

The update of AI and Machine learning has the potential to have profound impacts on the work of archives, whether in our role as records regulators when decisions are no longer made and documented by human actors, or in managing ever-expanding, ever more digital and complex collections. In early 2023 National Archives undertook a research project to delve deeper into the implications of these technologies for our work. According to the Library of Congress in its report on AI: ‘The majority of machine learning experiments … stem from a simple reality: human time, attention, and labor will always be severely limited in proportion to the enormous collections we might wish to describe and catalog ” Previous research at NAA has highlighted some of the challenges and ethical issues in using machine learning to automate digital record management. Machine learning products can embed bias and cannot replace informed decision making. The main ethical consideration with ML is that it is generally developed with speed in favour of transparency in terms of how decisions are being made. Government records require the highest levels of trust and therefore due consideration needs to be given to any technology adopted to automate archival or recordkeeping decisions. The National Archives’ research project investigates use cases of these technologies against the archival context of accountability and openness to create a framework to evaluate both off-the-shelf and bespoke use of ML with digital records.

This presentation describes the outcomes of a research project undertaken by the National Archives of Australia that aimed to gather information from other government archives and select Australian government agencies about their approach to large-big datasets in the government sector. In particular the project aimed to address the following three key business problems identified by archival authorities and agencies: Information management challenges - applying fundamental information management practice and requirements, particularly disposal requirements, to large-big datasets. Transfer, preservation and access - the size and complexity of large-big datasets pose a challenge for transfer. The cost and complexity of preserving and making accessible these systems and the data they hold is currently outside of National Archives’ capabilities. In addition, it is likely that many large-big datasets will be required by the agency for ongoing business use and are therefore not suitable for transfer. Distributed custody - while distributed custody arrangements under section 64 of the Archives Act appear to be an appropriate response to the business problems described above, they have proved difficult to finalise. In addition, agencies have expressed concern that the conditions imposed by such agreements may prove too costly or difficult to implement. Access arrangements under section 64 agreements can also be difficult to articulate and implement; this is the case for existing agreements for analogue records, and may be substantially more difficult for large-big datasets, especially those containing personal or sensitive information. These business problems pose a challenge for government archives around the world, many of which have a role in information management in their government domains, and which provide guidance and advice to their government agency clients on ensuring the integrity and trustworthiness of data over time.