AI and Heritage Research: how reliable is it?
AI has become a popular and useful tool for research on just about any topic. So, it might seem like the perfect tool for researching the history of your home: but can it really be relied on?
For those requiring heritage research and building analysis for a planning application, along with the rise in popularity of period dramas, historical fiction, and genealogical research, the appeal of discovering the rich history of our homes, and any potential hidden treasures, has never been greater, especially those who live in historic or listed buildings.
At ArchiWest we dedicate a lot of time to researching the history of homes and heritage buildings for a wide variety of clients, and consider ourselves specialists in the field. As AI is rapidly making regular search engines a thing of the past, we ask the question ‘can you really trust it when doing an in-depth study?’ The short answer is; occasionally.
When an AI model is created, it is given a set of information or data to learn from, it is then prompted to answer questions or complete tasks and those responses are graded, the process is then repeated over and over again. The intention is, by the time the AI reaches you, it can accurately answer any question you ask.
However, what is described as ‘AI’ is not actually artificial intelligence, but rather a Large Language Model or LLM. Essentially, a model that has been given so much information that it can simulate and understand natural language and a wide range of tasks. It can’t ‘think’ for itself, but it can ‘understand’ nuance and reasoning much better than ever before. LLM’s are trained on a given set of data, meaning the creators of that specific model decided what it does and does not ‘know’. They also decided what it should and should not answer. Therefore, it can rarely be considered an unbiased source.
As a product, most publicly or commercially available LLM’s are designed to make you want to keep using them, and part of the way they do this is to always have the answer they ‘think’ you want, even if not quite correct. This is often referred to as a ‘hallucinated answer’, and occurs when the AI cannot find an answer to your question but is ‘obliged’ to give a response.
Rather than cite a singular historical primary source, LLM’s will amalgamate all the answers to a question that it can find, offering up the most popular conclusion. However, secondary sources are often repeated over and over in different works, primarily because it is much easier to reference someone else’s work than find the original physical historic source – a task beyond the capabilities of an LLM if the physical source has not been digitalised. In our experience, LLM's often regurgitate information from unreliable sources, such a blog posts, social media posts, and obscure websites. An anecdotal real world example of this is when an LLM offered up an answer that had been sourced from a historic building group Facebook post, that had itself been AI generated. No actual evidence for the validity of the information could be ascertained, and therefore, the information had to be dismissed. With the dramatic increase in the number of AI generated articles and pages being posted online daily, this is likely to become more and more misleading and untrustworthy. Even if sources are quoted, these could just be more artificially generated content, creating a loop of inaccurate information.
Just as film makers are not obliged to portray historic events with accuracy, an artistic licence is an accepted element of media creativity, LLM's have the capacity for misleading unassuming researchers in their understanding of historical events, which should be openly recognised. In practice, relying on LLM's to produce an accurate heritage statement or historic building assessment, for anything more than a high level overview could be detrimental to a planning application if the information is found to be incorrect, and it is clear to see how this could and does happen.
However, this does not mean that LLM's cannot be useful, particularly in guiding you toward correct sources like Wikipedia, but it is important to check that the sources are reliable and to be aware of potential biases or ideologies. In our experience, the most helpful feature of LLM's is the ability to sort through architectural or historical source data more swiftly and point us in the right direction. We check information sources thoroughly, use multiple aspects of desk-based assessment and cross referencing, and visit physical archives wherever possible. Proper historical research takes time and effort, but often reveals fascinating events of a by gone age and is incredibly rewarding.
studio@ArchiWest.co.uk - 01934 311017
To find out more about heritage research services, or how we can assist with your project, get in touch with one of the team for an informal chat with a real live person about your requirements.
Comments