Policy documents are often technical and complex and it can take time to process what we are reading. Even more so when we are not working with these documents on a daily basis. At the same time,  they have a key function for decision-making processes. They determine what roles different stakeholders should play and what kind of knowledge is expected to inform the decisions we make. We can think of such prescribed exchanges of knowledge as "knowledge flows".

After following a workshop on the use of AI in research, I started to think about ways to incorporate AI in my project. I realized knowledge flows represent certain patterns in policy texts and may be recognized by an AI given enough context. Thus, I decided to start with a first full manual reading of some documents to determine typical patterns. I found typical "signal words" used to describe knowledge exchange (e.g., report to, consult with), and compiled a first set of examples of knowledge flows. Importantly, I found both complete descriptions of knowledge flows, as well as incomplete (only a sender / receiver specified). 

With that first material, I began working with ChatGPT. I started asking it to analyze articles or sections within a document and provide summaries and/or overviews of identified knowledge flows. This was a very interesting process where I had to find ways to optimize the prompt (set of instructions) to gradually get better output. I also had to find ways to try and correct ChatGPT because it can and does make mistakes. I believe the human in the loop is and will remain necessary. However, to gradually see the output improve and to find ways to correct it and get the results I needed was highly rewarding!

While I applied this analysis mainly on policies focusing on environmental management and wastewater management, this approach can be helpful to a larger audience as well. It enables us to get a better overview of policies, even when we are not especially knowledgeable about them. It can help us to understand intended roles, or to find out what contribution different stakeholders are supposed to give. Moreover, it can help us to point at potential weaknesses such as unclear roles or overly general provisions on stakeholder engagement. Thus, this can be a tool for both policy makers to strengthen existing policies and for other stakeholders that would like to explore ways to have their interests better reflected in decision-making.

If you are interested in my method, the prompt  or the results this kind of analysis can provide then have a look at my paper (open access). There, I present the results from my study in Indonesia, where I looked at the island of Gili Trawangan (Northwest Lombok). And perhaps you would like to analyze a policy document yourself? Then you could also try my custom GPT, the "knowledge flow identifier". I have recently tested it on other policy documents as well, also in another language than English (Dutch), with satisfactory results overall.

Feel free to reach out with any question you have!