UK- Generative AI framework for HMG

UK- Generative Framework for HMG

On the 18th January 2024, the UK government published the Generative framework for HMG. This framework provides guidance on Generative AI solutions and establishes a range of principles to ensure it is used safely.

This framework has been left incomplete and dynamic, due to the fact that the field of  Generative AI is still rapidly growing. This will be developed as and when the field of Generative AI expands.

Principles-

This paper expands on the 5 principles stated in the UK- Pro-Innovation approach to AI.  The framework creates 10 core principles for Generative AI use in government and public sector organisations.

Principle 1: You know what generative AI is and what its limitations are

Principle 2: You use generative AI lawfully, ethically and responsibly

Principle 3: You know how to keep generative AI tools secure

Principle 4: You have meaningful human control at the right stage

Principle 5: You understand how to manage the full generative AI lifecycle

Principle 6: You use the right tool for the job

Principle 7: You are open and collaborative

Principle 8: You work with commercial colleagues from the start

Principle 9: You have the skills and expertise needed to build and use generative AI

Principle 10: You use these principles alongside your organisation’s policies and have the right assurance in place

Generative AI- Definition

Generative AI is a form of Artificial Intelligence (AI) – a broad field which aims to use computers to emulate the products of human intelligence – or to build capabilities which go beyond human intelligence.

The paper notes specific types of Generative AI. These include foundation models, and Large Language Models (LLM’s)

Foundation models are large neural networks trained on large data sets to produce responses which resemble those data sets. These can be language based or non-text data.

Large Language Models (LLM’s)are foundation models specifically trained on language data and texts.

Application of Generative AI in Government

Their ability to process and produce language is highly useful in Government work. The paper lists specific uses within the Government such as;  the ability to perform complicated tasks and reduce staff workload.

Limitations of Generative AI

LLM’s predict the next word in a sequence and are primarily unaware of the content before them. Because of this, there are many limitations accompanied with this type of Gnerative AI:

Some of these include:

  1. Critical thinking and judgement: Although LLMs can give the appearance of reasoning, they are simply predicting the next most plausible word in their output, and may produce inaccurate or poorly-reasoned conclusions.
  2. Short-term memory: LLMs have a limited context window. They might lose track of the context of a conversation if it’s too long, leading to incoherent responses.
  3. Personal experience and context: LLMs lack personal experiences and emotions. Although their outputs may appear as if they come from a person, they do not have true understanding or a consciousness.

Building Generative AI solutions-

This section outlines the steps which must be taken when building a Generative AI solutions. The section outlines the following steps:

  1. Defining the goal-
  2. Identifying use cases- Must identify possible use cases led by business needs and individual needs. Focus on cases in which Generative AI holds significant possibilities. The framework outlines 8 possible use cases which are the most promising
  3. Use cases to avoid- The framework outlines many cases in which Generative AI is not appropriate. Some examples provided include; High explainability contexts, high risk applications and fully automated decision making.
  4. Building the team- you should ensure you have a multi-disciplinary team in place who have the capability to operate the solution.
  5. Acquiring Skill- The framework identifies a range of open learning resources to help acquire specific skills to work in AI.
  6. Creating the Generative AI support structure- this includes the following aspects;
  • AI strategy and adoption plan
  • AI principles
  • AI governance board
  • Communication board
  • AI sourcing and partnership strategy

Building the solution- This framework sets out key concept you must understand in order to build Generative AI solutions:

  1. Prompts are the primary input provided to an LLM. In the most simple case, a prompt may only be the user-prompt. In production systems, a prompt will have additional parts, such as meta-prompts, the chat history, and reference data to support explainability.
  2. Prompt engineering describes the process of adjusting LLM input to improve performance and accuracy. In its simplest form it may be testing different user-prompt formulations. In production systems, it will include adjustments, such as adding meta-prompts, provision of examples and data sources, and sometimes parameter tuning.
  3. User-prompts are whatever you type into e.g. a chat box. They are generally in the everyday natural language you use, e.g. ‘Write a summary of the generative AI framework’.
  4. Meta-prompts (also known as system prompts) are higher-level instructions that help direct an LLM to respond in a specific way. They can be used to instruct the model on how to generate responses to user-prompts, provide feedback, or handle certain types of content.
  5. Embedding is the process of transforming information such as words, or images into numerical values and relationships that the computer algorithms can understand and manipulate. Embeddings are typically stored in vector databases (see below).
  6. Retrieval Augmentation Generation (RAG) is a technique which uses reference data stored in vector databases (i.e. the embeddings) to ground a model’s answers to a user’s prompt. You could specify that the model cites its sources when returning information.
  7. Vector databases index and store data such as text in an indexed format easily searchable by models. The ability to store and efficiently retrieve information has been a key enabler in the progress of generative AI technology.
  8. Grounding is the process of linking the representations learned by the AI models to real-world entities or concepts. It is essential for making AI models understand and relate its learned information to real-world concepts. In the context of large language models, grounding is often achieved by a combination of prompt engineering, parameter tuning, and RAG.
  9. Chat history is a collection of prompts and responses. It is limited to a session. Different models may allow different session sizes. For example, Bing search sessions allow up to 30 user-prompts. The chat history is the memory of LLMs. Outside of the chat history LLMs are “stateless”. That means the model itself does not store chat history. If you wanted to permanently add information to a model you would need to fine-tune an existing model (or train one from scratch).
  10. Parameter tuning is the process of optimising the performance of the AI model for a specific task or data set by adjusting configuration settings.
  11. Model fine-tuning is the process of limited re-training of a model on new data. It can be done to enforce a desired behaviour. It also allows us to add data sets to a model permanently. Typically, fine-tuning will adjust only some layers of the model’s neural network. Depending on the information or behaviour to be trained, fine-tuning may be more expensive and complicated than prompt engineering. Experience with model tuning in HMG is currently limited and we are looking to expand on this topic in a future iteration of this framework.
  12. Open source models are publicly accessible, and their source code, architecture, and parameters are available for examination and modification by the broader community.
  13. Closed source models on the other hand, are proprietary and not openly accessible to the public. The inner workings and details of these models are kept confidential and are not shared openly.

Using Generative AI safely

When building Generative AI solutions, you must take into account the following considerations:

  1. Legal considerations
  2. Ethics
  3. Data protection and privacy
  4. Security and governance

The paper then expands on each of these considerations individually.

Security

The following security recommendations have been outlined within the framework:

  1. Design risk-driven security taking account of the OWASP Top 10 security risks for LLMs.
  2. Use a consistent risk rating methodology to assess the impact and likelihood of each risk.
  3. Minimise the attack surface by only using the required capabilities of the generative AI tool, for example, by avoiding sending user input directly to an LLM.
  4. Defend in depth by adding layers of security, for example, by using privacy enhancing technology (PET) to prevent data leakage and adding content filters to sanitise the prompts and responses from an LLM.
  5. Never use private data that needs different levels of access permissions based on the user who is viewing it, to fine-tune or train an LLM.
  6. Prevent LLM responses automatically leading to destructive or irreversible actions, such as sending emails or modifying records. In these situations a human must be present to review the action.
  7. Avoid using links to external resources in LLM responses that will be read by humans, and if external links are provided then the response must be filtered to remove malicious URLs.
  8. Treat all LLM generated code as inherently insecure and never use it directly in production without code review.
  9. Never enter any OFFICIAL information directly into public generative AI applications or APIs, unless it is already publicly available or cleared for publication. Exceptions may apply for specific applications with different data handling terms provided under commercial licences, for example, Microsoft Copilot, Azure Open AI, or Bing Enterprise Chat.
  10. Avoid putting LLM chatbots on public facing government websites, unless the risk of direct prompt injection is acceptable under the specific use case

Governance

Due to the risks surrounding security and data protection. Organisations should consider setting up an AI governance board as well as an ethics committee. As well as this an artificial intelligence and machine learning systems inventory to provide an  overview of all deployed AI systems.