This is a work in progress by the Service Innovation Lab. Pull requests are welcome on GitHub.

Machine Learning Introduction

As a policy analyst, you might be able to use machine learning to help you process qualitative data, but it's probably not going to replace manual analysis any time soon.

In this page:

  1. Why you should use it
  2. What it involves
  3. Ideal circumstances for use
  4. Limitations
  5. Machine learning can be time-consuming, data-hungry, and energy-hungry
  6. Machine learning is difficult to inspect
  7. Machine learning doesn't erase bias

Why you should use it

Policy consultation undertaken with modern technology can reach thousands and result in a high volume of submissions. While this is fantastic for getting a broad range of feedback, it can be difficult to process this amount of data.

Some policy teams are reaching out to machine learning to solve this, but it’s easy to get bamboozled; the jargon is complex and some vendors are happy for you to believe that it’s magic and will solve all of your problems. It is not magic, and it may well cause you new problems!

Some examples of where machine learning could be useful:

This post is a primer on machine learning to help you figure out whether it would be useful for you and your team, and how you might use it ethically and effectively.

What it involves

Machine learning is a blanket term for algorithms that learn from the past to predict future data. Machine learning algorithms generally fall into three categories: supervised learning, unsupervised learning, and reinforcement learning. These can have different applications in the policy environment.

In supervised learning:

For example, you could give a supervised learning algorithm 1,000 submissions that have been coded with themes, and from that it would be able to code the other 10,000.

In unsupervised learning:

For example, here is a visualisation of an unsupervised learning algorithm that has extracted topics from a subset of submissions on the Zero Carbon Bill.

In reinforcement learning:

Reinforcement learning can be used to model how people might react to policy changes in unexpected ways. For example, a team of researchers has proposed a model using reinforcement learning to help understand tax evasion to design more effective tax policies.

Ideal circumstances for use

You should be clear about what you want to achieve with machine learning, and make sure that you’re not unnecessarily using it for applications that would be much more easily and effectively performed by a person or a simple program.


Some examples of where machine learning is not useful:

Machine learning can be time-consuming, data-hungry, and energy-hungry

Finding, formatting and managing the data required to train a machine learning model is invariably time-consuming. It can take a lot of effort to get data into a consistent enough format to feed to a machine learning algorithm.

Once you do have the data, training the model itself can take a lot of time and computer power. In particular, deep learning algorithms can use enormous amounts of electricity, which comes with attendant carbon emissions - see Energy and Policy Considerations for Deep Learning in NLP.

Machine learning is difficult to inspect

If a normal computer program makes a mistake, it’s generally possible to find why that happened, and change the code to address future problems. This is very difficult with most machine learning algorithms, and impossible with some - see How Might an AI Explain Itself?. Keep this in mind when deciding whether to use machine learning in decisions that directly affect peoples’ lives, and particularly in decisions that must be reviewable.

Machine learning doesn’t erase bias

A machine learning model will reflect the patterns in the data it is trained on. If there are historical patterns of over or under reporting, these will be reflected in the model’s output.