Predictive analytics is becoming increasingly prevalent in the use of AI in education, especially in identifying struggling students who may need interventions to succeed. Machine learning models analyse large datasets to predict student outcomes, such as likelihood of passing a course, and support teams use these insights to guide interventions. However, as these systems become more ingrained, concerns over fairness arise. These concerns are particularly acute in remote or flexible education models, where student engagement is harder to monitor compared to traditional face-to-face learning.
This article explores how predictive analytics can both support and hinder student success. The main challenge addressed is ensuring these systems treat students fairly, regardless of their background. We will explore the sources of potential bias in these systems and discuss strategies to mitigate unfair outcomes.
The role of predictive analytics in AI in education
Predictive analytics systems for student success often work in two parts: first, machine learning models generate predictions based on historical data; second, institutions act on these predictions by providing interventions or making other key decisions. The systems used in education typically focus on three stages of a student’s journey:
Student conversion model – Predicts whether a prospective student will enrol.
Student pass model – Predicts the likelihood of a student passing a course.
Student return model – Predicts whether a student will continue studying in the following year.
These models can significantly improve the efficiency of interventions, allowing institutions using AI in education to direct resources where they are most needed. For example, marketing teams may focus on applicants who are more likely to convert into paying students, while tutors may intervene with students at risk of failing. However, this targeted approach raises questions about fairness: which students benefit most from these interventions? Are disadvantaged groups overlooked or unfairly treated by the models?
The fairness challenge
The key issue with predictive analytics is that it often relies on historical data, which can reflect existing societal biases. For instance, if a model is trained on data where certain groups (such as ethnic minorities or low-income students) have historically underperformed, the model may reinforce these patterns, predicting poorer outcomes for these groups in the future. This becomes a self-fulfilling prophecy, where disadvantaged students are less likely to receive necessary interventions and continue to fall behind.
There are three common scenarios when interventions based on these predictions are implemented:
1.Constant gap – Both advantaged and disadvantaged groups benefit equally from interventions, maintaining the existing performance gap.
2.Widening gap – The intervention helps advantaged students more than disadvantaged ones, increasing inequality.
3.Closing gap – Disadvantaged students benefit more, narrowing the performance gap.
The third scenario is the most desirable, as it reduces inequality. However, it requires interventions to be designed specifically to support disadvantaged groups, which might be viewed as favouring one group over another. Balancing fairness with the practical need to use resources efficiently is an ongoing challenge for institutions using predictive analytics.
Sources of discrimination
Biases in predictive models can stem from several stages in the development and implementation process:
Data Collection
Data collection is a critical step that can introduce bias. For instance, if the target variable (the outcome being predicted) reflects biased definitions of success, such as a focus on academic achievement alone, the model may disproportionately disadvantage students from minority or low-income backgrounds. Additionally, data drift—where the conditions in the real world change over time—can lead to models becoming less accurate. For example, data collected before the COVID-19 pandemic might not account for the new challenges faced by students during and after the pandemic.
Modelling
Models are designed to minimise errors, but they tend to perform better for majority groups, simply because there is more data available for these groups. In UK universities, for example, the majority of students are white, while black students make up a small minority. A model trained primarily on data from white students might not perform as well for black students, even if ethnicity is not explicitly included as a variable. The model assumes that all students share the same characteristics as the majority group, leading to potentially inaccurate predictions for minority students.
Deployment and Interventions
Once predictions are made, institutions must decide how to act. For example, if only a limited number of students can receive support, how should these students be chosen? One ethical dilemma is whether to prioritise students who are more likely to succeed with help or those who are least likely to succeed without it. Without careful planning, interventions may favour students already on the verge of success, leaving the most disadvantaged students without support.
Defining and measuring fairness
There are several ways to define fairness in predictive analytics, but not all can be applied simultaneously. Three common statistical measures of fairness are:
Independence – Predictions should not be influenced by protected characteristics such as race or gender.
Separation – The model should have similar error rates for different groups.
Sufficiency – The probability of the prediction being accurate should be the same for all groups.
It is often impossible to satisfy all three fairness criteria at once, meaning trade-offs must be made. For example, focusing on independence might mean the model ignores important differences between groups, reducing accuracy. On the other hand, prioritising separation or sufficiency could perpetuate existing inequalities. To manage these trade-offs, institutions using AI in education need to carefully consider the specific context and goals of their predictive systems.
Making predictive analytics systems fairer
To reduce bias and improve fairness in predictive systems used in AI in education, several steps can be taken:
Data oversight – Institutions must ensure that data is representative and free from bias. This might involve oversampling underrepresented groups to ensure models can make accurate predictions for all students.
Model evaluation – Models should be evaluated using fairness metrics, such as demographic parity or equalised odds, which measure their performance across different subgroups. These metrics help identify whether a model is disproportionately benefiting one group over another.
Collaboration between teams – Close collaboration between data scientists developing the models and the teams implementing interventions is crucial. Data scientists must understand the real-world implications of their models, while intervention teams need to be aware of the limitations of the predictions.
Randomised control trials (RCTs) – Conducting RCTs can help assess whether interventions based on predictive analytics are having the desired effect. By randomly assigning students to receive interventions, institutions can measure the true impact of these actions and adjust their strategies accordingly.
Predictive analytics holds great promise for improving student outcomes in higher education, but it also presents significant fairness challenges. By carefully considering how these systems are designed, implemented, and monitored, institutions can reduce bias and ensure that all students have an equal opportunity to succeed. As these technologies continue to evolve, it will be important to prioritise fairness to ensure that predictive analytics contributes to, rather than detracts from, equity in education.
References
1.Ackerman, S. et al. (2020). Detection of data drift and outliers affecting machine learning model performance over time.
2.Biswas, S. & Rajan, H. (2020). Do the machine learning models on a crowdsourced platform exhibit bias?
3.Kizilcec, R.F., & Lee, H. (2020). Algorithmic fairness in education.
4.Mehrabi, N. et al. (2021). A survey on bias and fairness in machine learning.
5.Zafar, M.B. et al. (2017). Fairness constraints: Mechanisms for fair classification.