Rights4Seniors Digital Evaluation

Executive Summary

Project background

Advice NI is a membership organisation that exists to provide leadership, representation and support for independent advice organisations to facilitate the delivery of high quality, sustainable advice services. 

Advice NI undertook the Ofcom-commissioned Rights4Seniors project with the recognition that Northern Ireland (NI) is the most digitally excluded region within the UK and that older people are among the most digitally excluded demographic, and with more services transitioning to online, it is becoming increasingly difficult for those experiencing digital exclusion to access their rights and entitlements and other benefits that the Internet brings.

The purpose of this project was to build the media literacy of a cohort of digitally excluded older people and give them greater confidence in carrying out basic but important online tasks. The project successfully delivered training to 105 older people recruited from communities all across NI and delivered Digital Champion training to 10 volunteers recruited by Advice NI from local communities.

A project evaluation was conducted using the Theory of Change Model to determine what difference was made (or not) in the level of participant digital skills; and what worked well, less well or could have been improved in relation to supporting cohorts with media literacy.

The evaluation required participants, both the older people (or learners) and Digital Champions, to complete a survey before their training and then again after their training. The ‘before’ and ‘after’ surveys for learners helped measure the difference in self-reported digital skills levels regarding online information, apps, banking, and scams and frauds to determine any change that had occurred as a result of completing the training. The ‘before’ and ‘after’ surveys for Digital Champions measured the change that occurred in their self-reported understanding of being a Digital Champion and whether their confidence in being mentors had changed. A focus group was also carried out with Digital Champions.

The evaluation relied on self-reported data as testing learners’ skills during their training was deemed intimidating and onerous. There was a mismatch in the numbers of ‘before’ surveys and ‘after’ surveys, where not all those who completed a ‘before’ survey completed an ‘after’ survey. The mismatch was so slight, it was decided to treat the ‘before’ and ‘after’ surveys as two homogenous batches rather than make a direct comparison of individual ‘before’ and ‘after’ survey responses.

Findings

Analysis of the findings from the surveys and focus group demonstrated that there was a considerable positive change in the digital skills of learners. At the start of the training, learners felt less confident, less aware and less able across all topics surveyed, and fewer learners felt very confident, aware or able. By the end of the training, the inverse was true.

A smaller percentage of learners was less confident, aware or able; while a substantially greater percentage was very confident, aware and able.

Overall, at the start of the training just 10% to 15% of learners felt very confident, aware, and able across all the topics surveyed. By the end of the training, 30% to 40% of learners’ felt confident, aware, and able across all the topics surveyed. This change suggests that learners were also able to carry out more online activities as a result of their training, because the data from the surveys showed that for learners, in all of the 11 digital skills surveyed, bar none, improvements were made and that in many cases, the improvements were substantial.

Some of the key findings from the learners’ surveys were: 

The 5 skills that showed the biggest improvement	Before training:	After training:	% Point Difference
Those who felt able to keep their username and password safe	10%	42%	+32%
Those who felt confident using online information to make decisions about products and services	12%	40%	+28%
Those who felt able to protect themselves from online financial scams and fraud	6%	34%	+28%
Those who felt able to identify reliable sources online	9%	35%	+26%
Those felt able to buy goods online	15%	39%	+24%

Lessons learned: what worked, what did not 

The evaluation also looked at lessons learned and what worked well or less well. Learners reported that the training was viewed as “practical”, “useful” and “thought-provoking”. Training materials were described as “clear” and “excellent”. To enhance the learning experience, training materials and surveys were translated into Polish for the Polish participants; and the Chinese Digital Champions translated the materials and surveys into Cantonese. Digital champions noted that the low level of experience among some learners required them to be flexible in how they used the training materials, and to deliver the training at a pace that suited the learners.

The interactive tasks in the training proved to be very popular among learners, in particular the fake news and scams tasks. The interactive tasks drew out conversation and gave learners a trusted environment where they felt safe to talk.

Suggested improvement to the training materials included adding an element to the training teaching people how to do a Last Will & Testament online, and what to do with an online presence when someone dies.

Digital Champions liked the fact that their training was accredited. They saw this as very beneficial to their own development.

In line with the evidence base around engaging older men in social activities, there was a gender imbalance in the training with three times as many female learners as male, indicating that for future projects, greater effort would be required to attract more male participants.

Legacy and next steps

Overall, the project made the desired impact on learners and achieved its outcomes. 

It proved to be successful for both learners and Digital Champions, providing learners with skills that should be of practical useful to them in their lives and providing Digital Champions with skills that enable them to continue being Digital Champions within their own communities.

The impact and benefits of the project are expected to extend beyond the project’s lifetime and beyond those directly involved. All 7 Digital Champions, who completed the survey, said they would be very likely to continue using their skills to help people in their community. The training also gave some Digital Champions an appetite to pursue further volunteering and training opportunities.

The project has potential next steps. Digital Champions expressed a desire to revisit learners after some time had passed to see how they were getting on, for example, to see if any of them had signed up for online banking after all, if they were using the cost comparison sites, etc. This perhaps suggests an opportunity to do a future piece of work that would involve contacting learners and asking them to complete another survey to get a sense of what skills they are still using in their lives.

Introduction

Advice NI

Advice NI is a membership organisation that exists to provide leadership, representation and support for independent advice organisations to facilitate the delivery of high quality, sustainable advice services.  Advice NI exists to provide its members with the capacity and tools to ensure effective advice services delivery.  This includes: advice and information management systems, funding and planning, quality assurance support, NVQs in advice and guidance, social policy co-ordination and ICT development.

Membership of Advice NI is normally for organisations that provide significant advice and information services to the public.  Advice NI has over 70 member organisations operating throughout Northern Ireland and providing information and advocacy services to over 110,000 people each year dealing with almost 250,000 enquiries on an extensive range of matters including: social security, housing, debt, consumer and employment issues.  For further information, please visit www.adviceni.net.

Project background

The Rights4Seniors project was commissioned by Ofcom. Making Sense of Media is Ofcom’s programme of work to help improve the online skills, knowledge and understanding of UK adults and children. Rights4Seniors was 1 of 13 projects commissioned in December 2022 to improve media literacy skills across four cohorts: Older adults; People living with disabilities, learning disabilities or cognitive impairment; Children and young people; Communities experiencing financial disadvantage.

Advice NI undertook the Rights4Seniors project with the recognition that NI is the most digitally excluded region within the UK and that older people, particularly those aged 75+, are among the most digitally excluded demographic. With more services, including more public services, transitioning to online, it is becoming increasingly difficult for those experiencing digital exclusion to access their rights and entitlements and other benefits that the Internet brings. Digital exclusion puts people of all ages at a disadvantage. We are also aware, from our past experience of delivering digital inclusion training to older people, that many older people experience financial disadvantage and are unaware of social security benefits they may be entitled to.

The aim of this project was to build the media literacy of a cohort of digitally excluded older people — called learners throughout this report — and give them greater confidence in carrying out basic but important online tasks such as identifying fraud and scams and checking cost comparison sites for better deals, etc. The project delivered this training to 105 digitally disadvantaged older people recruited from communities across NI. The project also delivered Digital Champion training to 10 volunteers recruited by Advice NI from local communities. The volunteers were trained as Digital Champions who in turn delivered the core digital training to learners. Digital Champion training was accredited OCN NI Level 2 training.

Providing learners and Digital Champions with essential digital skills through their training on the project should not only help those directly involved, but should extend the benefits of the project beyond its lifetime and beyond those directly involved. This report provides an evaluation of the training that was delivered through the project.

Evaluation aims and methodology

The aim of the project evaluation is to determine whether learners and Digital Champions benefited from taking part in the project by assessing what difference was made (or not) in the level of their digital skills; and what worked well, less well or could have been improved in relation to supporting cohorts with media literacy.

To meet these aims, the evaluation was designed to measure if a number of outcomes were met, namely: 

• the extent to which learners felt more confident using online tools; 
• the extent to which they felt more equipped to protect their online safety; 
• the extent to which they were more likely to be able to identify reliable online content; 
• the success of the project in engaging older people experiencing disadvantage; and 
• the success of the project in increasing participants financial wellbeing

The Theory of Change Model was used as an evaluation framework to focus on measuring how much the training achieved and what difference it made rather than simply counting activities. The Model was applied as a guide when designing the evaluation materials and required a ‘before’ and ‘after’ analysis of the situation under evaluation. Therefore, four surveys were designed in total: 2 surveys for learners to complete, 1 at the start of the training and 1 at the end; 2 surveys for Digital Champions to complete, 1 at the start of the training and 1 at the end. A focus group was also conducted with the Digital Champions to have a more informal conversation about their experience of delivering the training. The questions for all 4 surveys are provided in Appendix A and the focus group questions are provided in Appendix B. All of this was agreed with Ofcom at the outset.

For the ‘before’ learner survey, a series of questions were asked to help gauge:

• the current level of digital skills among learners regarding their digital awareness, attitudes and behaviours towards online scams and frauds; 
• their digital awareness, attitudes and behaviours towards online information;  • their knowledge of online money resources; and 
• their confidence using online tools. 

For the ‘after’ learner survey, the same set of questions were asked, with some additional questions aimed at measuring the potential legacy of the project. The ‘before’ and ‘after’ snapshots were then compared to determine any change that had occurred and whether the training had made a self-reported to the level of digital skills.

For the ‘before’ Digital Champions survey, the questions asked were aimed at determining learners’ understanding of what it means to be a Digital Champion. The same questions were asked in the ‘after’ survey along with a number of additional questions which attempted to measure the potential legacy of the project. A comparison of the ‘before’ and ‘after’ snapshots were used to determine any change that had occurred in their understanding of being a Digital Champion and whether their confidence in being mentors in their own areas had changed. Comments from the focus group were included where they seemed relevant and enhanced the survey data.

It was decided not to survey a sample of participants but instead to ask all participating learners and Digital Champions to complete the ‘before’ and ‘after’ surveys. A total of 105 learners participated in the training, exceeding the target of 94 set in the project tender document. The number of learners who completed the ‘before’ survey was 94 learners, and the number who completed the ‘after’ survey was 83, giving an extremely high response rate of 90% and 80% respectively.

Due to this slight mismatch in ‘before’ and ‘after’ survey responses, a decision had to be made about how to handle the mismatch for the purposes of the data analysis; i.e. whether a direct comparison should be carried out for the individual ‘before’ and ‘after’ survey responses, thus omitting the non-matching responses; or whether the ‘before’ responses should be bundled as a single cohort and the ‘after’ responses as a single cohort and then a direct comparison carried out of those two homogenous cohorts. To help make this decision, a random sample was taken of the individual ‘before’ and ‘after’ responses and a comparison conducted for five of the topics surveyed; a comparison was then conducted for the same five topics for the two homogenous cohorts. In both sets of results, the differences were so slight (and all results showed improvements in the ‘before’ and ‘after’ situation) that it was decided to go with the two homogenous cohorts. While this could be construed as a limitation of the evaluation, it did mean that none of the responses were omitted.

The responses from the surveys were exported into MS Excel and bar charts were created to compare the ‘before’ and ‘after’ situation for both learners and Digital Champions to demonstrate the change that took place for all cohorts after completing their respective training.

No real-time testing of learners took place e.g. multi-choice tests, quizzes, etc., as that was deemed too onerous and intimidating for learners. Rather, only the surveys and a focus group were used to determine what change took place for learners. This meant that the evaluation relied solely on self-reported data. This could be considered a limitation of the evaluation but for this project, it was deemed necessary.

Learners: A ‘Before’ and ‘After’ Comparison’

The bar charts created from the survey responses and an accompanying written narrative are provided in this section to present the findings from the surveys and the focus group.

Overview of Learners

Photographs from the training sessions are available in Appendix C. The 105 learners who participated in the training were selected from across NI using Advice NI’s existing networks and 8 out of the 11 council areas were represented as follows: 11% from Antrim and Newtownabbey Borough Council; 1% from Ards and North Down Borough Council; 26% from Belfast City Council; 18% from Causeway Coast and Glens Borough Council; 18% from Derry City and Strabane District Council; 3% from Fermanagh and Omagh District Council; 2% from Lisburn and Castlereagh City Council; and 21% from Newry, Mourne and Down District Council. The training was delivered in 14 separate venues.

The age of learners was well distributed, with 14% aged between 56 and 65, 43% aged between 66 and 75 and 42% aged 76 and over. Only 1% was 55 or under. A goal of the training was to attract the older age groups so it succeeded in this respect. Learners were largely female at 77%, with 23% being male

Learners were predominately white, as would be expected in a region that is over 90% white. Taking that into account, the mix of ethnic groups was a more than adequate representation of the NI population. Those identifying as white (English / Welsh / Scottish / Northern Irish / British) were 65%, those identifying as white Irish at 6% and those identifying as white Polish were 5%. Fourteen percent were Indian and 10% were Chinese.

Confidence using the Internet to find general information

Before the training started, learners were asked about their level of confidence in using the Internet to find general information e.g. daily news, consumer advice, money advice, instructional videos. Figure 1 depicts the change in confidence levels between the start and end of the training. At the outset, 14% were very confident but that increased to 29% by the end of the course.

Ability to use the Internet to find information about rights and entitlements

Learners were asked how able they were to use the Internet to find information about rights and entitlements e.g. benefits, pension, citizenship. The ‘before’ and ‘after’ change in ability is fully presented in figure 2, where just 10% were very able at the start of the training and by the end it tripled to 30% of learners being very able.

Awareness of the veracity of online information

When asked if they were aware that not all online information is true or reliable, 33% said they were not at all aware before training and that dropped to 12% when the training was over. Figure 3 provides a representation of the change for learners.

Ability to identify reliable sources online

The question about ability to identify which websites contain information that is true or reliable found that 55% of learners were not at all able when they began the training, while that plummeted to 11% by the end of the training. Figure 4 has a visual representation of the ‘before’ and ‘after’ change.

Confidence using online information to make decisions

Learners were asked about their level of confidence in using online information to make decisions about products and services. Before the training, a mere 12% said they were very confident. When the training was completed, 40% said they were very confident. In figure 5, the change for learners from the start to the end of the training is shown.

Awareness of online scams and fraud

At the start of the training, learners were asked whether they were aware of and could recognise online financial scams and fraud. A low of 10% said they were very aware at the outset, but that rose to 27% after the training. Figure 6 shows the change in awareness for learners.

 

Ability to check safety of online sources

Learners were asked when they started the training if they were able to check whether a website or App was safe. The majority, 60%, said they were not at all able before the training. Once their training was over, 35% of learners said they were not at all able. The change in ability is depicted in figure 7.

Ability to keep username and password safe

The starting position for learners when it came to keeping their username and password safe was 10% were very able. When asked at the end of the training, those who were very able was as high as 42%. The change over the duration of the training in learners’ ability is provided in figure 8.

Ability to safeguard against online scams and fraud

Learners were asked at the outset of the training whether they were able to protect themselves from online financial scams and fraud, and only 6% said they were very able. By the end of the training, 34% said they were very able. Figure 9 shows the ‘before’ and ‘after’ snapshot of ability.

Ability to buy goods online

When learners were asked about their ability to buy goods online through websites and Apps, over half, 52%, were not at all able. Once the training was over, there was a substantial decrease, down to 25%, in those who were not at all able. The change in ability to buy goods online is presented in figure 10.

Ability to use price comparison sites

Learners were asked at the start of the training how able they were at using price comparison sites e.g. electricity, gas, phone, broadband, TV, insurance, to get the best deal available. Just 10% said they were very able. When they completed the training, 30% said they were very able. In figure 11, the difference in ability from the start to the finish of the training is depicted.

Registration for online banking

At the start of the training, 51% were not registered. At the end of the training, those not registered had fallen to 32%, with a further 11% saying they were not registered but were considering doing so. Figure 12 provides a visual representation of the change in online banking registration.

Confidence using online banking services

When asked if they were confident using online banking services e.g. checking balance, checking recent transactions, transferring money, paying bills, 50% of learners started out being not at all confident. After completion of the training, 39% were not at all confident. Learners’ confidence ‘before’ and ‘after’ the training is shown in figure 13.

Ability to compare financial services online

When it came to being able to compare financial services online e.g. bank accounts, saving accounts, credit cards or loans, 10% of learners were very able. When asked the same question after completing the training, those who stated they were very able had risen to 30% were very able. The difference ‘before’ and ‘after’ training is shown in figure 14.

Digital Champions: A ‘Before’ and ‘After’ Comparison

Introductory questions

The Digital Champions’ training was always intended to be for a much smaller group than the core training for the older people, and a total of 10 volunteers completed the training. However, just 7 of these filled in the surveys. Because of the small number of Digital Champions, the raw numbers are reported rather than percentages.

Digital Champions came from 4 out of the 11 council areas across NI with distribution as follows: 1 from Antrim and Newtownabbey Borough Council; 1 from Ards and North Down Borough Council; 1 Armagh City, Banbridge and Craigavon Borough Council; and 4 from Belfast City Council. Their ages ranged from 55 or under (3), 56 to 65 (3) and 66 to 75 (1). None were in the 76 and over age bracket. The mix of male and female Digital Champions was fairly even with 3 females and 4 males. Four Digital Champions identified as white (English / Welsh / Scottish / Northern Irish / British), 2 were African and 1 person selected “rather not say”. Before starting the training, Digital Champions were asked if they understood the role of the Digital Champion and all of them answered that they did.

Awareness of barriers to getting and staying online

Before their training, Digital Champions were asked if they were aware of the barriers preventing people from getting and staying online. Before the course only 1 of the 7 trainers said that they were very aware, after the course 6 of the 7 trainers were very aware. The change in awareness that occurred for Digital Champions is presented in figure 15.

Ability to support learners

At the start of the training, when Digital Champions were asked to what extent they were able to support learners with different learning styles on their digital journey, none said they were very able but by the end of the training, all 7 Digital Champions said they were very able. Figure 16 depicts the change that took place as a result of the training.

Ability to keep login details for others safe

As figure 17 illustrates, when asked if they were able to help others keep their username and password safe, none of the Digital Champions said they were very able. After completing the training, all Digital Champions said they were very able.

Awareness of tools and resources for remote support

Digital Champions were asked at the outset if they were aware of the tools and resources available to help support learners remotely and none said they were very aware. When asked the same question after the training, 6 Digital Champions were very aware of what was available. Figure 19 presents the change from ‘before’ and ‘after’ training.

 Analysis of the findings

This section sets out an analysis of the findings that demonstrates how the project met its intended aims and outcomes. As explained previously, the project evaluation set out to determine whether learners and Digital Champions benefited from taking part in the project by assessing what difference was made (or not) in the level of their self-reported digital skills; and what worked well, less well or could have been improved in relation to supporting cohorts with media literacy. A number of outcomes had to be met in order to realise the project aims i.e. the extent to which learners felt more confident using online tools; the extent to which they felt more equipped to protect their online safety; the extent to which they were more likely to be able to identify reliable online content; the success of the project in engaging older people experiencing disadvantage; and the success of the project in increasing participants financial wellbeing. The following sections discuss the data in relation to the aims and outcomes. 

Outcomes: Extent to which learners felt more confident using online tools; Extent to which learners felt more equipped to protect their online safety; Extent to which learners were more likely to be able to identify reliable online content

The extent to which learners felt more confident using online tools was met through elements of the training that supported learners to use the Internet to find general information and to find information about rights and entitlements; and that supported learners to use online information to make decisions, buy goods online, use price comparison sites, register for online banking and use online banking services, and compare financial services online.

The extent to which learners felt more equipped to protect their online safety was met through elements of the training that supported learners to spot online scams and fraud, to check the safety of online sources, to keep their username and password safe and to safeguard against online scams and fraud.

The extent to which learners were more likely to be able to identify reliable online content was met through elements of the training that supported learners to judge the veracity of online information and identify reliable sources online.

The data from the surveys showed that for learners, in every digital skill surveyed, bar none, improvements were made and that in many cases, the improvements were substantial. The overall, cumulative change is shown in the boxplots in figures 20 and 21. Boxplots are a means of summarising a set of data. The shape of the boxplot shows how the data are distributed along with any outliers and is a useful way to visually compare different sets of data.

Figure 20 illustrates the levels of learner awareness, confidence and ability in carrying out online activities before the training. In all the topics surveyed, between 40% and 60% of learners were not at all aware, confident or able. These high percentages make it clear that learners had very low digital skills levels at the outset of the training. The ‘after’ box in figure 20 shows a very different picture. Learners’ lack of awareness, confidence or ability dropped to between 15% and 30%, demonstrating a substantial decrease in the percentage of learners who were not at all aware, confident or able to carry out online activities. In fact, for some digital skills the ‘before’ and ‘after’ differences were as much as 36%.

Further, the project saw more learners become very confident, aware and able in carrying out online activities. As figure 21 depicts, at the start of the training just 10% to 15% of learners were very confident, aware and able across all the topics surveyed. These very low percentages support the data in figure 20, and provide more evidence that learners had very low levels of digital skills before embarking on the training. By the end of the training, the situation was reversed, with learners’ confidence, awareness and ability increasing up to 30% and 40%. The change here is evidence of the increase in learners being able to carry out online activities as a result of their training.

There were two outliers, as seen in figure 21, where there were higher than average percentages of learners who were very aware or confident in a skill before the start of the training. In the first outlier, prior to starting the training there was already quite a high level of awareness that not all online information was true or reliable, where 44% of learners said they were very aware. After the training, that increased to 48%, indicating that the training still made a difference. The second outlier was regarding confidence using online banking services and 25% of learners said they were very confident at the start of the training. At the end of the training the percentage of those who were very confident went up to 35%, a 10% increase due to the training.

The survey question about online banking was not included in figures 20 and 21, as it had a different format from the others. Quite a high number of learners, 49%, were already registered for online banking when the training started, with the remaining 51% not registered. At the end of the training, the percentage registered had increased only slightly, up to 57%. And although 43% still remained unregistered for online banking, 11% of those were considering doing so as a result of the training. The change resulting from the training was less significant than for other topics surveyed but the still change showed improvement.

The data confirm that there was a clear shift or change in the digital skills of learners. At the start of the training, they mostly had less confidence, awareness and ability across all skills surveyed, and smaller percentages of them felt very confident, aware or able. By the end of the training, the inverse was true. A smaller percentage of learners was less confident, aware or able; while a greater percentage was very confident, aware and able.

It is also worth providing a brief analysis of the change that took place for Digital Champions. The 7 who completed the surveys started out with low and relatively low awareness of the barriers to getting and staying online; ability to support learners; ability to keep login details safe for others; awareness of the safety of their own login details; and awareness of the tools and resources for remote support. After their training, no Digital Champion said they were not at all aware or able and all but 1 said were very aware and able. These data show the dramatic change in skill levels for Digital Champions after they completed their training, verifying that the project outcomes with regard to Digital Champions were achieved, whereby their knowledge in training and signposting others increased, as did their confidence in helping others use online tools.

Overall, the findings indicate that learners benefited from taking the training and the change in their digital skills improved as a result with regard to their confidence using online tools; their ability to protect their online safety; and the likelihood of their being able to identify reliable online content. The findings also show that Digital Champions benefited from their training. It is fair to assert that the project made a positive difference to the level of self-reported digital skills and media literacy for learners and Digital Champions alike.

Outcomes: Success of the project in engaging older people experiencing disadvantage; Success of the project in increasing participants financial wellbeing

The project set out to engage older people and it did this successfully. All learners, with 1 exception, were in older age groups, with 85% of all learners aged 66 and over.

Regarding engaging older people experiencing disadvantage, in the context of this project, disadvantage refers to digital disadvantage because in a world where services and goods are increasingly moving to online platforms, not having the know-how to access those services and goods is a major disadvantage and is the modern day equivalent of illiteracy. In having gone through the training, the findings from the data showed that learners gained confidence in the full range of topics that were taught, and their awareness and ability increased. Learners gained new skills in using online tools to find information and make decisions, to buy goods online and use comparison sites, to lookout for scams and protect themselves online. As a result of taking part in the training, learners substantially reduced the disadvantage of digital exclusion and were more media literate.

Increasing participants financial wellbeing was another outcome of the project. With the steady closure of bank branches and ATMs, banks expect their customers to rely more on online banking. In addition, it is also true that better deals can be found for utilities, broadband, insurance, etc. using online cost comparison sites. Having the skills to make use of online services improves a person’s financial wellbeing. The training on this project aimed to address these challenges, which disproportionately impact older people, by teaching skills such as registering for online banking, using online banking services, and comparing online financial services. Again, the data demonstrated that learners were more skilled in these areas at the end of their training than they were to begin with, and clearly the training made a positive change.

Lessons learned: what worked, what did not

One of the purposes of the evaluation is to understand what worked well, less well or could have been improved when supporting cohorts with media literacy. This section presents a discussion of what was learned throughout the project. Much of the content was gathered from the focus group with the Digital Champions and from the opened ended questions that learners were asked at the end of their ‘after’ survey.

Digital media and supporting learners

The majority of learners said the training was “excellent” and “enjoyable”, describing it as “worthwhile”, “informative” and “interesting”: “Fantastic training that I would highly recommend to others.” The training was viewed as “practical”, “useful” and “thought provoking”: “Great training with lots of practical tips and advice.” These comments demonstrate that the media support provided by the project worked well for learners.

To enhance the learning experience, training materials and surveys were translated into Polish for the Polish participants; and the Chinese Digital Champions translated the materials and surveys into Cantonese. Training materials were described as “clear” and “excellent”, and learners liked the interactive nature of the training, which they said gave them “skills they could put into practice.”

In the focus group, Digital Champions agreed that the training materials such as slides and paper handouts were excellent and were good to have on hand. The materials were useful for giving structure to the training and the handouts helped as memory aids. They talked about their approach to the sessions and how they made use of the materials, pointing out that the trick was not to be too formal, but instead to be flexible and adapt to suit the audience.

Digital Champions noted the low level of experience among some learners, which required them as Digital Champions to adjust how they used the materials at first. For instance, they introduced some of the early topics verbally, leaving the materials / handouts aside until much later.

The materials were really useful and would often trigger discussion around an issue, and being flexible would allow for that discussion to happen. For instance, one Digital Champion said “you'd find from that conversation what was important because there'd be chat about maybe a certain topic and then you would know that they wanted to focus on that. So maybe we would spend more time on that particular thing and then we would have to we would adjust the rest of it. So, the materials are a guide and then you use them as suits.”

The focus group gleaned further insights from Digital Champions about the usefulness of tools and resources for remote support. Digital Champions talked about technical difficulties that made using the materials tricky at times. One suggested that a contingency against technical difficulties might be to have a handout containing discussion questions that needed to be covered in each session.

Digital Champions noted that the training brought up other issues of concern to the learners, specifically in relation to legal and financial issues. For example, some learners wanted to know what were the best banking options for online banking. Digital Champions knew they couldn’t give that kind of advice, that their role was only to show learners how.

The focus group revealed how Digital Champions were able to support learners through the training. For instance, several of the female learners were keen to get to grips with online banking, specifically because their husbands did all their online banking and they were genuinely concerned that if he died before them, they would not know what to do.

The focus group also drew out some suggested improvements to the learner’s materials. For example, it was suggested that some of the information in the Digital Champion notes be included in the notes given to learners e.g. Consumer Council Helpline Number and Learn My Way, so they have those to take away rather than having to writing it down. Another suggested improvement was adding an element to the training that taught people how to do a Last Will & Testament online, and also what to do with an online presence when someone dies.

Digital Champions

Learners had high praise for the volunteers who they thought were very “able”, “highly skilled”, “patient”, “diligent” and “moving at the right pace”: “Training was delivered with patience and understanding of learners needs. Covered the many concerns people had issues with but also topics that were completely new, very helpful.” “The volunteers had superb skills to transfer the information and from this I now feel more confident about going online.”

Digital Champions were asked in their ‘after’ survey what they liked about their digital champion training. They pointed out a number of positives, including learning alongside others, discussions, working in pairs, meeting new people, and interaction that allowed everyone’s voice to be heard. The Advice NI project tutors were praised for their effectiveness and for creating a nice atmosphere and friendly learning environment. The training helped build confidence and build new knowledge that was practical and relevant.

In the focus group, Digital Champions talked more about what they got out of their digital champion training, saying it increased their confidence levels. The fact that their training was accredited this time was also beneficial to their own development.

Learner confidence and overcoming fear

Multiple learners mentioned having greater confidence and improved understanding and awareness after the training: “I am grateful that I had a place on this training.” “I gained more confidence from being part of this small group.” “I have new information that allows me to be an active member of this new online world.” “I have more confidence now from the units covered in the training.” “I am not as apprehensive about online scams and feel more confident in protecting myself now when online.”

When asked what they thought was the main difference that they made for learners they trained, Digital Champions mentioned helping learners overcome their fear of using devices and of building confidence in learners, for example, getting them to a point where they were comfortable ordering goods online.

In the focus group, Digital Champions explicitly mentioned that learners were afraid and cautious about doing anything online and saw this a barrier. Digital Champions worked extra hard to help learners overcome their fears. For example, one Digital Champion said that “within the five weeks, there's only a certain amount you can do because so much can be done online now… [but] you’re hoping that you're giving them a bit more confidence to do something and take away some of the fear factor – where they might be afraid of making a mistake or ordering the wrong thing online; or maybe they wary of using online banking in case somebody's going to get their details; scams are another big fear.”

Once they were helped to overcame their fears, learners gained confidence quite quickly. According to one Digital Champion, “a few of them [learners] were very savvy when it came to the shopping element. You could teach them about refunds for example, how to get money back through PayPal. It was like, right, you can do that. And there is a complaint through PayPal. Get your money back. There you go.”

Interactive tasks

The interactive tasks in the training provided a lot of support to learners because they gave learners the scope to delve deeper and ask questions. A Digital Champion explained that “part of the beauty of the training was the ability for people to interact, not necessarily with the Digital Champion but amongst themselves, telling stories about their problems and their issues.” The fake news and scams tasks were particularly popular. One Digital Champion said that “the email scams are fascinating because they [learners] were getting them all wrong. And then when you point out what's wrong, oh, it's one of those moments in time and they realise, ah, that could have been me.”

The interactive tasks drew out conversation and gave learners a trusted environment where they felt safe to talk. As one Digital Champion said, “some learners showed a lot of courage in admitting that they’d been scammed or had lost money online; it took courage to share that. But sharing those experiences taught everybody a lot. So these teaching sessions create a forum for people to tell others about things and scams is one of those topics; the training opened a forum for people to share experiences good and bad and that is very beneficial.”

Gender

In line with the evidence base around engaging older men in social activities, there were almost three times as many female learners as there were male. This flags a gender imbalance that is worth noting for future projects and highlights the need to make a bigger effort to attract more male participants.

Legacy and next steps

In relation to legacy, the project made the desired impact on learners and Digital Champions and achieved its outcomes. Given this, the impact and benefits of the project are expected to extend beyond the project’s lifetime and beyond those directly involved. Further, all 7 Digital Champions said they would be very likely to continue using their skills to help people in their community. When asked to elaborate, they wrote about their desire and eagerness to continue helping others now that they were trained to do so. One Digital Champion explained how they understood the issues that discourage people from using the Internet and how they would help such people overcome these challenges. Another came to realise because of the training the importance of engaging with and supporting older people who could potentially become very disadvantaged in a digital age. The training also gave some Digital Champions an appetite to pursue further volunteering and training opportunities.

The project has potential next steps. Digital Champions expressed a desire to revisit learners after some time had passed to see how they were getting on, for example, to see if any of them had signed up for online banking after all, if they were using the cost comparison sites, etc. A Digital Champion mentioned how “there was a woman said that she'd always been afraid of cost comparison websites, and she felt we’d helped take the fear away and she was going to give it a go. So, I love to think that they did that.” This perhaps suggests an opportunity to do a future piece of work that would involve contacting learners and asking them to complete another survey to get a sense of what skills they are still using in their lives.

The project proved to be successful for both learners and Digital Champions, providing learners with online skills that should be essential to them in their lives and providing Digital Champions with skills that enable them to continue being Digital Champions within their own communities. This success means that it would be worthwhile to deliver the project again.

Appendix A – Survey Questions

Rights4Seniors Digital Skills Training (‘before’ survey)

1. What is your name?
2. What is your email address?
3. Select the council area you live in [list of the 11 council areas given]
4. Select the age range that best fits your age [55 or under; 56 – 65; 66 – 75; 76+]
5. What is your gender [Male; Female; Other; Rather not say]
6. Choose one option that best describes your ethnic group or background:
   • White (English / Welsh / Scottish / Northern Irish / British)
   • White (Irish)
   • White (Gypsy or Irish Traveller)
   • White and Black Caribbean (mixed race)
   • White and Black African (mixed race)
   • White and Asian (mixed race)
   • Asian (Indian)
   • Asian (Pakistani)
   • Asian (Bangladeshi)
   • Asian (Chinese)
   • African
   • Caribbean
   • Arab
   • Any other ethnic group (please describe)  
7. I am confident in using the Internet to find information online e.g. daily news, consumer advice, money advice, instructional videos [Not at all confident; Moderately confident; Very confident]
8. I am able to use the Internet to find information about my rights and entitlements e.g. benefits, pension, citizenship [Not at all able; Moderately able; Very able]
9. I am aware that not all online information is true or reliable [Not at all aware; Moderately aware; Very aware]
10. I am able to identify which websites contain information that is true or reliable [Not at all able; Moderately able; Very able]
11. I am confident in using online information to make decisions about products and services [Not at all confident; Moderately confident; Very confident]
12. I am aware of and can recognise online financial scams and fraud [Not at all aware; Moderately aware; Very aware]
13. I am able to check if a website or App is safe [Not at all able; Moderately able; Very able]
14. I am able to keep my username and password safe [Not at all able; Moderately able; Very able]
15. I am able to protect myself from online financial scams and fraud [Not at all able; Moderately able; Very able]
16. I am able to buy goods online through websites and Apps [Not at all able; Moderately able; Very able]
17. I am able to use price comparison sites e.g. electricity, gas, phone, broadband, TV, insurance to get the best deal available [Not at all able; Moderately able; Very able]
18. I am registered for online banking [Yes; No]
19. I am confident using my online banking services e.g. checking balance, checking recent transactions, transferring money, paying bills, etc. [Not at all confident; Moderately confident; Very confident]
20. I am able to compare financial services online e.g. bank accounts, saving accounts, credit cards or loans [Not at all able; Moderately able; Very able]

Rights4Seniors Digital Skills Training (‘after’ survey)

1. What is your name?
2. I am confident in using the Internet to find information online e.g. daily news, consumer advice, money advice, instructional videos [Not at all confident; Moderately confident; Very confident]
3. I am able to use the Internet to find information about my rights and entitlements e.g. benefits, pension, citizenship [Not at all able; Moderately able; Very able]
4. I am aware that not all online information is true or reliable [Not at all aware; Moderately aware; Very aware]
5. I am able to identify which websites contain information that is true or reliable [Not at all able; Moderately able; Very able]
6. I am confident in using online information to make decisions about products and services [Not at all confident; Moderately confident; Very confident]
7. I am aware of and can recognise online financial scams and fraud [Not at all aware; Moderately aware; Very aware]
8. I am able to check if a website or App is safe [Not at all able; Moderately able; Very able]
9. I am able to keep my username and password safe [Not at all able; Moderately able; Very able]
10. I am able to protect myself from online financial scams and fraud [Not at all able; Moderately able; Very able]
11. I am able to buy goods online through websites and Apps [Not at all able; Moderately able; Very able]
12. I am able to use price comparison sites e.g. electricity, gas, phone, broadband, TV, insurance to get the best deal available [Not at all able; Moderately able; Very able]
13. I am registered for online banking [Yes; No; No, but I'm considering it]
14. I am confident using my online banking services e.g. checking balance, checking recent transactions, transferring money, paying bills, etc. [Not at all confident; Moderately confident; Very confident]
15. I am able to compare financial services online e.g. bank accounts, saving accounts, credit cards or loans [Not at all able; Moderately able; Very able]
16. Have you any other comments about the training?

Digital Champion Training (‘before’ survey)

1. What is your name?
2. What is your email address?
3. Select the council area you live in [list of the 11 council areas given]
4. Select the age range that best fits your age [55 or under; 56 – 65; 66 – 75; 76+]
5. What is your gender [Male; Female; Other; Rather not say]
6. Choose one option that best describes your ethnic group or background:
   • White (English / Welsh / Scottish / Northern Irish / British)
   • White (Irish)
   • White (Gypsy or Irish Traveller)
   • White and Black Caribbean (mixed race)
   • White and Black African (mixed race)
   • White and Asian (mixed race)
   • Asian (Indian)
   • Asian (Pakistani)
   • Asian (Bangladeshi)
   • Asian (Chinese)
   • African
   • Caribbean
   • Arab
   • Any other ethnic group (please describe) 
7. I understand the role of a digital champion [Yes; No]
8. To what extent are you aware of the barriers preventing people from getting and staying online [Not at all aware; Moderately aware; Very aware]
9. To what extent are you able to support learners with different learning styles on their digital journey [Not at all able; Moderately able; Very able]
10. To what extent are you able to help others keep their username and password safe [Not at all able; Moderately able; Very able]
11. To what extent are you aware of how to keep your username and password safe if delivering training [Not at all aware; Moderately aware; Very aware]
12. To what extent are you aware of the tools and resources available to help support learners remotely [Not at all aware; Moderately aware; Very aware]
13. Is there anything about the training that is worrying you?
14. Is there anything else you would like to tell us?

Digital Champion Training (‘after’ survey)

1. What is your name?
2. I understand the role of a digital champion [Yes; No]
3. To what extent are you aware of the barriers preventing people from getting and staying online [Not at all aware; Moderately aware; Very aware]
4. To what extent are you able to support learners with different learning styles on their digital journey [Not at all able; Moderately able; Very able]
5. To what extent are you able to help others keep their username and password safe [Not at all able; Moderately able; Very able]
6. To what extent are you aware of how to keep your username and password safe if delivering training [Not at all aware; Moderately aware; Very aware]
7. To what extent are you aware of the tools and resources available to help support learners remotely [Not at all aware; Moderately aware; Very aware]
8. What did you like about the training?
9. What could be improved?
10. What do you think was the main difference that you made for people through the training?
11. Now that you’ve trained as a digital champion, to what extent are you likely to continue using your skills to help people in your community? [Not at all likely; Moderately likely; Very likely]
12. What makes you say this?

Appendix B – Focus Group Questions

1. Icebreaker question.
2. Why did you want to volunteer with Rights4Seniors? What attracted you to the role?
3. What were the main things you got out of the training? (Probe: What about/which parts of the training led to that/caused that? How did that help you deliver the training to others?)
4. How would you describe the training that you received? (Probe: To what extent did you feel prepared for the volunteer role? Is there anything that you had wished you had known sooner? Why? How useful were the materials? How did they help you? Prompt: did you need to adjust them?
5. What were the main things that you got out of the volunteering? (Probe: Was there anything that surprised you? Did you learn anything from learners, about yourself, about working with local communities?)
6. Think of three words to describe your volunteering experience on Rights4Seniors (Probe: What went well? Probe: Was there anything that you found difficult? How did you overcome this challenge or what would have helped you overcome this challenge?
7. What difference, if any, do you think your support has made in the lives of learners? (Probe: Can you give me an example?)
8. Is there anything else that you would like to share that we have not discussed today?

Appendix C – Photographs from Training Sessions

Contact information:

Advice NI
Forestview
Purdys Lane
Belfast
BT8 7AR
Tel: 028 9064 5919