Chapter 4 Specific Ed Tech

VAR and Learning Analytics

Although higher education and professional sports are obviously different worlds, both ed tech and video-assisted refereeing are concerned with the application of technology to fundamentally human enterprises, with the intention of improving them for those involved. Witnessing the rollout of the VAR technology at the men’s and women’s Football World Cup tournaments in 2018 and 2019, and in the UK Premier League for the 2019–20 season, provided some possible lessons for the application of technology in education, in particular the use of large data sets to analyze student behaviour.

Let’s examine first the history of VAR. In 2012, the Royal Netherlands Football Association (KNVB) set about trying to use technology to improve decision making in the game with a project called Refereeing 2.0 (KNVB, n.d.). In the 2013–14 season, it piloted the use of technology such as Hawk Eye, which had already been deployed successfully in cricket, to assess whether a ball had fully crossed the goal line. The pilot also developed the use of fifth and sixth officials who could examine video evidence and communicate with the on-pitch referees.

With detailed television coverage and mobile phone footage from the crowd, the use of video to support the referee in football seemed to be inevitable. As the president of the International Football Association (responsible for the rules of the game) put it, “with all the 4G and Wi-Fi in stadia today, the referee is the only person who can’t see exactly what is happening and he’s actually the only one who should” (qtd. in Medeiros, 2018). It was hard to argue against the implementation of video technology when every refereeing decision was being dissected in minute detail via television and social media. VAR had good intentions, namely, to eliminate an increasing number of obvious errors. The technology involves the use of video footage analyzed by the video assistant referee in a separate video suite. That referee relays information back to the on-field officials. The technology is now in use across most professional leagues, although the exact guidance on when and how it is used can vary.

On a positive note, there are aspects of VAR that really do help and have improved the overall game. Goal line technology, for instance, has removed the infuriating experience of disallowed goals when a ball has clearly crossed the line. However, the intersection of precise technology with dynamic and imprecise activity in football has led to incidents in which the technology provides a false sense of confidence about aspects not reducible to minute measurements. VAR decisions in which a ball has brushed a hair on someone’s hand and is deemed a handball, or in which a player is ruled offside by a fingertip, might be correct technically, but in reality the game and the rules were not developed to be so finely measured. Fans are increasingly frustrated as a seemingly good goal is subjected to forensic analysis and eventually disallowed. The application of such fine measurement to a human enterprise seems to be a mismatch, like using molecular changes in the brain to describe poetry. It can be done, but it rather misses the point. As Farry (2020) puts it, “football has always been a game defined as much by human error . . . [as] by human skill,” and VAR threatens that dynamic.

Turning to learning analytics in education, we can see a number of parallels. Learning analytics can be defined as “the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens, 2013, p. 1382). Because students spend a good deal of time in virtual environments, and because most education systems (e.g., the library, attendance, and student records) generate data about them, HEIs now possess a wealth of precise data about a student. It can include how long they spend looking at a resource, the number and average length of posts that they put on an online forum, their performance across computer-based assessments, how often they access library resources, and so on. This quantity of data can lead us to believe that we can measure a student’s comprehension of a subject to a fine degree, but as with football learning is much more inexact than the measurements might suggest. As with VAR, we can be misled into thinking that the precise measurement is significant rather than the overall quality. As with obvious errors in VAR, possessing rich and accurate data can better inform our decisions, but they need to be implemented sympathetically with a holistic view of the enterprise (be that football or education).

Another point of comparison is that VAR forces us to re-evaluate the role of humans in the system. Arguably, the application of technology in cricket has been more advantageous, with Hawk Eye and an established video review system to support increasingly complex decisions for umpires. Here the technology is implemented within a framework in which its role is to support the human decision makers. Similarly, learning analytics can be used to help an educator identify whether a student is struggling, whether a particularly tough part of a course is causing students to revisit materials often, or whether certain resources are not being used. In the online course delivery world, this type of data is the equivalent of detecting puzzled faces, stifled yawns, or stares out the window during a lecture, and the educator can make adjustments accordingly. As with VAR, though, there is the danger that learning analytics make the data the most important aspect, and that the decision could be made by an AI system, just as teaching could be deemed a task for AI. Implementing technology with the aim to support rather than replace actors in the field is a key principle that we can extract.

With VAR, we are also seeing how the technology changes the behaviour of humans who make decisions. If referees and umpires know that video technology will catch misdemeanours, then maybe they will be less likely to give fouls, or maybe they will trust their own judgment less. Johnson (2020) reports that by January 2 in the UK Premier League 2019–20 season a total of 63 refereeing decisions had been overturned by VAR. This was bound to have an effect on the people whose decisions were publicly reversed in this manner. Similarly, an educator might not trust their own judgment about a student if the technology tells them otherwise and if they know that they are being monitored on the basis of the data. A possible consequence of the quantity of data mentioned previously is that only data-driven decisions are trusted.

Perhaps most tellingly, much like a lot of ed tech, including learning analytics, VAR has not really solved the problems that it set out to eliminate, at least in the manner that people envisaged. Football fans are now in the strange position of thinking that VAR is ruining the game but do not want it scrapped. There was an increasing desire for video technology to be applied to football to solve incorrect offside decisions, missed penalty calls, and goals that should have been disallowed. The belief was that, if video-assisted refereeing was in use, then all of these problems would disappear. The introduction of VAR alleviated some of these complaints, but it also introduced a whole new set of issues, so now there are arguments about whether decisions should have been referred to VAR and whether the fine calls mentioned above should have been given. The debate has just moved the location from the pitch to the review room. Football games are still not the controversy-free utopia that many envisaged, and arguments about and dissatisfaction with rulings have probably only increased.

It is difficult, then, to say ultimately that the introduction of VAR has been worthwhile. Similarly, in education, the introduction of technology such as MOOCs, AI, or blockchain is often touted as solving problems of equity, access, scale, or efficiency. For instance, MOOCs were meant to democratize education by making courses free to all. But the lack of tutor support in a MOOC has meant that it is best suited to experienced, confident learners. This is indeed what a demographic analysis of MOOC learners has revealed (Christensen et al., 2013), with the result that, far from democratizing education, MOOCs might actually increase inequality. Although most forms of ed tech find a suitable audience and purpose, invariably they cost more than anticipated and do not have the global impacts touted at their inception.

VAR highlights that we should view technology as part of a broader system. It is likely that the implementation of VAR will improve, but that will happen not through technological development but through a more sympathetic and nuanced set of guidelines for its usage. The same is true of learning analytics and other implementations of ed tech. The technology needs to be understood as part of the wider educational context and not a solution in itself.

Overall, though VAR improves some decisions, there is the possibility that it dehumanizes aspects of football. Our enjoyment in watching a sport is precisely that it is not an exact science: it is unpredictable, sometimes chaotic, and conducted by people. That is what makes it worth returning to. Technology can certainly improve it, but its application needs to be cautious, and our expectations for its results need to be measured, for it will not lead to a sporting nirvana devoid of errors. Accepting the messiness of sport is part of its inherent appeal, and so it is with education. Although education is more controllable and perhaps predictable, it is still an exploit that we undertake because it connects to some very human aspect of self and identity. As with VAR, the role of technology in education is largely inevitable—we are not returning to a time without ubiquitous video in sport or one without the internet in education. It can also be potentially beneficial; however, as we are seeing with video technology in sport, it is to the detriment of the overall enterprise if it becomes the main focus.

Blockchain and Alchemy

Few technologies have excited as much attention without having a direct application for most people as “blockchain.” It seemed that there was no problem to which blockchain was not the solution. In this section, I examine this form of techno-solutionism and why blockchain represents a type of technology attractive to many.

But first some explanation of the technology. A blockchain is formed from a database shared across a network of computers. The network is public but encrypted, so when an update is made to the database, such as a new transaction, it is automatically updated across the network. This distributed nature makes it difficult to hack since any hacker would need to make changes across the network. Cryptocurrencies such as Bitcoin use blockchain to create a ledger that holds the records of Bitcoin transactions. The lack of a central location storing this database makes it secure and ideal for online, peer-to-peer transactions.

Around 2017, people began to suggest that it could have applications in education. In a review of its possible applications in education, Grech and Camilleri (2017) proposed four possible areas of impact:

1. A system for certification: records of achievement could be securely stored via blockchain and expanded to include credit transfer and recognition of informal learning.
2. Verification of validity: users can automatically check the validity of certificates without the need to contact the organizations that originally issued them.
3. Ownership of data: users could have increased ownership of and control over their own data, which would reduce data management costs for universities.
4. Cryptocurrency payments: institutions and individuals can use cryptocurrency payment methods, which could enhance grant or voucher-based funding models.

Similarly, Fagan (2018) reported on several university pilots and start-ups experimenting with blockchain approaches for credentialing and recognizing competency-based achievements, and the University of Bahrain announced that it was using blockchain to provide all students with a digital record of achievement (Galea-Pace, 2019).

Beyond these rather niche applications, there is a broader tendency to promote blockchain as a mystical solution to all manner of problems. For instance, in 2018, Chancellor of the Exchequer of the United Kingdom Phillip Hammond suggested that it was the means to solve the potential border issue with Ireland should the United Kingdom leave the European Union: “I don’t claim to be an expert on it but the most obvious technology is blockchain” (quoted in Cellan-Jones, 2018, para. 3). How blockchain would solve this problem and far larger social ones was not made clear. It was a magical solution.

Maintaining this aura of magic is not accidental. Blockchain, after all, is a solution that will be sold by providers, and transparency and understanding are not always in their interests. In an analysis of 43 blockchain applications, Burg et al. (2018, para. 5) found “no documentation or evidence of the results blockchain was purported to have achieved.” None of the reported solutions was willing to share data, results, or processes. The authors concluded that, “despite all the hype about how blockchain will bring unheralded transparency to processes and operations in low-trust environments, the industry is itself opaque” (para. 6).

Blockchain can be seen as the latest instantiation of a recurring theme in ed tech that can be termed “technology as alchemy.” “Alchemy” is a term often used to imply a magical solution, and a search for its application in scholarly articles reveals titles such as “Genetic Alchemy,” “The Alchemy of Finance,” “Computational Alchemy,” and “The Alchemy of Asset Securitization.” It is used loosely and metaphorically in most of these instances to imply a beneficial but secret and mysterious process of transmutation that is part science and part art. Holmyard (1990) divides alchemy into two parts: exoteric, concerned with preparing or discovering the philosopher’s stone, with its power of transmuting base metals into precious ones, and esoteric, devotion and mystical practice that lead to eternal life and in which the transmutation of base metals is merely symbolic (although no doubt convenient).

There is much that is interesting in the history of alchemy, including its origins in ancient China, its relations to Christianity and Islam, its influence on literature (Shakespeare and Chaucer made references to alchemy as well as its presence in the Harry Potter series), and the modelling of an early ecological thinking (Wilson et al., 2007). All of these perspectives, and many more, make alchemy a topic of interest in its own right. However, this analogy focuses on a more negative interpretation and its relation to the history of chemistry. The pursuit of unlimited precious metals in particular dominated experimentation in chemistry for centuries and reappeared in different cultures and at different times. Although it had mystical and religious elements for many, the dogged pursuit of alchemy was also characterized by the following:

• Greed: unlimited wealth awaited the successful alchemist.
• Obfuscation: alchemy persisted through rumour and secret formulas, adding to its allure. The process was never made public.
• Magical lexicon: this obfuscation worked not only by being secretive but also by creating a language difficult to penetrate.
• Vagueness: although the ultimate aim of producing gold was clear, it was accompanied by vagueness regarding other benefits, including immortality, spiritual awakening, and improved health.
• Occasional side benefits: almost inevitably, given the time devoted to it, there was the occasional chemical breakthrough that occurred as a side benefit of alchemy, such as the discovery of phosphorus.
• Persistence despite the lack of results: although there was no success in transmuting base metals into gold, people persisted, and indeed this complete lack of success was only seen as a reason to continue. Succeeding where others had failed represented an irresistible challenge, and some of the best scientific minds (e.g., Isaac Newton) were involved in this largely fruitless pursuit.

Although blockchain is not as elusive as alchemy, there are similarities to how it is sold and portrayed. Blockchain is by no means alone in employing an alchemical mindset in its promotion—proponents of AI, learning analytics, and automatic assessment can all be said to deploy similar tactics. From the perspective of blockchain, we can consider the similarities to the list of aspects of alchemy:

• Greed: the global education market is estimated at $6 trillion annually, and selling a universal solution across all providers linked to their most treasured asset (accreditation) would provide significant returns.
• Obfuscation: it is frequently made obscure by commercial interests with black box algorithms. As the study above highlights, they report questionable results that are difficult to verify and do not share their data.
• Magical lexicon: it has its own lexicon of algorithms, ledgers, and encryption that increasingly looks like magic to outsiders.
• Vagueness: there is often a vagueness about improved efficiency, learner agency, lifelong learning, and so on. The four potential impacts suggested by Grech and Camilleri (2017) indicate some of these ill-defined possible benefits, such as improved efficiency in institutions’ data management systems.
• Side benefits: perhaps not accidental, but amid all of the investment, it is likely that there will be some practical advantages of blockchain that will be over-reported. For example, instant access to trusted digital certificates without the need to contact institutions will benefit refugees whose original paper certificates might have been lost or destroyed.
• Persistence: Watters (2013a) talked of “zombie ideas” in ed tech that just refuse to die. Automatic tuition and micro-credentialing are among these ideas, and blockchain represents the latest technology to offer solutions for them.

This is not to suggest that blockchain cannot be successfully implemented and possibly solve specific issues that provide real benefits for learners. The objection here is to the overblown claims and the often-unspoken alchemical tradition that persists in ed tech, of which blockchain is merely the latest realization. The effective way to combat this is through openness (of data, algorithms, claims, and results), focusing on specific problems to address (instead of grand revolutions) and bringing a critical perspective to any “magical” solution.

The alchemical tradition is founded partly upon a lack of transparency and partly upon being a new, complete solution. Such approaches therefore exhibit the type of historical amnesia that besets much of ed tech. For instance, DeMillo (2019) writes about “How Blockchain Technology Will Disrupt Higher Education”: “It will do so by solving a problem that few of us realized we had: There is no reliably efficient and consistent way to keep track of a person’s entire educational history. That is why a worldwide effort is underway to use blockchain technology to tame the internet so that it can become a universal, permanent record of educational achievement.” This is in fact a problem that many people in education recognized and indeed one that they thought they had solved with eportfolios. The benefit of blockchain, DeMillo claims, is that it will open up what we recognize as assessment: “Students are more than transcripts and test scores. The college transcript is a 19th-century invention that has little to do with the educational institutions and workplaces of the 21st century.”

We can compare this with how Beetham (2005, p. 3) summarized the benefits of an eportfolio, which

• provide evidence of an individual’s progress and achievements
• [are] drawn from both formal and informal learning activities
• are personally managed and owned by the learner
• can be used for review, reflection, and personal development planning
• can be selectively accessed by other interested parties, e.g., teachers, peers, assessors, awarding bodies, prospective employers.

That has a lot of resonance with what DeMillo (2019) says that blockchain can deliver. Arguably, eportfolios have not been as successful as they could have been, but some of the issues in their uptake are related not to the technology but to the context within which they operate. For instance, employers generally say that they would like to have a complete portfolio of an applicant’s work, but they tend to fall back on CVs and interviews. Similarly, eportfolios require assessment in universities to be reshaped so that they are based upon discrete tasks more usefully added as stand-alone pieces of evidence.

Blockchain might represent a better way of achieving this result, but an article declaring how it will change the method of assessment should at least acknowledge the existence of eportfolios. The questions that it should be answering are how will blockchain do it better, and how will it overcome the problems that a decade or more of eportfolio work has not managed to address? The problem with an alchemical mindset is that these questions seem to be mundane compared with the fantastical offering proposed.

As with alchemy, the danger of blockchain is that there will be wasted time, effort, and money in the pursuit of an unattainable goal instead of focusing on smaller, achievable ones. In alchemy, once experimenters stopped trying to produce gold, they went on to discover elements, invent medicines, and create all manner of new materials that could be used every day. As educational technologists, then, we should always be wary of any technology that has the whiff of alchemy about it, and the traits above provide a useful checklist against which to review any technological solution.

MOOC Metaphors

Massive open online courses can be regarded as the educational technology that garnered the most interest over the past decade, starting with experiments by the likes of George Siemens and Stephen Downes; seeing large-scale investment and media attention, with 2012 being declared the year of the MOOC (Pappano, 2012); engaging in a wealth of associated research (Veletsianos & Shepherdson, 2016); and settling into more practical forms of application. As such, they also attracted a lot of metaphorical thinking as people struggled to understand what they were in relation to conventional higher education. Were MOOCs education’s “MP3” (Shirky, 2012), broadcast (McAndrew & Scanlon, 2013), rhizomes (Cormier, 2008), or a shop window (Wakefield et al., 2018)? I will consider two metaphors, one from Downes and one of my own, as a means of exploring how metaphors help us to think about a new technology.

VLE/LMS

The virtual learning environment (VLE) or learning management system (LMS)—largely synonymous, with LMS favoured in the North American context—is in many ways the default educational technology. It grew out of the elearning boom at the end of the 1990s, when many institutions were deploying a mixture of technologies to implement online learning. An online course back then might have combined websites for content, a third-party tool for computer conferencing, in-house tools for submitting assignments, an open-source piece of software for quizzes, and so on. Although this range of options meant that the environment could be tailored to the educator’s needs, pedagogical requirements, or just simple preferences, it could be confusing for students navigating different configurations across multiple courses. Also, setting up and working with these various tools required both a certain degree of technical knowledge (even to have conversations with IT people who might do the hard work) and an interest in doing so. Many academics, justifiably, are just not interested in ed tech; they have their own domain knowledge to focus on, and ed tech can be a distraction or impediment.

Enter the VLE, which provided a neat collection of the most popular tools in one package, with a uniform interface, and links to other university systems, such as registration, student records, the library, and content management. This enabled a consistent experience for students across the university, uniform staff development, and centralized IT support. By the mid-2000s, the VLE was a mainstream part of nearly every university’s infrastructure. Its rise was dramatic, but many who had been involved in elearning in the early years criticized its uniformity—every course was now the same, and part of the experimentation with technology and associated pedagogy had been lost, as we saw in the section “Rewilding Ed Tech.”

In this section, I look at some of the metaphors that we apply to VLEs. The first ones can be found in the very terms that we use to refer to them. A learning management system has implications of control and expands existing terminology, such as “content management system.” Learning and by extension learners are managed in this system like resources or content. The term “virtual learning environment” possibly has a more expansive connotation, an environment being something that people can explore and in which they can spend time. Given that most of the commercial LMSs derive from North America, where that term dominates, perhaps this language has shaped the development of the technology. Table 1 provides some views of knowledge and instruction, and it can be argued that LMS maps onto the view of knowledge as “a quantity or packet of content waiting to be transmitted,” whereas VLE represents the belief that “a person’s meanings are constructed by interaction with one’s environment.” If so, then the terminology might have gone some way toward shaping the development of the tools.

A common way to think of VLEs is as tools in a toolbox. This metaphor brings to mind a tradesperson, such as a carpenter or plumber, selecting the appropriate tool for the task from the familiar toolbox. Although this is a potentially useful framing, it suggests that the educator has a similar level of expertise regarding elearning tools, but in fact many educators will not have used those outside the VLE. It is as if the tradesperson were gifted a toolbox on her 18th birthday and never upgraded it. An actual toolbox is something that the individual adds to based upon experience and preference—the contents of one person’s toolbox are unlikely to be the same as another’s, but everyone’s VLE is the same.

In essence, people tend to find VLEs rather dull. This is partly because they are made to meet a standard need—in this they are like that other oft-criticized but widely used tool in education, PowerPoint. But what the VLE and PowerPoint have in common is that they were in the first wave of digital democratization tools. Such tools cannot be too far removed from traditional practice, or people simply will not adopt them. So, they provide a useful stepping stone toward a more digitally enhanced future. The issue with both is that for many they represent not a potential stage on a journey but the endpoint. Their ease of use and similarity to existing practice are seductive in this sense; they do not suggest or require much change in existing educational practice.

Thus, we have boring courses in VLEs and boring, bullet-pointed presentations in PowerPoint. There is nothing intrinsic in the tools that means boring is the only possible outcome—good presenters will have excellent PowerPoint presentations, and good teachers will have excellent VLE courses. Yet there is something about their proximity to standard practice that means the end result is all too often uninspiring.

Some proponents of VLEs will suggest that one version is superior, but in reality the differences among VLEs are small. Moodle, for example, is often described as a constructivist VLE, and Canvas has proponents because of its easy use. They are not all the same, but there is a tendency to overemphasize their differences. The point of a VLE is to provide a uniform collection of tools. This similarity is more significant than any difference.

The impact of VLEs is largely the same, whichever version is adopted. The problem with VLEs, like PowerPoint, lies not with the technology itself but with how institutions adopt such technology. VLEs are a considerable investment in terms of licences, resources, and time. Often they cannot be changed in and out given this investment. So, what happens is that institutions develop administrative structures and processes couched in terms of the specific technology. All of these processes can be viewed as sediment building up around an object, like something dropped in the mud. As more sediment accumulates around it over the years, it becomes harder to dislodge.

In this section, I have proposed three metaphors associated with VLEs: a toolbox, PowerPoint, and sediment. As with MOOCs, VLEs are a technology that has seen considerable application of metaphors to aid their implementation, uptake, and debate. Because of this prominent position in the ed tech landscape, VLEs and their associated metaphors are an interesting case study of how we think about education itself. In Table 1, see below, Wilson (1995) proposed that underlying metaphors for knowledge would shape how we viewed instruction.

This underlying view will then shape the metaphor that you apply to a VLE. Farrelly et al. (2020) reviewed the different metaphors associated with VLEs in publications such as journal articles and blog posts. They proposed the following six categories of VLE metaphor.

• Straitjacket: in this metaphor, the VLE is seen as constraining the educator, for instance by reference to silos.
• Behemoth: this category suggests that constraints arise from the nature of the VLE “industry” and associated processes, so the sediment metaphor above would be an example.
• Digital carpark: in this concept, the VLE is characterized largely as a repository or content dump rather than a place of potential learning and interaction.
• Safe space: this category covers a range of metaphors highlighting the way that VLEs provide a supportive environment, such as “security blanket.”
• Smorgasbord: this category suggests that VLEs offer a wide variety of choices in terms of functionality. The toolbox might be an example of such a metaphor.
• Pathfinder: in this concept, VLEs act as pioneers for further technology and practices, an example being Trojan horse metaphors.

Even if the labels used here are not ones that you identify with, they indicate that how people talk about VLEs is nearly always couched in a metaphor. That metaphor is used either to express a viewpoint about VLEs (often critical) or as a means of thinking about how to implement them. Either way, the chosen metaphor, and perhaps more significantly its resonance with others, will influence how the technology is perceived. The VLE is arguably the central ed tech of the digital era, and similarly how metaphors are used to explain it represents the most cogent example of metaphors in ed tech.