How much do different technology-enhanced learning projects have in common? Delving into the literature cited by the TLRP TEL projects

When I used to work for the TLRP TEL Programme, I had a bit of a side project going to use network analysis to explore the literature cited by the different projects, as a way of pulling out the classic papers and common ground across the programme.

If you’ve not familiar with the TLRP TEL programme, it followed on from the main phase of the Teaching and Learning Research Programme. It was jointly funded by the ESRC and EPSRC, and the main phase ran from 2008 to 2011, during which eight major projects were funded, spanning a range of topics in Technology-Enhanced Learning. These included (thanks to the Internet Archive for links):

  • Echoes: developed a multimodal digital environment to help develop children’s social interactions and support exploratory learning
  • Ensemble: semantic web technologies for case-based learning in Higher Education
  • HapTEL: developed haptic technology for dentistry students
  • Interlife: used Second Life to help support young people to develop social skills and navigate transitions
  • LDSE: A learning design support environment for teachers and lecturers
  • Migen: Intelligent support for mathematical generalisation
  • Personal inquiry: Designing for Evidence-based Enquiry across Formal and Informal Settings of Learning
  • Synergynet: Multi-touch tabletops for collaborative learning

With such a diverse range of technologies and settings, there was logically a question throughout of what it was that brought everyone together; what was distinct about TEL. The programme addressed this by organising events and publications around a range of cross-programme themes – links to the main outputs below:

Another way of exploring the threads that ran through the programme would be to look for commonalities in the literature which the projects cite. This could potentially have formed the basis of a collection of classic works in TEL as a reference bank, a bit like the one that Chris Davies and Rebecca Eynon produced in 2015 (‘Education and Technology – major themes in Education’). I had a first go at this about ten years ago (!), using Touchgraph. It was a bit clunky though, and formatting the data took a while, so sadly it went unfinished. This was before I had discovered Gephi though, and what was a lot of work back then is now a lot easier. (I used this approach on the Openness and Education literature recently, for example).

It has remained something I’d like to have followed-up on though. Being versed in Gephi it is now a lot easier for me to do this, and the time delay at least means that all the papers which were being submitted, reviewed or in press are now available for inclusion.

Getting the data

I couldn’t find my original files, so set about getting the data from scratch. The TEL website is now gone, so I looked up the projects in the ESRC Research Catalogue website. The project pages in the catalogue list a variety of publication types associated with each of the projects. It had been my intention to include the full range of publications in the analysis; however, it quickly became apparent that many of the items were not available online, either having not been online originally or that the links were no longer functional (particularly conference items). As a result, I decided to focus on just the journal papers associated with each project as these were the most consistently available (even a couple of those had disappeared in the relatively short space of time since the end of the programme) (and readers, take note: please use your institutional repositories!).

This yielded a list of 66 target papers across the eight projects. Four could not be located, and a further four could be found, but I wasn’t able to access the full texts. A total of 58 papers were therefore included. From each, the reference lists were copied into Excel, and (where multiple records existed) consistently formatted. The data was exported as a two-column CSV file, of ‘source’ (cited article) and ‘target’ (TEL project paper which cited it) and imported into Gephi.

Exploring the network

The network contained a total of 1,978 nodes (each node being either one of the sampled papers, or any resource cited by them – including other papers, books, chapters, conference items, government sources or websites). When visualising the network, it wasn’t immediately obvious where the bounds of each project lay. If there was nothing in common at all, you would expect to see eight distinct clusters of papers, one for each project. However, in practice, there were varying degrees of overlap (click to view a larger version):

To see where the project boundaries do lie, I colour-coded the sample of journal papers according to projects (I’ve also adjusted the node size the reflect in-degree, to make them stand out a bit more) (again, click to open a bigger version):

(note that the black node was a joint publication, with authors from three of the projects.)

It is quite interesting here to see how closely related the projects were to each other, and also within projects there is a reflection of the extent to which different strands within the projects were writing together or working in parallel.

The nodes that I am really interested in are the ones which represent articles which were cited by two or more of the projects. To narrow it down, all the items which were cited only once can be removed, which dramatically reduces the size of the network (now 315 nodes):

I examined the network to look for examples which met the criteria (cited by at least two of the projects – I didn’t count items associated with the black node though as this had a disproportionate overlap with a paper in one of the three projects it was associated with) and labelled the results:

The result of this was a list of 42 publications (in alphabetical order):

Anastopoulou, S., Sharples, M., Ainsworth, S. & Crook, C. (2009) Personal inquiry: linking the cultures of home and school with technology mediated science inquiry. In Mobile Learning Cultures across Education, Work and Leisure (eds N. Pachler & J. Seipold), pp. 55-58. WLE Centre, London. Proceedings of the 3rd WLE Mobile Learning Symposium, London, 27th March 2009. ISSN 1753?3385.

Anderson, T. & Whitelock, D. (2004) The educational semantic web: Visioning and practicing the future of education. Journal of Interactive Media in Education, 1: 1-15.

Berners-Lee, T., Hendler, J. & Lassila, O. (2001) The Semantic Web. Scientific American 284, 34-43.

Bruner, J.S. (1966) Toward a Theory of Instruction. The Belknap Press, Cambridge, MA.

Cohen, L., Manion, L., & Morrison, K. (2000) Research methods in education (5th ed.). Taylor & Francis Ltd.

Conlon, T. & Pain, H. (1996) Persistent collaboration: a methodology for applied AIED. International Journal of Artificial Intelligence in Education, 7, 219-252.

Dillenbourg, P., Baker, M., Blaye, A. & O’Malley, C. (1996) The evolution of research on collaborative learning. In E. Spada & P. Reiman (Eds), Learning in Humans and Machine: Towards an interdisciplinary learning science (pp. 189-211). Oxford: Elsevier.

Druin, A. (2002) The role of children in the design of new technology. Behaviour and Information Technology 21(1), 1-25.

Engestrom, Y. (1999) Activity theory and individual and social transformation. In Perspectives on Activity Theory (eds Y. Engestrom, R. Miettinen & R.?L. Punamaki), pp. 19-38. Cambridge University Press, Cambridge, UK.

Facer, K. & Sandford, R. (2010) The next 25 years? Future scenarios and future directions for education and technology. Journal of Computer Assisted Learning 26, 74-93.

Fullan, M. (1991) The new meaning of educational change. Cassell, London.

Good, J. & Robertson, J. (2006) CARSS: A framework for learner centred design with children. International Journal of Artificial Intelligence in Education, 16, 381-413.

Greeno, J. (1991) Number sense as situated knowing in a conceptual domain. Journal for Research in Mathematics Education, 22 (3), pp. 170-218.

Kirschner, P., Sweller, J., & Clark, R.E. (2006) Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential and inquiry-based teaching. Educational Psychologist, 41(2), 75-86.

Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hamalainen, R., Hakkinen, P. & Fisher, F. (2007) Specifying computer-supported collaboration scripts. International Journal of Computer-Supported Collaborative Learning, 2(2), 211-224.

Kolb, D. A. (1984) Experiential learning: Experience as a source of learning and development. Englewood Cliffs, NJ: Prentice-Hall, Inc.

Koper, R. & Olivier, B. (2004) Representing the learning design of units of learning. Educational Technology and Society 7, 97� 111.

Kraemer, K., Dedrick, J. & Sharma, P. (2009) One laptop per child: vision vs. reality. Communications of the ACM, 52 (6), pp. 66-73.

Landauer, T. (1995) The trouble with computers: usefulness, usability, and productivity. MIT Press, Cambridge.

Laurillard, D. (2002) Rethinking university teaching: A conversational framework for the effective use of educational technology, 2nd ed., London: Routledge.

Lave, J. & Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation . Cambridge: Cambridge University Press.

Lewins, A. & Silver, Christina (2007). Using software in qualitative research: A step-by-step guide. London: Sage.

Merrill, M.D. (1994) Instructional design theory. Englewood Cliffs, NJ: Educational Technology Publication.

Moss, J., & Beatty, R. (2006). Knowledge building in mathematics: supporting collaborative learning in pattern problems. International Journal of Computer-Supported Collaborative Learning, 1, 441-465.

Mulholland P., Collins, T., Gaved, M., Wright, M., Sharples, M., Greenhalgh, C., Kerawalla, L., Scanlon, E., and Littleton, K. (2009). Activity guide: an approach to scripting inquiry learning. In Proceedings of 14th International Conference on Artificial Intelligence in Education, Brighton 6-10 July 2009.

Naismith, L., Lonsdale, P., Vavoula, G., & Sharples, M. (2004). Literature review in mobile technologies and learning, NESTA Futurelab Series.

Noss, R., Healy, L. & Hoyles, C. (1997) The construction of mathematical meanings: connecting the visual with the symbolic. Educational Studies in Mathematics 33, 203-233.

Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas. Basic Books, New York.

Piper, A., O’Brien, E., Morris, M., & Winograd, T. (2006) SIDES: A cooperative tabletop computer game for social skills development. Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work, 10. ACM.

Roschelle, J. (2003). Unlocking the learning value of wireless mobile devices. Journal of Computer Assisted Learning, 19(3), pp. 260-272.

Roschelle, J., & Pea, R. (2002). A walk on the WILD side: How wireless handhelds may change computer-supported collaborative learning International Journal of Cognition and Technology, 1(1), 145-168.

Schon, D. (1987) Educating the reflective practitioner: Toward a new design for teaching and learning in the professions, San Francisco: Jossey-Bass Publishers.

Schon, D.A. (1983) The Reflective Practioner: how professionals think in action. Temple Smith, London.

Schwartz, D.L., Brophy, S., Lin, X. & Bransford, J.D. (1999) Software for managing complex learning: examples from an educational psychology course. Educational Technology Research and Development 47, 39-59.

Sfard, A. (1998) `On Two Metaphors for Learning and the Dangers of Choosing Just One’, Educational Researcher 27(2): 4-13.

Sharples, M., Taylor, J., & Vavoula, G. (2007). A theory of learning for the mobile age. In R. Andrews, & C. Haythornthwaite (Eds.), The Sage handbook of e-learning research (pp. 221-247). London: Sage Publications Ltd.

Slotta, J.D. (2010). Evolving the classrooms of the future: The interplay of pedagogy, technology and community. In K. M�kitalo-Siegl, F. Kaplan, J. Zottmann, & F. Fischer (Eds.) Classroom of the future. Orchestrating collaborative spaces (pp. 215-242). Rotterdam: Sense.

Vygotsky, L.S. (1978) Mind in society. The development of higher psychological processes. Cambridge, MA: Harvard University Press.

Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23.

Wenger, E. (1998) Communities of Practice: Learning, Meaning and Identity. Cambridge: Cambridge University Press.

Wingate, U. (2007), A framework for transition: supporting learning to learn in higher education. Higher Education Quarterly, 61 (3), pp. 391-405.

Yin, R.K. (2003) Case study research: Design and methods. London: Sage Publications.

There were a couple which I was slightly surprised did not make the cut (Larry Cuban, oversold and underused, and Ton de Jong on computer simulations). The list includes a few methodological texts (e.g. Yin, Cohen et al.) and some which are perhaps slightly too specialised, but I think that overall anyone as a student or newer researcher in TEL would benefit from reading or being aware of most of the literature surfaced by this approach.

The other advantage of time passing between the TEL programme and now is that it would also be possible to look at onward citations of the projects’ papers, which could be interesting to see how the topics addressed by the TEL projects have changed over time and where they sit in relation to current research agendas in TEL.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: