Skip to main content Skip to navigation

Department of Computer Science

Reflections on making construals as a new digital skill

Meurig Beynon, Jonathan Foss, Elizabeth Hudnott, Steve Russ, Chris Hall, Russell Boyatt, Emma King
University of Warwick UK
wmb@dcs.warwick.ac.uk

Hamish Macleod, Jen Ross
University of Edinburgh UK
h.a.macleod@ed.ac.uk

Dimitris Alimisis, Emmanouil Zoulias, Rene Alimisi
Edumotiva Greece
info@edumotiva.eu

Erkki Sutinen, Ilkka Jormanainen, Carolina Islas, Andrés Moreno
University of Eastern Finland Finland
ilkka.jormanainen@cs.uef.fi

Piet Kommers
Helix-5 The Netherlands
pkommers@gmail.com

Peter Tomcsanyi, Michal Winczer
Comenius University Slovakia
tomcsanyi@slovanet.sk

Abstract

Making construals is a practical approach to computing that was originally developed for and by computer science undergraduates. It is the central theme of an EU project aimed at disseminating the relevant principles to a broader audience. This has involved bringing together technical experts in making construals and international experts in educational technology in the organisation of wide-ranging learning activities. This paper, which complements the more detailed tutorial on making construals in (Beynon et al. 2015), documents issues that have been raised in the encounter.

Keywords

construal; computing; spreadsheets; educational technology; school education; constructionism; open educational resources; educational games; software development

Figure 2

Figure 1: A basic Shopping construal

1. INTRODUCTION

Making construals is a new digital skill that can be seen as complementary to ‘writing computer programs’. The key characteristics of making construals are introduced in (Beynon et al. 2015), a tutorial paper in which the use of JS-EDEN, a prototype online environment for making construals (the “MCE”), is illustrated by making an elementary construal of shopping activity. The MCE and a basic Shopping construal are depicted in Figure 1. As discussed and illustrated in (Beynon et al. 2015), many variants of the Shopping construal can be developed with relatively modest computing expertise. These include traditional educational resources that might be used to scaffold learning in teaching ‘life-skills’ and simple educational games.

Making construals is a by-product of a well-established research project (Empirical Modelling 2015) that was first developed for – and by – computer science undergraduates at the University of Warwick. It is currently the theme of an EU Erasmus+ project (“CONSTRUIT!”) aimed at dissemination to a wider audience. Adapting resources that were originally conceived for computing specialists for this purpose has entailed a significant shift in emphasis. Though the technical development of the MCE and the construals described in (Beynon et al. 2015) is being led by members of the CONSTRUIT! team at Warwick, it has been informed through dialogue with expert international consultants in educational technology and by several collaborative project activities, including workshops with schoolchildren at SciFest in Joensuu, Finland and with schoolteachers in Athens. This companion paper to (Beynon et al. 2015) summarises the diverse issues that have been raised in this process. These summaries do not necessarily represent a consensus view of the consultants, but inform an agenda for subsequent discussion.

The principles of making construals will first be briefly outlined with reference to Figure 1 and Figure 2 below (for more details, see (Beynon et al. 2015)). In Figure 2, the construal is an interactive digital artefact that offers an experience to its maker. As in the appreciation of a work of art, the construal has no formal semantics – its significance is determined by the way in which it is associated in the experience of its maker with a referent. By way of illustration, the Canvas window in Figure 1 depicts a construal of a shopping scenario as from the perspective of a customer. In this window, the images refer to items of fruit that are for sale and have been selected for purchase and to coins held in the purse or tendered in payment by the customer. The experiential connection between the construal and its referent is derived from a correspondence between entities that the maker can observe in each. With reference to Figure 1, in addition to the entities that are patently observable in the Canvas window (such as the coins in the purse, or items for sale), the Shopping construal incorporates entities such as the total cost of the items selected for purchase, and the amount of money that has been tendered as observables. As illustrated in Figure 1, the status of these observables can be inspected within the MCE via the Observable List. As in a spreadsheet, the values of certain observables are defined by dependency in terms of others. For instance, whether the customer has enough money to buy the currently selected items will depend on whether the entire contents of their purse cover the total cost.

Figure 2 depicts the relationship between construal and referent as it is experienced by the maker as of one moment in time. The maker’s understanding of this relationship, which gives the construal its informal meaning, is built up through interaction with the construal and its referent as these evolve over time. This understanding incorporates expectations about the impact of an interaction that are based on knowledge gained from prior experiment. The maker can in principle change observables in an unconstrained fashion, but in practice only makes such interactions as help to establish and refine understanding of the relationship between the construal and its referent. Refinement will typically involve exploring the perspective of different agents. In the Shopping construal, the customer is not in general able to fix the price of the items for sale for instance – this is the prerogative of the shopkeeper. The integrity of the construal relies on certain assumptions about the stability of the context for interaction. For instance, we would not buy perishable fruit by mail order, and in some cultures the price of items would be open to negotiation. The semantic framework that most appropriately applies to the making activity depicted in Figure 2 regards all semantic relationships as pragmatically determined and rooted in the connections that can be made in the personal experience of maker. As elaborated in (Beynon et al. 2015), this is the fundamental tenet of William James’s radical empiricism.

2. ORIENTATION ON SOFTWARE DEVELOPMENT

Making construals establishes a close link between how we think about a subject domain ('how we construe a situation') and how we might develop software to simulate behaviours and address problems drawn from that domain. In this respect, it represents a new way in which to realise program-like behaviours. When we write a program, its behaviour is prescribed in accordance with a specification. The specification need not be preconceived; it may be fluid and evolving. But – though it may be implicit – it is always present, if only (as in an agile development) as 'what the software does now'. A construal by contrast is an informal representation of the playground for agency within a situation as we currently conceive it. In the construal, there are counterparts for the patterns of agency and interaction that we encounter through experience and experiment. And where the interactions with a program are constrained by a user-interface that is closely aligned with its specification, the ways of interacting with and interpreting a construal are open-ended and informed by what 'makes sense' in the maker's imagination.

Figure 1

Figure 2. Making a Digital Construal

Previous papers have built on these ideas to argue that making construals is a ‘radical new conception’ in educational technology (see e.g. (Beynon 2007)). Educationalists, having seen so much hype around the latest technologies, have a proper scepticism about such claims to novelty. Their wariness may reflect the influence of traditional computer science thinking: from a perspective informed by elementary exercises in programming, developing software can be conceived as ‘implementing algorithms to fulfil a preconceived formally specified functionality’, and there is an abstract mathematical sense in which any two correct implementations are equivalent. In reality, orthodox thinking about software is very ill-suited to themes such as constructionism (Papert 1980; Beynon & Harfield 2010), adaptable open educational resources (Wikipedia 2015) and transformative practices in online teaching (Ross et al. 2013; Beynon & Zhu 2013). An aspiration for the CONSTRUIT! project is to give an account of computing that does fuller justice to these well-conceived and important educational initiatives and supplies a more appropriate grounding for their practices.

For the present, the idea of conceiving a foundation for software in anything other than pure computational terms is so far from mainstream that it has little visibility. When exposed to construals at project events, some computing specialists feel compelled to find out what lies behind the script, as if only a computational account of a construal could be meaningful (cf. an examiner who insisted on working from the code of the exam spreadsheet). Novices likewise, having made much effort to train themselves in computational thinking, may find it difficult to approach the computer as a source of experience rather than an engine for implementing recipes for action.

The insistence on deconstructing the personal experience of connection that is depicted in Figure 1 by expressing it in computational terms exactly parallels what James opposes through his radical empiricist stance:

“… the parts of experience hold together from next to next by relations that are themselves parts of experience. The directly apprehended universe needs, in short, no extraneous trans-empirical connective support, but possesses in its own right a concatenated and continuous structure.” The Meaning of Truth (James 1975), Preface: p.xiii.

For further elaboration of the notion that, however essential the role of writing code, software development must necessarily be grounded in lived experience, see (Beynon 2012). (The relevance of a Jamesian philosophical stance was quite independently identified by the eminent software pioneer Peter Naur (Naur 1995)).

This is not to suggest that, in its present immature state, the technology for making construals is well-suited to serious software development. But the general principle that coding should in some way connect directly with experience is important. On the one hand, as Dourish (Dourish 2014) observes, certain aspects of code – such as the pervasive use of databases – have a material impact on our everyday experience. On the other hand, how the abstract computational ingredients of coding are realised in experiential terms is not in general explicitly expressed in the code itself. A part of the code might embody the model-view-controller pattern for instance, so that it implicitly establishes an indivisible link between the ‘view’ and the ‘model’ such as is expressed in a definitive script. Nor should the primary concern here be the user-experience: in general, the interaction between computational agents in a software system has to be construed and orchestrated in terms that have first to be conceived experientially. It would be of interest to explore to what extent these issues are addressed in accounts such as Paul Ford’s What is Code? (Ford 2015) and David Berry’s The Philosophy of Software (Berry 2011). But the rationale for making construals is much more in keeping with Chris Granger’s critique of coding as ‘not the new literacy’ (Granger 2015).

3. THE PEDAGOGICAL PERSPECTIVE

There are many different ways in which construals can contribute to learning about a domain. Making a new construal of your own can be a way of learning, as can interacting with an extant construal. As will be illustrated in section 5 below, which considers learning how to make construals, the skills required for making a new construal and for exploring an extant construal are very much the same. The focus here is on how we learn about a domain or a referent by making or adapting construals.

Making construals encourages the learner to adopt an unusual stance. Much technology to support learning is associated with bodies of knowledge that have been well-documented and formalised. The learner’s interactions are staged within a frame that has been carefully prepared, possibly even so that there is an intended ‘correct’ path to follow. The archetypal context for making construals is one in which the learner is invited to act with greater autonomy. Making the connection between a construal and a referent that is depicted in Figure 2 is a personal matter that draws on different aspects of the learner’s experience. The concern is not only with what the learner directly observes but with its resonance with everything else to which they are sensitive. And whilst the learner is led to identify observables and dependencies in the role of a model-builder, this is with a view to appreciating the subtlety and richness of their immediate experience of the domain more fully. In sharp contrast, the objective for traditional model-making is to find simplifying mechanisms that can be applied to the referent to serve the specific goal in hand. What is abstracted from the referent in this way is in effect detached from the domain.

In analysing how students learn to program in languages such as Lisp, Prolog, Smalltalk and Logo, three aspects of experience are significant:

  • prior experience of implementing algorithms;
  • what is being directly experienced in interacting with the tutorial environment;
  • what intuitions have been acquired through experience of the subject domain.

In evaluating the quality of the learning experience, it is the way in which these different kinds of experience are brought together that counts. In particular, these three ingredients of experience guide the learner in making the link between familiar and new concepts.

In making construals, the above aspects of the learner’s experience have counterparts that are respectively associated with the modeller’s understanding, the evolving construal and its emerging referent. A distinctive feature of the MCE is the degree of active control it gives over the way in which these elements are brought together in the learner’s experience (cf. the way in which visualisations of states and views of observables can be freely configured). This encourages the student to actively control and optimise the balance between prior and new ("restructured") concepts.

Learning to make construals entails a reorientation that presents a challenge of somewhat the same kind that is associated with adopting a declarative rather than a procedural approach to programming. Not all students will welcome this reorientation, but it potentially has unique value for learning. This stems from the emphasis on helping the learner to rethink topics that might initially appear trivial. This is particularly well-illustrated in the large body of project work that has been developed using the previous variants of the EDEN interpreter (Empirical Modelling 2006; Empirical Modelling 2013) and has yet to be replicated in JS-EDEN. By way of illustration, consider the construals of heapsorting (Beynon 2008a), elementary group theory (Beynon 2008b) and the Clayton Tunnel railway disaster (Chan & Harfield 2008).

4. THE TECHNOLOGICAL PERSPECTIVE

The relationship between construals and other established technologies is complex. Many technologies appear to be addressing closely related agendas and delivering superficially similar products.

Making construals is significant as paradigmatic for a particular way of exploiting computer-based technology. Where computational thinking focuses on activities for which the archetype is ‘a mind following a rule’ (Hodges 2002; Turing 1936), making construals is concerned with ‘a person making sense of a situation’. The transition from informal personal accounts to formal objective understanding is characteristic of such sense-making. Deriving programs from a construal, as illustrated in (Beynon et al. 2015), is just one instance of how making construals enables this transition to take place seamlessly. In keeping with the notion that “The directly apprehended universe needs, in short, no extraneous trans-empirical connective support” the distinction between the informal and formal components in this transition is in the mind of the maker. In one context (cf. Figure 1): ‘this is what reliably seems to be the case’ and, in another: ‘this is what we can confidently deem to be the case’.

The transition from the empirical to formal perspectives is prominent in many settings. It has a critical role in expert systems (Jackson 1998) and design science (Hevner et al. 2004). Working in close proximity to the formal perspective attracts particular interest because of aspirations for automation and AI. By comparison, the human-centred activities associated with making construals serve a potentially less welcome role – making use of computing to provoke us into thinking more deeply.

Polya’s reflections on teaching mathematics (Polya 2014) help to highlight the unusual qualities of construals. As Polya observes, though mathematics is done inductively, it is taught deductively. This can make the subject boring, alienating the learner from the researcher. As an environment that can enable the learner to cope with uncertain and chaotic elements, making construals gives valuable support for exposing the empirical roots of formal results. In keeping with Polya’s remarks concerning the importance of reflection and review for learning, the history of interactions with a construal also provides an as-if-live record of the processes of rationalisation and problem-solving.

The narrative of making construals is likewise closely parallel to that of design science (Hevner et al. 2004). The learners are developing construals (“design artefacts”) and thus constantly relating their work to emerging demands – perhaps from their own imagination (their “local contribution” in design science terms) and finally, and probably throughout the process, adding to the science (“global contribution”). This is a plausible narrative, but makes challenging demands where the nature of the design artefacts is concerned. In (Hevner et al. 2004, p.89), Hevner discusses a mode of describing the design artefact that is based on searching for solutions to a family of design constraints. He then observes that in general finding such a solution is computationally infeasible and proposes to address this by relaxing constraints. Approaches of this nature, which operate with formal specifications and use sophisticated algorithms, give nothing like the delicate control over details of design and nuances of meaning that making construals affords. As in other contexts, claims concerning the character and quality of methods become rhetorical in the absence of appropriate representations.

Basic principles of ‘making construals’ have featured in practice in several fields. They have been directly exploited to some degree by Jormanainen and Sutinen (Jormanainen & Sutinen 2012) in the context of monitoring teaching in a robotics lab. There are clear resonances with the applications of spreadsheets in education (Baker & Sugden 2007). As discussed in (Beynon et al. 2002), they have also – in effect – been advocated as a basis for expert systems (cf. (Rothenfluh & Gennari 1996)).

The problematic aspect of incorporating the principles of making construals alongside more orthodox representations is that this creates a fundamental ontological conflict. Though the maker may retain the prerogative to say ‘this is what we can confidently deem to be the case’ this is of no consequence if the integrity of the enclosing environment is predicated on the assumption that it is in some absolute sense the case. This helps to explain why merely adding dependency as an extra feature in an otherwise orthodox programming environment introduces problematic semantic issues (cf. (Roe & Beynon 2007; Tomcsanyi 2003)).

One of the objectives for CONSTRUIT! is to enable people other than computing specialists to make their own software applications. This is in the spirit of Mark Hatch's Maker Movement Manifesto (Hatch 2014), which promotes 'making' as an activity that is accessible to all. Hatch's focus is on making physical things, but computing and its associated technologies have a significant implicit role. As he observes: "The tools of making have never been cheaper, easier to use, or more powerful." ... "It may take some practice to get good at some kinds of making, but technology has begun to make creating easy enough that everyone can make.". Curiously, a better understanding of technology for computing is required if Hatch’s thesis is to extend to making virtual products.

By way of example, consider the Shopping construal discussed in (Beynon et al. 2015), as depicted in Figure 1. As illustrated in Figure 1, making construals – in the MCE in its present form – must still engage with traditional coding. Claims for the accessibility of making construals are only plausible if the procedural components (cf. functions / agents in Figure 1) can take a simpler form than they do in general in traditional programming. There are interesting parallels to be drawn with approaches that have been developed to teach programming (cf. (Victor 2012b; Victor 2012a; Boyatt et al. 2014)). Moreno et al (Moreno et al. 2013) seek to aid the learner’s comprehension by introducing code visualisation and animation. This at first sight resembles the situation depicted in Figure 2, where the referent is a program, the visualisation is a construal, and the maker is the learner.

In fact, it is very challenging to cast the animation scenario in terms of making construals. The connection between two behaviours – an animation and a program execution – cannot be directly experienced in the same way that the state of a spreadsheet-like construal is connected with a situation. This suggests that the connection in experience is made state-by-state between the running animation and the executing program. A state-by-state inspection of the program and animation in execution can be staged by running the animation tool at a very slow speed, but it is then unclear what would be meant by experimental interventions on the part of maker of the kind that might correspond to redefinition of observables subject to dependencies in the construal and referent.

The above discussion highlights the fact that, though it is quite possible to execute arbitrarily complex procedural code in the MCE, this is in general conceptually problematic when making construals. Procedural code nonetheless has its place in the definition of functions and actions. For instance, in the Shopping construal, automating the action of buying the contents of the basket is a natural extension of the manual process of repeatedly selecting a coin from your purse at random until ‘offered’ exceeds ‘totalcost’. Such a procedure should – it is to be hoped – be simple enough for the maker to express. If not, it can potentially be automatically generated.

5. LEARNING HOW TO MAKE CONSTRUALS

The focus for CONSTRUIT! research over the first year has been on showing that comprehending and making construals is potentially accessible to the non-specialist. The discussion of the Shopping construal in (Beynon et al. 2015) illustrates the key principles and techniques involved. Our previous experience of teaching undergraduate computer students to make construals (cf. (Empirical Modelling 2005)) suggests that novices typically begin by exploring an existing construal that is connected with their chosen topic of investigation, and develop their skills through making adaptations. In some cases, this leads them to take ownership of the construal through making their own extensions. In any event, once students have developed the ability to understand and extend someone else’s construal, they have the basic competence to start making their personal construals from scratch.

As illustrated in the derivation of a simple game from the Shopping construal in (Beynon et al. 2015), if we have some explicit extension, modification or specialisation of an existing construal in mind there are three stages in the adaptation:

  • conception: without reference to the precise syntactic form and structure of their definitions, draw up a plan for the conceptual reorganisation of observables. This may entail re-disposing or replicating visual components, revising existing observables and actions, and possibly introducing new observables and actions.
  • identification: identify the precise names and definitions of the observables and agents to be adapted through open-ended interaction with the construal. This activity exploits the range of viewers that are available in the MCE – such as the Observable- / Function- / Agent- / Symbol-Lists and Dependency Maps. Domain knowledge may also be helpful in guiding the search activity.
  • reconfiguration: having identified the relevant observables and agents, modify their definitions according to the plan drawn up at the conception stage.

The most important consideration in this process of comprehension and adaptation is that the transition from the initial construal to the revised construal takes place in the maker’s stream-of-thought. That is to say, all interactions with the construal, whatever their status (exploratory interactions, revisions, tests, experiments, corrections, aberrations, etc) are conducted with reference to the live construal and its real or imagined referent as depicted in Figure 2.

In practice, the above sequence of stages would not necessarily follow a strict sequence. Prior identification and exploration of observables and agents in the construal might be required in order to make a plan for revision. In an ambitious development there would also be many phases of the conception-identification-reconfiguration sequence. The MCE provides support for generating the current script (to be saved in the external file system) and for recording intermediate states which the maker can restore if needed.

Introducing the three aspects of conception, identification and reconfiguration is helpful in framing the agenda for future work on learning how to make construals.

In the conception phase, the activity somewhat resembles the activities that have been conceived as ways of teaching computer science principles and concepts without using the computer (cf. ‘computer science unplugged’ (CS Unplugged 2015) and ‘Barefoot’ (Barefoot Computing 2015)). In effect, describing computing activities in non-technical human terms abstracts away the technically sophisticated apparatus that enables machines to manipulate and ‘interpret’ real-world observables and dependencies that a human interpreter takes for granted. This shifts the focus of attention from the purely technical to high-level issues more central to good design practice in computing. By way of illustration, consider how the adaptation of the Shopping construal to allow many instances of each item in (Beynon et al. 2015) highlights a limitation in the representation of the stock of items - it does not allow the maker to record the identities of individual items.

In the identification phase, a key idea is that – whatever the name of an observable – its significance can be established empirically through interaction. For a construal such as Shopping that is conceived as a primary source for many variants, it is helpful to introduce an inventory of observables in the form of a dictionary. The empirical nature of the underlying semantics makes choosing observable names and defining their meanings challenging – consider the ambiguity in the meaning of the observable ‘purse’ in the Shopping construal for instance. An important principle in revising construals is that refinement should preserve the cues that make connections in the stream-of-thought. For instance, one revision of the initial Shopping construal involved introducing more appropriate names for observables (so that ‘bill’ became ‘totalcost’ for instance). Revising the construal by renaming the observable ‘bill’ broke the links between Shopping and other construals that had already been derived from it, as these had referenced ‘bill’. It is more appropriate to make such a revision by introducing a definition of ‘totalcost’ as a synonym for ‘bill’.

Reconfiguration demands a repertoire of skills that can be developed and supported as an independent technical computing activity. In teaching such skills, it will be important to show how to model observables, dependency and agency using definitions, functions and actions. An important principle is that functions and actions, though specified in JS-EDEN in procedural code, can be derived through automating sequences of definitions that are first rehearsed manually. In teaching such techniques, standard processes can be highlighted. For instance, a common form of experiment entails the sequence of actions:

   retrieve and submit the current definition of X
   repeat {
      edit the definition of X and resubmit
      if further experiment is required {
   	recall the original definition of X from history
   	resubmit the original definition
      }
   }

A presentation environment integrated within JS-EDEN also enables definitions of observables to be embedded in the presentation. These can be automatically invoked by the learner either to be directly submitted through the Input Window or presented as templates to be edited prior to submission.

6. CONCLUSION

Ways of thinking about computing that are not based on learning to code (cf. (Granger 2015)) are particularly important in the contemporary world. Making construals has promise as an alternative approach to computing that can be more accessible and intelligible to the non-specialist. Addressing the issues discussed in this paper has an important part to play in unlocking its full potential.

ACKNOWLEDGEMENTS

We are much indebted to Tim Monks, Nick Pope, Antony Harfield and Joe Butler for their work on the development of the JS-EDEN environment. We wish to thank Veronica Durie and Maxim Mosgovoy for helpful feedback. We are also grateful to Mike Joy for acting as the Coordinator for CONSTRUIT!. This project has been funded with support from the European Commission under the Erasmus+ programme (2014-1-UK01-KA200-001818). This publication reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

REFERENCES

Baker, J. & Sugden, S., 2007. Spreadsheets in Education – The First 25 Years. Spreadsheets in Education (eJSiE), 1(1). Available at: http://epublications.bond.edu.au/ejsie/vol1/iss1/2 [Accessed July 8, 2015].

Barefoot Computing, 2015. primary computing Barefoot Computing. Available at: http://barefootcas.org.uk/ [Accessed July 8, 2015].

Berry, D., 2011. The philosophy of software. Code and mediation in the digital age. London, UK: Pal …. Available at: https://wiki.sfu.ca/fall11/cmns353d100/images/6/6e/Berry_What_is_Code.pdf [Accessed July 8, 2015].

Beynon, M., 2007. Computing technology for learning - in need of a radical new conception. Journal of Educational Technology & Society. Available at: http://www.jstor.org/stable/jeductechsoci.10.1.94 [Accessed July 8, 2015].

Beynon, M., 2008a. empublic: heapsortBeynon2008. Available at: http://empublic.dcs.warwick.ac.uk/projects/heapsortBeynon2008/ [Accessed July 8, 2015].

Beynon, M. et al., 2015. Making construals as a new digital skill: dissolving the program - and the programmer - interface. In International Conference on Interactive Technologies and Games (iTAG). Nottingham, UK: IEEE.

Beynon, M., 2012. Realising software development as a lived experience. Proceedings of the ACM international symposium on New ideas, new paradigms, and reflections on programming and software, pp.229–244. Available at: http://dl.acm.org/citation.cfm?id=2384613 [Accessed July 8, 2015].

Beynon, M., 2008b. Web EDEN: Groups of order 8. Available at: http://www.dcs.warwick.ac.uk/~wmb/webeden/Group8OpenDayNov2008.html [Accessed July 8, 2015].

Beynon, M. & Harfield, A., 2010. Constructionism through construal by computer. In Constructionism 2010. Paris, France. Available at: http://eprints.dcs.warwick.ac.uk/242/ [Accessed July 8, 2015].

Beynon, M., Rasmequan, S. & Russ, S., 2002. A new paradigm for computer-based decision support. Decision Support Systems. Available at: http://www.sciencedirect.com/science/article/pii/S0167923601001403 [Accessed July 8, 2015].

Beynon, M. & Zhu, H., 2013. Constructivist Computing - a New Paradigm for Open and Distance Learning? In Proceedings of the 25th ICDE World Conference. Tianjin, China.

Boyatt, R., Beynon, M. & Beynon, M., 2014. Ghosts of Programming Past, Present and Yet to Come. In Proceedings of 25th Annual Psychology of Programming Interest Group Annual Conference 2014. Brighton, UK, pp. 171–182.

Chan, E. & Harfield, A., 2008. empublic: claytontunnelChanHarfield2005. Available at: http://empublic.dcs.warwick.ac.uk/projects/claytontunnelChanHarfield2005/ [Accessed July 8, 2015].

CS Unplugged, 2015. Computer Science Unplugged. Available at: http://csunplugged.org/ [Accessed July 8, 2015].

Dourish, P., 2014. No SQL: The Shifting Materialities of Database Technology?: Computational Culture. Computational Culture. Available at: http://computationalculture.net/article/no-sql-the-shifting-materialities-of-database-technology [Accessed July 8, 2015].

Empirical Modelling, 2015. Empirical Modelling. Available at: http://go.warwick.ac.uk/em [Accessed July 8, 2015].

Empirical Modelling, 2006. Projects archive. Available at: http://www2.warwick.ac.uk/fac/sci/dcs/research/em/projects/ [Accessed July 8, 2015].

Empirical Modelling, 2013. Web Eden. Available at: http://www2.warwick.ac.uk/fac/sci/dcs/research/em/software/webeden/ [Accessed July 8, 2015].

Empirical Modelling, 2005. Web-EM-1. Available at: http://www2.warwick.ac.uk/fac/sci/dcs/research/em/publications/web-em/01/ [Accessed July 8, 2015].

Ford, P., 2015. What is Code? | Bloomberg. Bloomberg. Available at: http://www.bloomberg.com/graphics/2015-paul-ford-what-is-code/ [Accessed July 8, 2015].

Granger, C., 2015. Coding is not the new literacy. Available at: http://www.chris-granger.com/2015/01/26/coding-is-not-the-new-literacy/ [Accessed July 8, 2015].

Hatch, M., 2014. The maker movement manifesto, McGraw-Hill Education. Available at: http://cds.cern.ch/record/1643837 [Accessed July 8, 2015].

Hevner, A.R. et al., 2004. Design Science in Information Systems Research. MIS Quarterly, 28, pp.75–105. Available at: http://www.redi-bw.de/db/ebsco.php/search.ebscohost.com/login.aspx?direct=true&db=buh&AN=12581935&site=ehost-live.

Hodges, A., 2002. Alan Turing. Available at: http://plato.stanford.edu/entries/turing/ [Accessed July 8, 2015].

Jackson, P., 1998. Introduction to expert systems 3rd Ed., Addison Wesley. Available at: http://www.osti.gov/scitech/biblio/5675197 [Accessed July 8, 2015].

James, W., 1975. The Meaning of Truth (1907) New Ed., Harvard University Press. Available at: https://books.google.com/books?hl=en&lr=&id=FVRvjj7LRAQC&pgis=1 [Accessed July 8, 2015].

Jormanainen, I. & Sutinen, E., 2012. Using Data Mining to Support Teacher’s Intervention in a Robotics Class. In 2012 IEEE Fourth International Conference On Digital Game And Intelligent Toy Enhanced Learning. IEEE, pp. 39–46. Available at: http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=6185578 [Accessed July 8, 2015].

Moreno, A., Joy, M. & Sutinen, E., 2013. Roles of animation tools in understanding programming concepts. Journal of Educational Multimedia and …. Available at: http://www.editlib.org/p/40533/ [Accessed July 8, 2015].

Naur, P., 1995. Knowing and the Mystique of Logic and Rules, Springer Science & Business Media. Available at: https://books.google.com/books?hl=en&lr=&id=oY5qVymjaW4C&pgis=1 [Accessed July 8, 2015].

Papert, S., 1980. Mindstorms: children, computers, and powerful ideas. Available at: http://dl.acm.org/citation.cfm?id=1095592 [Accessed July 8, 2015].

Polya, G., 2014. How to Solve It: A New Aspect of Mathematical Method: A New Aspect of Mathematical Method, Princeton University Press. Available at: https://books.google.com/books?hl=en&lr=&id=X3xsgXjTGgoC&pgis=1 [Accessed July 8, 2015].

Roe, C. & Beynon, W., 2007. Dependency by definition in Imagine-d Logo: applications and implications. Available at: http://eprints.dcs.warwick.ac.uk/252/ [Accessed July 8, 2015].

Ross, J. et al., 2013. manifesto for teaching online on WordPress.com. Available at: https://onlineteachingmanifesto.wordpress.com/ [Accessed July 8, 2015].

Rothenfluh, T. & Gennari, J., 1996. Reusable ontologies, knowledge-acquisition tools, and performance systems: PROTÉGÉ-II solutions to Sisyphus-2. International Journal of …. Available at: http://www.sciencedirect.com/science/article/pii/S1071581996900178 [Accessed July 8, 2015].

Tomcsanyi, P., 2003. 39. Implementing object dependencies in Imagine Logo. Available at: http://matchsz.inf.elte.hu/colabs/Porto/pubs/TP.pdf [Accessed July 8, 2015].

Turing, A., 1936. On computable numbers, with an application to the Entscheidungsproblem. J. of Math, 5, pp.345–363. Available at: https://people.cs.umass.edu/~immerman/cs601/TuringPaper1936.pdf [Accessed July 8, 2015].

Victor, B., 2012a. Inventing on Principle. CUSEC 2012. Available at: http://worrydream.com/#!/InventingOnPrinciple [Accessed July 8, 2015].

Victor, B., 2012b. Learnable Programming. Available at: http://worrydream.com/LearnableProgramming/ [Accessed July 8, 2015].

Wikipedia, 2015. Open Educational Resources. Available at: https://en.wikipedia.org/wiki/Open_educational_resources.