3.3 Evaluation of design

In terms of Engeström’s (2002) model of activity theory discussed in the preceding chapter, the subject of the activity system is taken to be each group of students (be they dyads, quads or groups of eight) which was taken, in turn, twice to the respective field sites (the neighbourhoods) in order to participate in the intervention.  The object in the present activity is seen as the physical and human features in the neighbourhoods (such as reservoirs, quarries, and public housing tower blocks), the spatial and socio-economic relations of which were elucidated upon – into a fuller perspective of understanding the neighbourhood geographically – through participation in the wayfinding and debating (through the Structured Academic Controversy) activities which constituted the intervention.

This ‘fuller geographical understanding’ is expressed in this thesis through outcomes such as the production – by each group of eight – of, inter alia, sketch maps of the neighbourhood, transcripts of exchanges through messaging, the consensus following the Structured Academic Controversy, and the culminating PowerPoint presentation.  The instruments which the students used would therefore include the camera-phone which each pair was issued with, the text- and picture-messages which they sent to each other as they participated in the intervention, and the SIM phone accounts which facilitated the exchange of messages in the first place.

These exchanges of messages, which characterized both the orienteering and Structured Academic Controversy parts of the intervention, took place within a set of rules, which were given during the briefings to the participants.  An example of such a rule would be that participants were not allowed to communicate gesturally to each other should they inadvertently come within line-of-sight during any part of the intervention.  These rules therefore described a clear division of labour, which refers primarily to the division between leading and following pairs (during the first part of the intervention) and that between proposing and opposing teams (during the second part).  Finally, the community in which the activity represented by the intervention would be situated, can be thought of as being that of way-finders, navigators, and other geographical explorers in general.

Vygotsky (1978: 35) elaborates that children, with the help of speech (or, as is the case in the present intervention, with the help of messaging technologies), create “a time field which is just as perceptible and real as the visual-spatial field”.  This very creative ability forms the basis of the orienteering activity in the intervention.  In their attempts to guide the so-called following pairs along the route that they themselves had just traversed minutes earlier, the leading pairs were, in Vygotsky’s (1978: 36) words, “reconstructing the separate activities that were a part of the required operations of the task”.

Concurrent to this reconstruction was the complementary obligation on the leading pairs to extend this ‘time field for action’ forward.  In other words, the leading pairs had to make a record (be it photographic or textual) of salient elements in their present perceptual field, for subsequent communication to the following pair in the not-too-distant future.  As Vygotsky (1978: 36) would have it, “future activity that can be included in an ongoing activity is represented by signs”.

More recently, Cole (Griffin and Cole, 1984; Cole, 1988) has drawn attention to the need to further expand Engeström’s model to take into account contradictions and tensions present within activity systems.  

For example, primary contradictions occur within each element of the activity system, between the element’s exchange value and use value.  For example, in the activity system pertaining to the present intervention, one set of instruments are the SIM accounts, which afford monthly airtime.  Without this airtime, the intervention would not have been possible as no messages would have been able to be sent.  So, on the one hand, participants were actively encouraged during the briefings to send as many text- and picture-messages as they wished, in order to build a rich corpus of discourse for subsequent analysis, yet, on the other, the concern was always there that the monthly airtime quota would be used up prematurely should too many messages be sent before the month was up.

Secondary contradictions occur between elements, as new elements emerge.  For example, in the activity system pertaining to the present intervention, while care was taken during the briefings to set the rule that participants were only allowed to communicate by text- and picture-messages and not by voice-calls (in fact, the phones were configured in such a way to disable the making and receiving of voice-calls), one group hit upon the idea of supplementing their text- and picture-messages with photographs of the sketch maps, while another creatively recorded verbal directions and sent them as audio files.  Decisions had therefore to be taken as to whether such innovations were indeed permissible, given the objectives of the exercise.  

As discussed in the preceding chapter, Wells (2002) argues for a representation of activity theory which recognizes the reciprocal influences that participants in a dialogue have on each other through the text that they co-construct (Figure 1).

This approach is particularly relevant to understanding the interactions that took place in the intervention described in this thesis.  For example, as elaborated upon earlier, the object of the activity facilitated by the intervention was seen as the development in geographical understanding – as manifested by descriptions of, and discussions around, the spatial and socio-economic relations – of the physical and human features in the respective field sites.

In Cole’s (1996) terms, these features – such as reservoirs, quarries, and public housing tower blocks – are at once both material (existing in the physical space as Wartofsky’s (1979) primary artifacts) and symbolic.

In fact, it is this very symbolism invested by the naïve geographies of adolescents into features of the natural and man-made environments (which are understood quite differently by geography teachers and, by extension, professional geographers) that is the cornerstone of the issue under investigation by this thesis.

As Figure 1 above makes clear, this tertiary artifact (Wartofsky, 1979) – the geographical understanding of physical and human elements in neighbourhoods – is an unequal, contested and dynamically evolving space which lies within the respective ZPDs of the subjects.  It is an artifact constructed in part from its constituent secondary artifacts (as represented by the transcripts of text- and picture-messages analysed in subsequent chapters).  Wells (1994), commenting in the parallel but not identical context of writing, draws attention to this dichotomy by calling for a "separation to be made between writing as process and writing as product - between the activity of composing and the text that is composed.  In the activity of composing, the resources of written language are used as a tool to mediate both communication and the thinking that is necessarily involved; the written text that results from the composing process, on the other hand, is an artifact - both a physical object and a fixed representation of the meanings that are made.  But, because this artifact continues to exist beyond the point of utterance, it may become, in its turn, a tool to be used in further processes of knowledge construction and dissemination". (p.6)

In Wells’s terms, the present intervention makes a clear distinction between the activity of composing text- and picture-messages, and the messages themselves.  Through the composition of messages, language is used as a tool to mediate the communication and spatial cognition; the messages themselves are artifacts, both materially and symbolically.  However, because the messages existed digitally in the subjects’ mobile phones even after they had been composed, they became, in turn, tools used in further processes of knowledge construction.

The inequalities depicted in Figure 1 would have been manifested in both phases of the intervention – during the orienteering phase, the so-called leading pairs would, by dint of their having traversed the very route that they were attempting to communicate to the following pairs, more often than not have been in a more privileged position in terms of the discourse; during the Structured Academic Controversy phase, the power relations would have been more dynamic, not least because of the built-in requirement for the teams to reverse perspectives midway through the discussion.

In summary, as the participants in the study explored and shared their findings and understandings about the neighbourhood with each other – through the constitutive dialogue of the multi-modal exchange of text- and picture-messages (in turn, Wartofsky’s (1979) secondary artifacts) – they would have come to a greater appreciation of the geographical context of the nature of land-use issues pertaining to that neighbourhood, both in terms of its present potential for development as well as the prospects thereof.  Table 1 summarizes the list of data types analyzed in the present study.

Boud and Prosser (2002) remind us the characteristics of a “high quality learning environment”.  They describe learner activities which support learner engagement, acknowledge the learning context, seek to challenge learners, and provide practice and feedback.  To what extent does the above design satisfy these criteria?  It is my contention that there was a high degree of learner engagement, since to a large extent the nature of the investigation was driven by the synchronous exchange between the teams themselves. 

Similarly, the tasks were certainly contextualized, both within the syllabus, and more importantly, within ‘real life’.  Getting lost in an unfamiliar neighbourhood is something which almost anyone can identify with.  It is also through this aspect of the design that learners were challenged.

Finally, because there was more than one task to be performed, and teams can further take turns to navigate by both traditional as well as non-traditional means, students were given sufficient opportunities to practise their orienteering skills, inter alia.  Students also derived at least two modes of feedback: first, teams provided each other feedback in real-time, in the field; second, teams would have had the opportunity to receive feedback on their conceptualization of the neighbourhood from their peers, based on the printouts and composition of the tableau, once back in class.  In this way, the design permitted students to reflect upon, and respond to the feedback, during the crafting of the end-product.

The discourse analysis used in the design was proprietary to the study, focusing primarily on the semiotics of space and place among adolescents.  This was because there is a paucity of prior literature and research on the ‘social language’ (as Gee (1999) would express it) of text- and picture-message-based wayfinding.  Instead, studies of adolescent discourse outside of the formal school curriculum (for example, Moje, 2002) have tended to focus on issues of identity-formation and social-bonding, both of which are beyond the scope of the present study.  Put another way, the present study is less interested in how space and the perceptions thereof influence adolescent identities, and much more in the particular elements in space which adolescents find geographically meaningful, so as to better inform teaching practice.

It should be clear by now that the vehicle chosen to shed light on these semiotic constructions was the use of text- and picture-messaging.  This design decision sits well with Lankshear’s and Knobel’s (2004) so-called ‘principle of productive appropriation and extension in learning’, which is one of several principles they advocate should drive ‘pedagogy for i-Mode’ (so named after Japan’s popular mobile internet service).  In this principle, Lankshear and Knobel caution that too often, the relevant knowledge and competencies that learners already possess are either ignored or subverted within classroom learning (for example, handphones are often viewed by teachers as objects of distraction).  As a counterpoint, they suggest that designers of pedagogy should look for ways to reduce or ameliorate conflict between social identities during learning.  Their premise is that if learners already know how to perform certain discursive roles and tasks (for example, composing and sending text- and picture-messages in non-curricular contexts) that can be carried over to new discursive spaces, this can be used to advantage to enable learning and proficiency in a new area (such as using messaging technologies for collaborative exploration during fieldwork). 

This very ‘carrying over’ of discursive roles and tasks to new spaces is the essence of what Kress (2000) terms ‘synaesthesia’, which he defines as ‘the transduction of meaning from one semiotic mode to another’.  Synaesthesia, in turn, inspires the three essential components of described by the New London Group’s (1996) multiliteracies pedagogy, namely the designed, the designing, and the redesigned.

In the context of the present study, the designed (which is the available and pre-existing resources for meaning-making) is represented by the knowledge that the participants in the field-studies brought from the formal geography curriculum; this knowledge was transformed (through the designing – the process of shaping emergent meaning through re-presentation and re-contextualisation) in the course of the two parts of the field intervention, into the redesigned (the outcome of the designing which the meaning-makers (what Halliday (1977) would term the ‘meaners’ – those who mean) have re-made for themselves), which would variously be the multimedia messages and maps constructed during the peer critiques (Mitchell’s (1989) ‘imagetext’ – composite synthetic works which combine image and text), and finally the PowerPoint files used in the group presentations.

More generally, criteria for trustworthiness of findings (Lincoln and Guba, 1985) include credibility, transferability, dependability and conformity.  Credibility was enhanced by observing several procedural precautions.  For example, intensive interaction was employed instead of prolonged engagement, primarily because the duration of the study, from the point of view of each class of students, was only between two to three months.  Rapport with the students (which is a pre-requisite of intensive interaction) was relatively quickly established during the initial student briefing and during the pre-tests (the establishment of rapport was helped first through member checks, and second by the fact that the researcher has had many years of experience teaching adolescents of this age group).  Thereafter, the students were in contact with the researcher on an almost daily basis.

A second precaution taken to enhance credibility was the triangulation of data collection through multiple and varied data sources, such as audio and video recordings of the pre- and post-tests and observations of student activity in the field (during both the orienteering and perspectives tasks) and in class (during the peer critique), and student-authored multimedia documents presenting their collaborative learning experience to their peers.  Because these data came in different formats, and originated in different contexts, features common among them were reasonably representative of the students.  Triangulation was also achieved through an exchange of perspectives of the various helpers involved in the field, as the data was being generated by the students.

Third, member checks were used, by providing the students with a copy of the transcript from their respective perspectives task and permitting the delivery of feedback, both from the subjects themselves, as well as from their peers (during the peer critique).

Fourth, peer debriefing was carried out with two disinterested peers, namely a fellow doctoral student and a colleague with specific experience in implementing messaging technologies in a school context.  Discussions were regular, and focussed mainly on the data and the methodology of the study.  In addition, a peer review by a geography teacher was carried out on the PowerPoint files submitted by the participants.

Fifth, negative case analysis was used in the analysis of transcripts, specifically to establish that the various categories inferred regarding modes of discourse could satisfactorily describe and encompass all the messages sent and received.  This was done through what Glaser and Strauss (1967) term the ‘constant comparative method’, in which a set of categories is continually modified in response to analyses of instances recorded.

Finally, highlighting of the emic (folk) perspective (that is, of popular, everyday descriptions, Garfinkel (1967)) was achieved through ensuring that the messaging was transcribed verbatim and with the unique spelling and grammatical structures associated with typing text messages on a numeric (as opposed to QWERTY) keypad.  Care was also taken to include any discourse which was off-topic.
   
The dependability and confirmability of the study was ensured through the maintenance of a properly managed audit trail / reflexive journal by the publication online of a weblog, which recorded the decisions made and their reasons, questions posed, setbacks and significant events and activities.  Other records available include the afore-mentioned audio-visual recordings, results of the pre- and post-tests, materials from the pilot study, instruments for data analysis, implementation schedules and the logistics of the study, and printed transcripts.