|Concordance lines from COCA.|
In this blog post I comment on Alex Boulton's 2010 article from the journal of Language Learning, "Data-Driven Learning: Taking the Computer Out of the Equation." I presented on this article with my roommate Nick Velde for an SLA course at NAU. I find this article to be relevant to my own teaching context, where many graduate TAs teaching in our university's IEP have been heavily influenced by research and courses in corpus linguistics, which is a specialty of our faculty here. There is often a desire to then apply corpus linguistics research via data-driven learning to the classroom so that students can inductively learn about patterns of language in their actual contexts.
Background on data-driven learning
While this constructivist approach has its critics, there are many proponents of it as well. Boulton points out in his article that one of the arguments for Data-Driven Learning (DDL) is that it can lead to greater learner autonomy. He describes DDL as a process where "learners are not taught overt rules, but they explore corpora to detect patterns among multiple language samples" (p. 535). A benefit of this learning process is that it is likely to promote a number of cognitive skills like predicting and interpreting. However, critics have questioned whether the investment of time, money and resources results in a justifiable payoff. In the studies reviewed in his article, Boulton points out that much of the research on the effectiveness of DDL involves small-scale qualitative studies. And for the few quantitative studies that have been performed in this area results have thus far been small with relatively few statistically significant outcomes. Much of this research has to do with students preferences and responses to DDL and does not answer the question of overall effectiveness of the approach.
An additional issue is that much of the research demonstrates that DDL is often used with more advanced learners, implying that it is of limited effectiveness for lower level students. Boulton points out that this may not be the case and that a few studies have shown DDL can be effective to the same degree that it has been with advanced learners. But research with lower level learners seems to be lacking at this point. This research gap is likely due to the assumption that the cognitive burden that comes with this type of DDL will be much harder for lower level learners to process than higher level learners.
Boulton makes the argument that paper-based corpus materials might help to lessen this cognitive burden of using a new software, a new learning approach and new materials. By using selected concordance lines in paper-based materials for classroom instruction, he posits learners will be provided with they type of scaffolding and guidance that they need, especially at the lower levels. He proposes the use of paper-based materials as an alternative to hands-on concordancing techniques, where learners interact with the concordance lines via computer in computer labs, which, Boulton notes, is the dominant approach to DDL taken by most of the studies he reviews.
Boulton's study involved 89 students enrolled in a 2nd year architecture school in France. Nearly all the students were French and were non-native English speakers. He was able to collect 71 questionnaires following an experimental session. The students in his study "had compulsory 90-min English classes once a week in groups of 15–20, but none had any prior experience of DDL" (p. 542). Based on the student's TOIEC scores, they are described as low-intermediate learners.
He chose language items of difficulty from student's in-class essays. Then 10 items were chosen to produce paper-based concordance materials for. 1-5 items were used for part of the group and 6-10 for the other participants as a control group.
The teaching materials Boulton used in his study consisted of "two short booklets covering the same language items for all students, but reversing the traditional and DDL treatment for each" (p. 543). He also describes how the materials included "two introductory questions for each item followed by a single page of either corpus data or dictionary information, interspersed with specific questions to focus attention" (p. 543).
These materials were given to teachers who had no prior experience using DDL methods, a point the researcher felt distinguished this study from previous ones where the researcher is also the teacher. A pre- and post-test were given in addition to the questionnaire.
Results showed only a 19.38% improvement on the post-test. Results from a one-way ANOVA showed that a minimal significant difference between the dictionary items and the corpus items (p = .15). The results of the questionnaire showed that 51 students preferred to do more DDL work whereas 28 students wanted to do more dictionary activities. 58 students felt that corpus work would help them avoid certain errors in the future as opposed to 37 who felt dictionary work would allow them to do this better.
An open ended question asked students whether they preferred paper-based learning to hands-on instruction. The response to this question is perhaps what I find most interesting in this study. Boulton reported that "nearly half (25 of the 55 who responded) believed that prepared exercises would get straight to the point and avoid wasting time, and teacher guidance would be essential to avoid drawing wrong conclusions from the mass of data" (p. 553). I think this provides some insight on how learners might view the hands-on approach to DDL verses the paper-based approach Boulton used in his study. It gets at a sort of cost verses benefits ratio, which I think is important for considering the implementation of a DDL approach.
One of the other interesting results of the study is that the lower-level learners did show improvement on some of the language items. This runs counter to the perception that advanced learners benefit more from DDL than lower-level learners.
Take away message for teachers?
The conclusions that Boulton draws at the end of is article are that the use of paper-based materials should be used to help scaffold learners to use hands-on DDL. He also suggests that the elimination of the computer makes learning easier. However, these paper-based materials are very time consuming to prepare, which threatens the practicality of such an approach. Then there is the issue of teacher expertise using online corpus tools. The technical knowledge required to manipulate and query corpus data might limit the accessibility of such an approach to a relatively small group of teachers. It appears that their is still much more research needed in this area. But I find this article does contribute to DDL approaches and gives me something to consider in terms of scaffolding instruction using concordances.
Boulton, A. (2010). Data-Driven learning: Taking the computer out of the equation. Language Learning, 60 (3), pp. 534–572.