February 14th, 2008
The Trouble with Information: How students gather and evaluate online resources.
Intro: Increasingly, university students are required to find and synthesize online resources to complete academic assignments. I’m interested in studying the process students use to complete these assignments…where do they start, what are their priorities, where do they go and how much time do they spend on the search process versus the composing process?
To answer these questions, I compared undergraduate and graduate student performance on a writing task that required them to gather information online and briefly respond to a writing prompt.
Drawing on Flowers and Hayes’ (1981) cognitive model of composition, I am studying the gather, evaluation, and integration processes involved in writing academic texts. I’m using an expert-novice comparison to get at differences in source use (Wineburg, 1991). To cast a broad net, I’m using a combination of established qualitative (Coiro, 2007) and quantitative (Azevedo & Cromley, 2004; Brand-Gruwel, et al., 2005; Holscher & Strube, 2000; Lazonder, 2000; Metzger, Flanagin, & Zwarun, 2003) models of studying online literacy practices. I started my study with two main questions:
1) how do experts and novices differ in their overall process when engaging in an online academic research task?
2) which, if any, of these practices predict the quality of the final product?
I define experts and novices by years spent in school. Ideally, I would like to compare faculty and/or researchers with undergraduates, but for this study, my experts are pre-service teachers (graduates) and my novices are first-week freshman (undergraduates).
I use a mediational model that first examines how the two groups differ and then identifies predictors of high performance.
Method:
data collected during Fall quarter 2007
154 participants
65 experts (TEP students enrolled in Copeland’s “Teaching with Technology” course)
89 novices (first quarter freshmen enrolled in Writing 2 and 2E courses)
Procedure and materials:
Held in Phelps computer labs, each session lasted 70 minutes
Participants were first given pre-questionnaire that included questions about domain knowledge and interest, technical skills, and general demographic information. [show sample]
Participants were then given a prompt and told they had 50 minutes to write a 1-2 page response using information they found online. [show prompt]
During the gathering and composing phase, students were told when they had 30 and 10 minutes remaining.
After students submitted their work, they completed a post-questionnaire which included questions about their process (which sites they used, how they evaluate credibility) as well as follow-up questions about domain knowledge and interest. [show example]
Once students left, log files were collected from each computer. [show example] Log files included information about computer actions: how many sites they visited, how many times they revised their search term, how many times they returned to a site, how many links they followed within sites, and keystrokes (e.g., text entries and copy/paste).
Analysis:
Developed rubric to score student written responses. [show rubric] The main challenge was figuring out how to measure use of source materials, specifically, how to measure critical engagement with these materials. Used a combination of counting (quantitative) and holistic (qualitative) scoring.
Findings in progress
I am in the process of analyzing my data. What do you think will be a difference in the way experts and novices begin their search? My hope was that experts would use a database or at least Google Scholar or Eric Digests to begin their search. My preliminary results show that 82% of the expert participants and 72% of the novices started with Google. Less than five participants in each group started with Wikipedia, Dogpile, or ask. One expert started with Eric and one novice started with Google Scholar.
Furthermore, as I start the paper scoring, I’m finding that a majority of participants in both groups used the top five Google results for their first or second search terms. At first, this finding was a bit disheartening, but then I found that the preliminary differences seem to lie in how each group uses the source. For example, experts tend to consider the implications of the source material rather than simply inserting it into their texts.
Participants in both groups use personal experience and observation, but differ in the ways they use it — a few of the experts evaluate their experience in terms of other source materials, while most of the novices use personal experience/observation to make unsupported generalizations.
Future directions:
–Complete data analysis
–Conduct textual analysis using Pairwise to quantify degree of re-mixing (Jenkins, 2007) occurring in student academic texts (compare student texts with web pages they visited).
–Study usage patterns in non-academic settings: task-specific and recreational browsing among different age groups.
Share on Facebook
February 5th, 2008
When I started my dissertation research, I struggled to describe what I was doing. Was I studying online reading, digital literacy, information processing? While the exact phrasing is still up for grabs, I understand the process, the experience I’m studying. I’m studying how students use online resources to complete academic assignments. Specifically, I’m trying to measure the cognitive processes involved in gathering, evaluating, and integrating online sources to compose academic texts. This first step is part of a larger agenda. I am starting with students in a relatively controlled environment to develop baseline measures to later study how this process plays out in other scenarios, such as task-oriented searches or recreational browsing. I’m using a cognitive science model to identify differences in the processes of domain expert and novice students’ approaches to this task. However, this model is just the starting point. I am also blending qualitative methods of textual analysis from the fields of Education and English to look more deeply at the practices behind the process. Specifically, I’m interested in quantifying Jenkins’ theories of re-mixing. In Convergence Culture, Jenkins (2007) describes re-mixing as a process by which users blend a variety of media from multiple sources to create their own work. Basically, this process is what new media is: combining media from multiple sources and modes to create a new whole. Certainly, new technologies fit into this paradigm, but more importantly, how the users appropriate technologies to suit their needs is what interests me. While media theories can certainly describe trends in use, cognitive science provides an actual method of applying and testing these theories.
Share on Facebook
February 3rd, 2008
The Body Shop in Santa Barbara closed this month. I stopped by last week to see movers actually taking out the last of the boxes. Most people may not know, but for a while the majority of Body Shops were franchised, to empower women to become small business owners. Our local Body Shop was franchised by Anita Roddick (she had a house in SB) for, I think the past 8 years, but maybe longer.
I went to The Body Shop in the Ventura mall yesterday, a 30-45 minute drive, because I wanted to support the stores — I want them to continue to exist. When I walked in, I said without thinking, “this doesn’t look like The Body Shop.” In fact, it could have been any mall store. Gone were the trademark green walls. There were no provocative slogans hanging in the windows or on the walls. A sales girl introduced me to their new line “Japanese Cherry Blossom.” I asked if it were community trade and she said no, but the body butter had “organic” olive oil and the unique thing about the product was they used real cherry blossoms. I felt like a small part of me died.
“Made with Passion” is L’Oreal’s new slogan for Body Shop. As a writing teacher, I tell my students to show, not tell. Anita’s Body Shop would never need a slogan like that. Walking into the store, happening upon a magazine ad, or reading the product labels assaulted us with passion. Her little beauty products were world-changing, confronting everything from the plight of third-world workers to women’s distorted body images.
Although I tried Body Shop’s products in the early 80’s, I didn’t start using them on a daily basis until ten years ago. I watched a documentary about a community who produced shea butter for the Body Shop. Community members were paid a living wage, worked in decent conditions, appeared to improve their living conditions, and received health benefits such as dental and medical care. Best of all, the community were cultivating something locally in an environmentally sustainable way that had been cultivated by the same community for centuries. That afternoon, I went to the Body Shop and bought three of their Brazil Nut products.
Every morning, I use the Brazil Nut body butter and I feel like in a small way, I’m doing my part. That’s the beauty of Anita’s work: she allowed all of us to collectively make a difference, just by using her products.
Before curbside recycling programs were widely available in the U.S., Body Shop stores offered a 10% discount on purchases if you brought back used product containers for recycling. I’d save my containers in a Body Shop bag bearing one of my favorite slogans “If you think education is expensive, try ignorance.” Again, whenever I brought the containers in for recycling, I felt like I was part of something much larger.
I’ve always felt empowered to vote with my dollar. As often as possible, I buy local organic foods, support local businesses who support local charities, and generally try to make as many of my purchasing decisions matter in the larger scope. Anita certainly made this easier. As I was preparing to go to Ventura yesterday, I took stock of all of the Body Shop products I use — pretty much 60% of the products I use are from Body Shop.
I think I’ll continue to buy the body butter as long as it remains a community trade item, but the new lines of shower gels and lotions that are “made with passion” plus a lot of non-descript ingredients don’t really seem to uniquely contribute to bettering anything, so now I’m going to seek out up-and-coming lines attempting to accomplish Anita’s goals.
Cause-marketing makes it difficult to distinguish products that are truly benefitting our global community from those that aren’t. Anita pushed the envelope with her campaigns and somehow balanced conscientious production with amazing products. Her team must be incredible, but it does seem that one woman made a global difference.
I highly recommend her books “Business as Unusual” and “Take it Personally” — they’re certainly not books you curl around to relax with a cup of tea. They’re energizing and inspiring.
I think I’m rambling now, but just wanted to pay tribute to the amazing force that was Anita Roddick and the difference she’s made in my life. Goodbye, Santa Barbara Body Shop. Thank you, Anita.
Share on Facebook
