Archive by Author

Some Thoughts on the Development of a DBR Argumentative Grammar

This week’s introduction to design based research (DBR) hosted by Rafi Santo and Dixie Ching has kicked my brain gears into a real grind mode. Last night, I jumped into a Twitter chat about the challenges of coalescing an “argumentative grammar” for DBR, but I still have a lot of questions about what such a grammar could look like, and its role in studying and reforming education (and education research). As the DBR community struggles to define this grammar, here are some of my thoughts/concerns:

Since DBR often (always?) involves a process of cooperation between researchers and practitioners (teachers, school admins, other stakeholders) designing interventions, there is a need for a common language that both researchers and practitioners understand and can use to converge on shared goals, interpret events, and augment/alter the design of an intervention as needed. I’ve found that in my work with practitioners (and as one myself), the theories and evidence that help structure an intervention are often inaccessible to those outside of the research community. As a result, this can create tensions around power (e.g., seeing interventions as top-down, rather than grass-roots), feelings of alienation, and even resistance. Obviously, these can have a huge effect on the success and scale of an intervention – especially if practitioners perceive a threat that they are being treated as “guinea pigs” rather than as partners. As researchers, I think this means we need to find ways to talk about our research in ways that are humanizing, give credence to the lived experiences of practitioners, and generate points of entry for practitioners to see how learning/educational theory relates to their work.
An argumentative grammar is necessary for any genre/approach/framework of science (social or natural) for many reasons, and so developing an argumentative grammar is one way for DBR to establish its legitimacy among other paradigms of scientific research. I worry a little that in an attempt to prove the robustness and validity of DBR as a “hard science,” we will lose touch with practitioners by building theories and methods that leave little room for them to feel central to the research-design process. I’m not trying to be pessimistic – I just want to highlight this as a concern for the DBR community to consider.

I’m sure more thoughts/concerns will emerge as the DML Commons DBR “course” unfolds over the next several weeks – and I certainly do not have answers at the moment for any of these questions. But I DO love that the DML community is bringing these issues to the fore, and making a real attempt to make these conversations accessible to a wide range of participants.

Charting my way to and through design-based research

As we’ve kicked off this first week of the DML Commons design research course, I’ve been reflecting on the evolution of my own relationship to design-based research (DBR). It’s a fairly short amount of time, all said, that I could even say that I’ve had this relationship – I’ve been formally designing learning experiences and environments since 2006, started collaborating with educational researchers in earnest in 2008 and then headed off the graduate school in 2010. Over these years my relationship to DBR has of course deepened (and certainly become more explicit and intentional), but I think part of what I see when I look back is how my early experiences working as a practitioner in an informal learning organization dedicated to social justice continues to inform how I think about and engage with design-based research approaches today.

My first proper job was at a fantastic organization called Global Kids, where I had the luck to have Barry Joseph as my boss. For anyone who knows Barry, it’ll be no surprise that a lot of educational design work I did was highly experimental, utilizing emergent new media (games, virtual worlds, social media, mobile technology, digital mapping, badging, podcasting, you name it, we tried it) to figure out how it might be leveraged to support learning and engagement around social justice issues. We often didn’t know in advance exactly what kind of learning might be supported through different types of explorations, but we had hunches about affordances and we were good at learning along the way, documenting emergent best practices (design principles, if you will) for leveraging emergent forms of new media for learning. Global Kids was a dynamic place partly because we did so much experimentation, but also because we took the time to document best practices and design principles, something I would later learn is central to design-based research approaches.

At the same time as we focused on experimentation, at Global Kids we also focused on how theories around digital learning and literacies coming from the academy might intersect with what we were doing on the ground. The most significant example of this for me was the collaboration I ended up working on with Henry Jenkins (then of MIT, now USC) on Project New Media Literacies in 2008 and 2009. As part of the collaboration, I designed a curriculum and associated afterschool program called Media Masters that focused on new media production, social justice, and new media literacies. The project aimed to explicitly take the theory around these new literacies that Henry and his colleagues had identified as an “in the wild” phenomenon and figuring out how to create activities, participation structures and collaborative, production-oriented digital projects that would allow youth to engage in and reflect on these literacies. The collaboration with Henry’s team showed me the promise and potential of using theory, in this case theory about emergent competencies, as a driving force behind doing principled and intentional design. Again, I was engaging in practices central to design-based research without quite knowing it.

Eventually, the many opportunities I had to collaborate with learning researchers while at Global Kids led me into the world of research myself. I headed to grad school to study learning sciences at Indiana University, a place that felt right for me because of its simultaneous valuation of learning theory, research methods and design. It was during grad school that I was ‘formally’ introduced to DBR, and the possibilities of what it might look like to design in a way that was guided by principled inquiry; one where particular theories and conjectures about learning could be indexed (or embodied, as Bill Sandoval would say) within a given educational design.

As I went through my program, I moved deeper into the literature on DBR and began to engage in new design projects within the context of my graduate assistantship, I was able to more critically assess and consider issues at the intersection of research, design and theory development, and issues of who’s involved in this intersection and how they interact. I spent a lot of energy thinking through a variety of limitations I was seeing within the DBR paradigm, even as I was seeing and experiencing its potentials. Some of these limitations were related to its ability to generate theory, and what sort of theory it could produce (Cobb et al. 2003 [pdf] talk about “humble” theory as a product of DBR). Others were related to the reality that while DBR had been actively proposed and refined back in the late ’80s/early ’90s, looking back from my seat in the early 2010’s I wasn’t seeing major projects that used these methods and actually seemed be making big impact in education (something that others in the literature had noticed as well). I was seeing issues relating to both DBR’s promises regarding basic research as well as those related to applied research (the intersection of which is known as “Pasteur’s Quadrant”, oft-cited in the DBR lit). Finally, I was deeply unsettled by how educators were often positioned in the DBR literature. Lots of language I was seeing disempowered educators – issues of “fidelity of implementation” and “lethal mutations” that occur when the original vision of an innovation is lost or somehow ‘corrupted’ by an adopting educator. As a practitioner myself, I felt irked. And I felt like there must be a way to resolve these tensions.

I thought and wrote about these issues during my qualifying exams, and while I won’t go into the whole logic of what I wrote (for those interested, here’s the paper [pdf]), I essentially argued that the most impactful and viable uses for DBR are versions of the methodology that “double down” on hyper-locality of both the theories generated and innovations developed and where practitioners are the ones leading what problems get addressed and subsequent innovations designed. While I didn’t go deeply into it in the paper, the implications were that researchers might play different roles than those that had been usually implied in the central corpus of the DBR literature to date. Rather than leading and being the source of innovations that practitioners then adopt, I saw researchers more playing playing roles related to facilitation, measurement, analysis/theory generation and generally providing technical assistance and capacity to a process deeply rooted in practitioner innovation and agency. As someone who never stopped identifying as an education practitioner, it felt like these issues of practitioner agency needed to be central in order for DBR to be a framework that was going to be meaningful to me.

It’s only in the latest phase of my work that I’ve had an opportunity, and set of collaborators, that would allow me to figure out how to actually make these ways of working happen on the ground. The work I’ve been doing with Dixie Ching, Kylie Peppler and Chris Hoadley in Hive Research Lab over the past two years has turned into the space where the commitments and ideals I put on paper during my qualifying exams are being born out through lived experience, and the theories I have around what relationships and joint work might look like between researchers and practitioners are themselves getting ‘humbled’. We’re learning a lot about the messiness and challenges of trying to live the values we have around research. Dixie and I will speak about it more in an upcoming webinar, but the idea of a researcher group embedded within a network of out of school learning organizations was one initiated by practitioner leaders within Hive NYC Learning Network, deeply informed by months of talking to the practitioners it might serve, and in its implementation has been characterized by rich relationships, collaborative design between researchers and practitioners, and an embedded approach that aims to break down walls between research and practice while maintaining the distinctive capacities that a range of stakeholders can bring to the table. And as we’ve been doing this work, we’ve moved again back into the literature to find new and promising trends – emergent work on research practice partnerships, design-based implementation research, co-design and participatory design are forming a basis for what it might mean to take many of the core ideas around design-based research and move them into the context of joint work with a network of talented practitioners in Hive NYC.

It’s been, and continues to be, a fantastic, and challenging, opportunity to try to make this stuff happen on the ground in a way that really reflects the ways I want to work. Reflecting on my journey to and through the practices and ideas of design-based research, coming from way back to my days at Global Kids, is part of what helps me to contextualize and continually refine the work I’m grateful to be doing today.

CHATting about Triangles and Representations

I wrote this post last Friday as I was thinking through some issues around cultural historical activity theory and legitimate participation. Thought I’d post it here, too, just in case folks were interested.

***************

This week I’ve been organizing a poster presentation on the use of Cultural Historical Activity Theory (CHAT) as a lens for thinking about privilege, power, and legitimate participation in high school science. Even though the theory has its detractors, many (Anthony & Clark, 2011; Roth & Lee, 2007; Trust, 2015; Yamagata-Lynch, 2007;)(including myself) have found CHAT to be particularly useful for considering systemic problems (or “tensions” as they’re commonly referred) that (1) inhibit the accomplishment of certain goals (e.g., successful implementation of a one-to-one laptop program) within a social organization (e.g., a school), and (2) end up advantaging certain points of view, ways of behaving, or cultural backgrounds over others. Obviously, any theory or tool that can be used to help researchers, scholars, or practitioners understand problems of practice, especially when they relate to issues of equity, is a good thing to have, but I’ve found that I often have a hard time getting started with a CHAT because the concept of “activity” is a slippery one to define.

Technically, “activity” refers to any kind of goal-motivated behavior. This is different, however, from what we might call a “task” because, as it’s been theorized, an activity, unlike a task, takes place with a social context. And context matters. Consider the task of getting dressed. This task has a very different meaning depending on when and where it’s done – telling someone that you got dressed in your own room in the morning might earn you a totally different reaction from telling them that you got dressed in a stranger’s house or a public place in broad daylight. It’s the context that makes this task seem common and mundane, or strange and embarrassing. (Really, in a way, this means that all tasks are activities, because when is a task not contextualized? But I digress…)

“Context” is also a slippery term, because it can include pretty much anything. Socioculturalists might say that context is made up of everything that exists in a place in the moment, as well as everything that led up to that stuff being there for that moment – the history of the physical objects, the cultural values that give those objects meaning, the arrangement of objects and people in the setting… each of these things matters in how one decides to use a tool and for what purpose, as well as how one’s behavior gets interpreted.

All of these nebulous characteristics of the activity context intersect and interact with each other, and together, they make up the activity “system.” Without getting to deep into the theory of it all, one of the advantages of CHAT is that it gives us a way to divvy these characteristics into six categories: subjects (the individual or group performing the task), the object (the goal of doing the task), mediating artifacts (the tools that are used to achieve the goal), rules (normative or expected ways of doing the task), community (the large social arrangement/organization in which the task is being performed), and the division of labor (the roles of the people/groups within the larger social organization for accomplishing the task). Now, I don’t know about you, but at this point, my head starts to swim with all of these terms and trying to figure out who is doing what and why and with whomever and with what tools and where did those tools come from, anyway? This is when I take out a pen and paper and start trying to draw things that can help me visualize what my brain is trying to keep track of. Luckily, CHAT has this canonical representation that it seems pretty much everyone who talks about activity theory uses, and it looks like this:

CHATting about Triangles and Representations

Cultural Historical Activity Theory

Look at all those triangles! What’s lovely about this representation is that it gives the impression that everything here is interconnected. And that’s the point. Don’t you just love it when simple graphics can convey complex ideas? Yay CHAT. Go team!

So that’s all good and awesome, until you start trying to think about what constitutes an activity. Where did the activity begin? Where did it end? Depending on the grain size of the activity, the various nodes might look completely different. This is very problematic when trying to consider complex activities like implementing a curricular intervention or a district-wide technology initiative. Each of these long term activities is made up of possibly infinite, but meaningful, activities (Lisa Yamagata-Lynch wrote a fantastic piece that addresses these same dilemmas).

In my dissertation, I used CHAT to look at the different ways in which students used technology in a one-to-one laptop classroom, and early on in my analysis, I found myself drowning in a pile of papers with little triangles sketched on them. My problem is that I was trying to figure out how to capture the essence of an activity when it wasn’t explicitly demarcated for me. An activity could be as long as a subject unit, or as brief as recording a single phrase of text from a lecture slide. It was up to me to decide how each of these activities were significant, and which ones related to my research questions. It’s part of that subjective research experience that gives us qualitative researchers limited street cred among our quantitative, objectivist, and positivist colleagues. But that’s another story all together…

Okay, we’re still not done. I said something earlier about CHAT being useful for looking at systemic problems or “tensions.” Let’s get back to that now.

So, the idea is that once we’ve charted all of the constituent “nodes” of the CHAT triangle for a given activity, we can go about examining how elements of these nodes interact with each other in ways that either help the subject achieve the goals of the activity, or inhibit it. For instance, let’s say you (subject) want to use the Internet (mediating artifact) to help you look up the score of a basketball game (object), but you don’t have access to the Internet. We might represent that as a tension between the mediating artifact (Internet) and object (looking up a basketball score). Or let’s say you’re a teacher (subject) and you want to learn about a new software tool (mediating artifact) that you might think would be particularly helpful for a certain lesson you are trying to teach (object), but there’s no one at your school, like a technology integration specialist, who can help you learn about or implement that software into your lesson planning (division of labor). We could represent that as a tension between the division of labor and object of the triangle. To my knowledge, there isn’t a single convention that is used to represent tensions on the CHAT triangle, but some common examples include placing an “x” or lightning bolts along the side of the triangle where the tension exists.

Now, these are relatively simple examples of tensions, so you might already be thinking, “What about more complex problems, such as a tension between rules and division of labor? Or tensions that emerge between interacting activity systems? How can we represent these?” That’s a good question, and it’s where I find myself now. The problem I am having at the moment is coming up with a way that gives credence to how the various elements of activity systems mediate each other, but also provides an elegant way of showing how a tension impacts the object of activity, as well as introduces the concept of “legitimate participation” (Lave & Wenger famously coined the term “legitimate peripheral participation” to describe how individuals are inducted into communities of practice) I feel like much of what has been done to date to represent complex tensions doesn’t adequately (or at least cleanly) show these in, say, the way a line graph can show how the amount of something increases and decreases over time. In preserving the sticky image of triangles, what I’ve been playing with is something like this:

A new representation of CHAT?

What this does for me, is highlight the centrality of issues around “legitimacy” in activities, as something that is influenced (or even governed by?) the relationship of one’s participation to the privileged norms of behavior (rules), roles (division of labor), and use of tools (mediating artifacts) of a social group (community). I’m not convinced that this does any better job of achieving that than the standard CHAT triangle. But if anyone out there has any ideas, I’m all ears (and triangles)!