Methods

This study was designed to determine the broad contours of what writing looks like in diverse disciplines at this research site, and especially the prevalence, purposes, and value attached to multimodal composition across the university. I opted to use survey methods in order to cast a wide net and gather data from a large number of respondents, even if the data would not ultimately allow me to capture individual textual examples with any depth or precision. The main information these surveys sought to collect was a fairly comprehensive accounting of the kinds of media faculty and students use when they compose. The questionnaires I designed then also asked questions about how often and under what circumstances the text types respondents selected were created. I took this approach because, as noted in the literature review, there are at present more qualitative case studies of multimodal writing in the disciplines than there are surveys, even though curriculum decisions may need to depend more heavily on large-scale data than situated thick description.

The one notable exception is Reid et al.’s (2016) survey-based study summarized in the literature review. Since my study’s objectives were closely related to Reid et al.’s, I designed my study as a replication study—which, as Raucci’s (2021) recent nuanced CCC essay points out, does not mean it uses the exact same methodology, but instead that it pursues similar questions using comparable methods, which allows us to meaningfully compare conclusions. Haswell (2005) has called for more research in composition studies that is replicable, aggregable, and data-supported (RAD), noting that most of the articles professionals see published in the field’s flagship journals do not meet these criteria. Since Haswell made this case, he has often been cited but much less often taken up, especially on his appeals to replication and aggregation in research. My study made use of Reid et al.’s (2016) RAD design, which explicitly invites replication, in order to 1) corroborate the results of the their study in a different time at a different site, and 2) build on the methodology Reid et al. had constructed in order to also ask new questions.

While my survey of academics used a modified list of text types drawn from Reid et al (2016), my survey instrument diverged from their questionnaire in a number of significant ways. First, I asked not only what kinds of texts faculty produced but also how frequently and in what areas of their work. This entailed much more intensive and qualified reporting on the part of respondents than did Reid et al.’s more streamlined checklist. As a consequence, I chose not to solicit much in the way of qualitative open-ended responses like Reid et al. did, including only one optional field for additional comments at the end of the questionnaire. Keeping the survey lean with a relatively low level of participant investment required (average response time was only eight minutes) enabled me to get a large sample (see below). One final and major difference in methodology is that I asked only one general question about assignments in the questionnaire for faculty. However, unlike Reid et al’s study, my study also featured a companion questionnaire distributed to students across the disciplines in order to gain insight into what students were being assigned. The student questionnaire asked participants to catalogue the kinds of texts they produced for academic contexts, using a very similar list of text types as the one distributed in the survey of academics so that the data from both surveys could be analyzed alongside one another.