|
|
CHAPTER
1 –
PURPOSE OF STUDY
Brief
Purpose
The purpose of this study is to design
a comprehensive, replicable, extensible tool for comparing card sorting
methodologies as they are described in selected literature. The card sorting
methods under examination are broadly classified as one of two types, known
as “open
sorting” and “closed sorting” (Deaton, 2002) (Morville & Rosenfeld, 2001) (Maurer & Warfel,
2004).
Card sorting “involves [the]
sorting [of] a series of cards, each labeled with a piece of content or
functionality, into groups that make sense
to users or participants” (Mauer & Warfel,
2002, p.2). Closed sorting is defined as “[a methodology] in which the
groupings are defined by the researcher and the subject is putting object cards
into the defined groups” (Deaton, 2002, p.4).Open sorting is defined as “[a methodology] in which subjects
can determine their own groupings by first sorting the cards and then labeling
the resulting piles” (Deaton, 2002, p.4).
A wide range of professionals use card
sorting methods, including information architects, website designers,
usability specialists, and related professionals, collectively called “practitioners” in
this paper. People who work in these disciplines are responsible for the
design and testing of navigation systems and taxonomic structures for information systems. However, practitioners
often view the information domain from different perspective and “frame of reference”
than the intended users of the information. Card sorting methods can help practitioners
understand the users’ “mental models” and may provide insight into how users
would group content to perform common tasks (Akerelrea & Zimmerman, 2002, p. 438) (Mauer &
Warfel, 2004, p.1) (Morville & Rosenfeld, 2001, p.235).
This study is designed as a literature
review of selected references related to card sorting methodologies used by
information architects and usability practitioners. Using a combination of
the conceptual analysis process (Palmquist, et. al, 2005) and the constant
comparative method (Glaser & Strauss, 1967), the characteristics of the
various card sorting methods are interactively coded and then grouped into
categories. The results of the coding are displayed in a table, which seeks
to identify themes and patterns that emerge from the data rather than being
imposed on the data (Glaser & Strauss, 1967).
The primary outcome of this study is a
replicable and extensible tool (see Table 1: Twelve Categories of Card
Sorting Characteristics), formatted as a set of criteria, to assist
practitioners when comparing and choosing a card sorting method, or
combinations of methods, for use in a given situation.
Full
Purpose
The field of information architecture
is so new that it is still evolving as a recognized discipline (Morville
& Rosenfeld, 2001). As a result, this study is designed for a broad range
of professionals who work in the area of information architecture and who
use, or may be interested in using, card sorting methods as input into in the
design of an information system.
In a discussion of “who is qualified
to practice information
architecture,” Morville and Rosenfeld (2001) suggest that a wide range of
disciplines may collectively provide insight into this evolving science.
Among the practices mentioned are graphic and information designers,
information and library science
professionals, usability engineers, marketing professionals, and computer
science professionals (Morville & Rosenfeld, 2001). According to Morville
and Rosenfeld (2001), graphic and information design professionals are
interested in the communication of information with visual and verbal
clarity. Information and library science professionals study the efficient
and intuitive categorization and organization of information. Usability
engineers, often called Human Computer Interaction (HCI) professionals,
evaluate and assess how users interact with an information system or software
interface. Marketing professionals are expert at defining and understanding
audiences. Computer scientists and software programmers can provide technical
methods for identifying content and they are responsible for the design of
software interfaces (Morville & Rosenfeld, 2001).
Even though professionals in each of
these disciplines use card sorting as part of their work, their goals are not
always the same. For example, the disciplines of usability and information
architecture, although similar in some respects, have significantly different
purposes (Lash, 2002). The usability of an information resource may include
navigation, categorization, and labeling but usability also includes fonts,
colors, and other visual aspects. Information architecture encompasses the
navigation, categorization, and labeling of information but also is concerned
with other information issues, such as metadata and content management (Lash,
2002).
The confluence of professionals
involved in the field of information architecture can create a confusing mix,
even among the professionals themselves. For example, Lash (2002) states that
while the difference between the fields of information architecture and
usability is relatively distinct, for many people the distinction is often
blurred. This is understandable since professionals whose roles fall within
these disciplines often perform duties that cross over into other disciplines
(Lash, 2002). Thus, the intended audience for this study is a cross
disciplinary group of professionals who work in the field of information
management. In this paper, this larger group of related professionals is
referred to as “practitioners.” This term designates an inclusive description of people who use, or may be interested in
using card sorting methods as input into in the design of an information
system.
Practitioners of all types use card sorting
to elicit end user input into the organization of an information structure
(Mauer & Warfel, 2004) (Deaton, 2002). A practitioner may design a card
sorting exercise by choosing between variants of card sorting methodologies,
including open sorting, closed sorting, multiple sorting, and successive
sorting (Deaton, 2002). The most commonly used methods, open and closed
sorting (Mauer & Warfel, 2004), are examined in this study. Open card
sorting is generally used to elicit user input in the initial information
design phase (Mauer & Warfel, 2004, p.2) (Boutelle & Sinha, 2004,
p.350) (Deaton, 2002). Closed sorting is typically used for testing proposed
or existing designs, or for testing information categories and labels that
emerge from an open sort exercise (Mauer & Warfel, 2004, p.2) (Boutelle
& Sinha, 2004, p.350) (Deaton, 2002).
Many practitioners contend that card
sorting can be valuable in the early design or redesign stages of an
information system (Faiks & Hyland, 2000) (Fuccella, 1997). However, card
sorting results should not necessarily dictate the design of the information
resource, but rather should be used as one source of input in the design
process (Deaton, 2002) (Mauer & Warfel, 2004) (McGeorge & Rugg,
2003).
This study is designed as a literature
review of articles and research related to card sorting methodologies used by
information architects and usability practitioners. As a research
methodology, a literature review provides a “theoretical perspective” of a body
of knowledge and provides the researcher with a valuable source of data
gleaned from previous research conducted in the discipline (Leedy & Ormrod, 2001, p. 70). The data are
reviewed using a combination of the conceptual analysis process (Palmquist,
et. al, 2005) and the constant comparative method (Glaser & Strauss,
1967). Through emergent identification and selective reduction, the
characteristics of the various card sorting methodologies are interactively
coded. The data analysis process seeks to:
- Identify quantitative characteristics of
open and closed card sort methods. Examples of such quantitative
characteristics include, but are not limited to: the number of
participants, number of cards, length of session, and others.
- Identify qualitative characteristics of
open and closed card sort methods. Examples of such qualitative
characteristics include, but are not limited to: authors’
perspectives on individual vs. group sort, methods for participant or
content selection, and others.
As distinct characteristics of the card
sorting methods are identified, they are grouped with similar characteristics
identified across the selected literature. When a group of characteristics
reaches a significant point of saturation, categories are created and the
properties of the categories are identified using a grounded theory approach
(see Figure 1). This approach seeks to identify themes and patterns that
emerge from the data rather than being imposed on the data (Glaser &
Strauss, 1967).
Figure 1: Combined Conceptual Analysis and Constant Comparative
Method Process
The primary outcome of this study is a
replicable and extensible tool (see Table 1: Twelve Categories of Card
Sorting Characteristics), formatted as a set of criteria, to assist
practitioners with comparing and choosing an open or closed card sorting
method, or combinations of methods, for use in a given situation. Results
from the data analysis are framed for use by practitioners, who may find
value in this tool for a number of reasons. Specifically the tool is designed
to:
- Reveal characteristics of card sorting
methods that may be overlooked in card sorting exercise design;
- Provide a tool for practitioners to compare
the methods of other practitioners;
- Aid with the design of a card sorting
exercise based on standardized criteria; and
- Provide a method for extending the data set
by coding literature from other card sorting guidelines and case
reports.
Significance
of the Study
Card sorting is most valuable in the
early development stages of an information system because it provides an
opportunity for users to provide input into the design of an information
structure rather than evaluating a structure that has already been designed
(Faiks & Hyland, 2000). Other benefits of using card sorting in the early
design phases include increased usability, reduced subjectivity introduced
into the design by developers or internal pressures, and increased acceptance
of the design by end users (Hahsler & Simon, 2001).
A review of the literature reveals a
number of articles that describe the design of card sorting exercises;
however, none of the resources identified by this researcher offers a
compendium of card sorting methods in the manner presented in this study. The
literature reviewed in this study is grouped into three broad categories:
- Brief summaries of card sorting exercises
as they are described in literature written by practitioners.
- In-depth descriptions of a single card
sorting exercise, called a “case report” in this paper.
- Generalized descriptions and
recommendations for card sorting exercises, termed “guidelines”
in this paper.
Mauer and Warfel (2004 state that card
sorting is briefly mentioned in a few texts, but contend there “is not a
definitive article that describes the technique and its variants and explains
the issues to watch out for.” The authors
provide a generalized set of guidelines for the design of card sorting
exercises (Mauer & Warfel,
2004).
Akerelrea and Zimmerman (2002) briefly
summarize literature on card sorting techniques used by a number of
practitioners, including Fuccella (1997), Fucella & Pizzolato (1998),
Koubec & Montjoy (1991), Dearholt, McDonald, Papp, & Schvaneveldt
(1986), Martin (1999), and Nielsen (1993, 2000). While Akerelrea &
Zimmerman do not provide an analysis of the card sorting literature they
review, they do provide a generalized set of card sorting guidelines. They
conclude with a recommendation for further research in card sorting methods,
including a “comparison of the various card sorting methodologies” (Akerelrea & Zimmerman, 2002). This study attempts to
reach that goal.
Limitations
to the Research
No time frame is specified for the
selection of literature although the predominance of literature is dated from
1994-2005. The works of founding masters of social science research
methodology, such as Glaser and Strauss (1967), are given particular credit
and attention. Foundational studies in the application of card sorting methods
to information design are noted, including Dearholt et al. (1986). The
literature referenced in this study is gathered from the following sources
and is subject to the following criteria:
- Source: Online library databases of the University
of Minnesota and the University of Oregon.
- Criteria: Articles from these sources include
refereed journals, papers presented at conferences, extended abstracts,
and non-refereed journals and periodicals.
- Source: The World Wide Web (WWW).
- Criteria: Articles from the WWW must meet minimum
criteria that include the author(s) name, date of publication, and the
article must contain cited references. The credentials of the author(s)
must be included in the article or available from another source. Articles
or postings from commercial web sites, BLOGS, discussion boards, and
other non-qualified sites are specifically excluded (Kapoun, 1998).
- Source: Books and periodicals
- Criteria: Books and periodicals are admissible if they
have been cited or recommended in other admissible literature.
It should be noted that some of the WWW
articles used in this study are written by information design practitioners
who present descriptive, generalized guidelines on card sorting methods based
on their personal observations and experience. These materials are referred
to as “guidelines”
in the data analysis and in the study. The decision to include these guidelines is made in response to the
seeming shortage of substantially qualified case reports and a desire to
include practitioner reports of applied, field based recommendations for card
sorting as an information design methodology.
The qualitative research process used
in this study is intended to provide sufficient controls to verify the
validity of the research methodology; however, the data under analysis are
not under the control of the researcher. Thus, a full set of comparable data
are not available in all categories for all samplings of the literature.
There is neither a claim, nor the intent, that the sampling of literature
reviewed in this study should be considered inclusively representative of the
literature available on card sorting methodologies.
The criteria for admissibility of
sampling data are broadly guided by the “theoretical relevance” of the data (Glaser & Strauss, 1967). This flexible approach
allows for the selection of data with the intent to identify as many
categories and properties of the categories as possible, rather than
restricting the selection to data that saturate a prescribed set of
categories. The inclusion of data gleaned from diverse sources and that
describe the use of card sorting methods in varied situations is also
conducive to the discovery of a “generalized substantive theory” (Glaser & Strauss, 1967, p.49-54).
This study seeks to identify and
categorize the characteristics of open and closed card sorting methodologies
and to assign properties to the categories. This study does not deeply
examine other types of card sorting methods, or the data analysis
methodologies used to interpret the results of the card sorting exercise. The
analysis of the card sorting data can be either qualitative or quantitative
(Akerelrea & Zimmerman, 2002) (Deaton, 2002) (Fucella & Pizzolato,
1998). If the method of analysis is provided or practitioner perspectives on
the value of quantitative or qualitative analysis of data are offered, they
are coded as a property of the card sorting method. A brief narrative
analysis describes the quantitative and qualitative reconciliation of card
sorting categories that emerge from an open sort. According to Deaton (2002)
it is essential to first determine the method for analysis of the card
sorting data before designing the card sorting exercise, yet many card
sorting articles do not mention the method of data analysis (Deaton, 2002).
A large amount of qualitative data
exists that describe physical, environmental, and hospitality considerations
when conducting a card sorting study. Examples of this data include but are
not limited to:
- The preparation of the physical media used
for the card sort, such as how to prepare index cards (i.e. Word mail
merge, etc.), lamination, using sticky notes, computerized sorting
applications, etc.
- The environmental aspects of the area used
for the sorting exercise, such as a quiet room, a table large enough for
the participant to spread the cards out, etc.
- Hospitality recommendations, such as give
the participants a break, provide refreshments; make the participants
comfortable, etc.
· Although these are important considerations for
conducting a card sorting exercise, they are determined to be ancillary to
the actual design of the card sorting exercise and are not
coded as characteristics of the method. The researcher wishes to note
however, they could be coded in a more broadly framed analysis.
The researcher hopes that practitioners
will find value in the representation of the data as presented in the outcome
of the study – the hypertext Table 1: Twelve Categories of Card Sorting.
However, limitations exist in this condensed view of the literature. The hypertext “tool tip” limits entries to 255
characters, often causing contextual explanations or author quotes to be
truncated. The full conceptual and contextual intent of the author(s) of the
articles under review is not conveyed well by this tool. As a preliminary study, the data are not coded
to the fullest extent possible and considerable potential exists for the
further identification of categories and for the addition of properties to
existing categories. Thus, the reader is highly encouraged to study these
resources to make their own determination on the validity of constructs used
by this researcher for the classification and assignment of properties to the
characteristics.
Problem
Area
Card sorting is a time-tested method of
data collection in the social sciences (Deaton, 2002). Coxon (1999) refers to
literature on card sorting dating as far back as 1935, citing a number of
studies conducted from 1956 to 1991 that use sorting techniques in the fields
of psychology, anthropology, sociology, and mathematics (Coxon, 1999). A card
sorting study conducted in the design phase of a UNIX command documentation
interface is presented by Dearholt, et al. (1986), with references to the use
of hierarchical clustering methods for computer interface design as far back
as 1967. A frequently cited paper by Nielsen & Sano (1994) may have
spurred the use of card sorting as a method for gaining insight into user
preferences in the design of web based information systems (Nielsen &
Sano, 1994).
As a research methodology, card sorting
is often described as a relatively simple, inexpensive method of gaining
insight into user preferences for the organization of information.
Practitioners have described card sorting as:
- “quick, inexpensive,
and reliable” (Mauer &
Warfel, 2004)
- “a relatively
simple process from the participant’s point of view” (Kidwell & Martin, 2001)
- “a powerful,
but relatively straightforward methodology for designing websites based on
user expectations and feedback” (Fuccella & Pizzolato, 1998)
- “particularly
useful for understanding users’ perceptions of relationships between items”
(Martin, 1999)
- “so simple a 6
year old could do it” (Gordon, 2002)
- “easy to replicate”
“a relatively easy task, for both those administering the study and
those participating in it” (Faiks & Hyland, 2000)
However, Boorman and Arabie (1972)
suggest that “it is perhaps a consequence of the deceptive simplicity of the
method of sorting that so many of its problematic aspects have remained unexamined”
(Boorman and Arabie, 1972, as
quoted by Coxon, 1999). According to Gray and Salzman (1998), the design of
many usability evaluation methods (UEM) experiments fall short of meeting the
rigors of scientific integrity in that “neither the data they produce nor the
conclusions drawn from the data are
reliable or valid” (Gray & Salzman,
1998, p. 206). Carey et al. (2002) contend that “many of the UCD
[user centered design] methods discussed in the literature are not effective or
practical for a variety of reasons…there is a need for practical UCD guidelines based on the
collective wisdom of the industry-wide community of UCD practitioners” (Carey et al.,
2002 p. 471). Akerelrea and Zimmerman (2002) suggest that different usability
specialists, (supposedly) using the same usability methods, elicit vastly different results
(Akerelrea & Zimmerman, 2002). They contend that the credibility of
usability methods has come under criticism in recent years and to “minimize such criticisms,”
suggest further empirical research to “enhance the effectiveness of all usability testing
methodologies.” Eight areas are identified where further research in card sorting
may be needed (Akerelrea &
Zimmerman, 2002).
- A comparative analysis of the different
card sorting methodologies
- Empirical assessments of the quantitative
versus qualitative analyses of the data
- Empirical assessments of group and
individual card sorting methodologies
- Considerations of validity and reliability
of card sorting methodologies
- Assessments of potential differences across
different populations and cultures
- Comparisons of results between random and
purposeful recruitment of participants
- Determination of the optimal number of
participants
- Establish a standard for the number of “idea” cards
per sorting
(Akerelrea & Zimmerman, 2002, p.
443)
On to
Chapter 2
|
