|
|
|
|
|
Sorting Out Card Sorting |
|
|
|
|
|
[Home] [Abstract] [TOC] [Chapter 1] [Chapter 2] [Chapter 3] [Chapter 4] [Chapter 5] [Appendix A] [References] |
|
|
|
|
|
|
|
|
|
|
||
|
|
|
|
|
|
||
|
|
CHAPTER III – METHOD The overarching method of inquiry for this study is the Literature Review (Leedy & Ormrod, 2001) (Proctor & Taylor, 2005). The objective of this study is to aggregate and analyze selected literature on card sorting methods. As a research methodology, literature review is useful for summarizing similarities and differences found within the literature, identifying what is known and formulating questions about what is not known, and for discovering controversy within the literature (Proctor & Taylor, 2005). An outline of each phase of the research process is presented below, followed by a detailed description. Data Collection
Data Analysis
Data Presentation
The first search for card sorting
literature is conducted on the Internet by querying the search engines Google
and Profusion using the basic phrases “card sorting” “card-sorting” or “card sort.”
This search identifies a number of practitioner guidelines that are used as a
foundation for additional search
terminology. Further Internet searches include various combinations of “information architecture,”
“usability,” and “human computer interaction.” These basic search terms are
also used to search the online library of the Once the literature is collected, it is reviewed to identify characteristics of card sorting methods. The identification of the characteristics of each card sorting method described in the literature under review begins by categorizing the method as either open or closed sorting. During the first phase of the analysis, a hybrid approach of conceptual analysis (Palmquist, et al. 2005) and constant comparative method (Glaser & Strauss, 1967) is used. Categories are created for similar characteristics of the card sorting methods. During the second phase of the analysis, properties of the data within each of the defined categories are determined and values assigned to the properties, as described below. This study draws data from generalized guidelines written by practitioners and from other forms of empirical research published in juried journals and periodicals. A pre-screening of the literature reviewed in this study reveals that similar observations and data are available in these types of sources, including, for example, the following list of characteristics:
The data analysis process seeks to explicate the characteristics in open and closed card sorting methods in the follow broad two categories:
Conceptual Analysis and Constant Comparative Method The data are reviewed using a conceptual analysis process (Palmquist, et. al, 2005). Data not specifically relevant to a card sorting methodology or technique are ignored. The data are coded only for the existence of the characteristic and not for the frequency of appearance. When a specific characteristic of a card sorting method is identified in a reference, it is assigned to a group of similar characteristics (Palmquist, et. al, 2005). A notation of the presence of the characteristic is made in a spreadsheet column and the column is labeled. If the characteristic is quantitative in nature (i.e. number of cards to sort, number of participants, etc.) the notation is the data itself, such as the numeric value or a range of values assigned to the property of the characteristic. If the characteristic is qualitative in nature, conceptual, or easily classified (such as guideline or case report), an “X” mark is noted in a labeled column. If the researcher determines that additional explanation is needed, a hypertext “tool tip” reference is created. Hovering over the hypertext link with a mouse cursor in the electronic version of the spreadsheet will reveal a direct quotation or a paraphrase from the literature that substantiates the researchers’ interpretation behind the notation. When a characteristic group reaches sufficient saturation, a category of card sorting characteristics is created. If a characteristic is identified and a category exists with similar characteristics identified in other literature, the characteristic is evaluated to determine if it warrants the creation of a new characteristic group or if it should be added to the existing category. The researcher then reviews previously coded literature to reveal if specific properties of the phenomena exist that may have been overlooked. This organic and iterative process may lead to the identification of additional properties of the category, the division of a category into two or more categories, or the recombination of two or more categories into a single category. In order to assign property values to conceptual semantics of the language, a level of generalization is accepted. The researcher “rates” the authors’ perceptions of the characteristic as positive, neutral, or negative. This property is similar to a Likert scale (Usability First, n.d.) but this researcher does not attempt to infer any degrees of positive or negative. This rating is noted in the coding as X+ (positive) Xo (neutral or no perception) and X- (negative). A positive perception of group card sorting is determined by the use of positive terminology, such as “a benefit of group sorting is …” (Mauer and Warfel, 2004) and “sorting collectively can produce valuable information” (Deaton, 2002). For example, as a distinct characteristic of a card sorting method, a Group Card Sorting category is formed with properties of Positive, Neutral, and Negative. This positive condition is suggestive of “minimized differences” in the category (Glaser & Strauss, 1967). Conversely, a negative property of the Group Card Sorting category is determined by negative perceptions, such as “if the participants work as a group … individual approaches to the information organization might be lost” (Ahlstrom & Allendoerfer, 2004). The presence of both positive and negative properties in the Group Card Sorting category is representative of “maximized differences” in the category (Glaser and Strauss, 1967). Qualitative and Quantitative Data Leedy and Ormrod (2001) suggest that it would be irresponsible to assume a single research methodology could effectively analyze all the data with validity and reliability (Leedy & Ormrod, 2001, p. 100-101). In this study, both quantitative and qualitative data are coded through a hybrid process, combining conceptual analysis (Palmquist et al. 2005) and constant comparative method (Glaser and Strauss, 1967). Quantitative data include such characteristics as number of cards to sort and the number of participants in the experiment. Qualitative data include such characteristics as the methods used for selection of participants, methods used for the target audience identification, and methods for identifying content used in the card sort. Grounded Theory and Constant Comparative Method For this study, the data gathered from the conceptual analysis process is further interpreted using a grounded theory approach, which seeks to identify themes and patterns that emerge from the data rather than being imposed on the data (Glaser & Strauss, 1967). The researcher must be “theoretically sensitive,” continually seeking new insights into the data itself (Glaser & Strauss, 1967, p. 46). The constant comparative method of interpretation is “concerned with generating and plausibly suggesting (but not provisionally testing) many categories, properties, and hypotheses about general problems” (Glaser & Strauss, 1967, p. 104). The intent of this study is to identify the characteristics of card sorting methods; suggest categories to group those characteristics, assign properties to the categories, and saturate the categories with substantive data drawn from research and practice. This study does not seek to test the reliability or validity of these properties or methods. The outcome of this study is described as a replicable and extensible tool, formatted as a set of criteria, to assist practitioners with comparing and choosing an open or closed card sorting method, or combinations of methods, for use in a given situation. The following set of definitions is presented to clarify the intent of this outcome. The American Heritage Dictionary (2000) defines ‘replicable’ as “[able] to duplicate, copy, reproduce, or repeat.” ‘Extensible’ is defined as “a system that can be modified by changing or adding features.” A ‘tool’ is defined as “something regarded as necessary to the carrying out of one's occupation or profession.” To ‘compare’ is “to examine in order to note the similarities or differences of.” ‘Criteria’ are defined as “a standard, rule, or test on which a judgment or decision can be based” (American Heritage Dictionary, 2000). In order to achieve the primary outcome of this study, a global view of a relatively large body of literature is taken. The objective of this study is to identify and categorize, within the confines of the literature under review, the characteristics of card sorting methods as they are described or documented by practitioners. The identified characteristics, when categorized, may be viewed as criteria that practitioners should consider when designing a card sorting exercise. The global view presented in this study is intended as a starting point for further analysis and comparison. This study does not draw conclusions from the data, nor does it suggest the data comprehensively identify all card sorting characteristics. A spreadsheet is used to record and display the results of the conceptual analysis data, providing the reader with a condensed visual overview of the card sorting characteristics identified in the study (see Table 1: Twelve Categories of Card Sorting Characteristics). In literature where both open and closed sorting methods, or where alternative methods are described, the literature is coded in two separate rows of the spreadsheet. The hypertext version of the completed spreadsheet is presented in the Microsoft .mht file format and is designed with Microsoft Excel 2003. A table containing preliminary data is embedded in this Microsoft Word 2003 document, and an interactive spreadsheet may be viewed with the Microsoft Internet Explorer browser in a web-based representation of the data. No other configurations are tested. A printed copy of the spreadsheet does not display the extended hypertext notations underlying the coding of the data. For a full experience of this tool, it is highly recommended the data be reviewed using a hypertext version.
|
|
|
|
||
|
|
Read in Browser | Download PDF ©2005
Steven D. Hannah
|
|