METACOGNITION: EXAMINING THE COMPONENTS OF A FUZZY CONCEPT

: Metacognition loosely refers to one’s “thinking about thinking” and is often defined by its accompanying skills (such as monitoring and evaluating). Despite the tendency for researchers to use metacognition as an overarching umbrella term, cognitive and educational theorists argue as to whether metacognition is a single construct or made up of distinct, differentiable factors. Given the lack of clarity in the definition of metacognition and its potential components, the purpose of this investigation is to determine whether a two-factor model, representing knowledge and regulation of metacognition, or five-factor model, representing metacognitive knowledge, planning, monitoring, regulation/control, and evaluation, emerges following both exploratory and confirmatory factor analyses. Participants (N =644) from a select number of classes at a large Midwestern university we selected to complete the Metacognition Questionnaire, a 30 item survey designed to measure five components of metacognition that are rarely measured concurrently. An exploratory factor analysis (EFA) revealed a two-factor model resembling metacognitive knowledge and regulation. This two-factor model had Further confirmatory factor analyses (CFA) showed that the two-factor model outperformed the five-factor model based on the fit indices. This study confirms that the componential view of metacognition should be based on the same two-factor model that has been used in previous literature. Educational implications of this study are discussed.


Introduction
Metacognition is a fuzzy concept but widely utilized by the research community in multiple fields, including psychology, education, learning sciences, neuroscience, and clinical psychology. Metacognition is often defined by its' accompanying skills-monitoring, evaluating, strategy use--or defined as an umbrella term, for instance "thinking about thinking." Further, Flavell (1976) put forward that metacognition as "one's knowledge concerning one's own cognitive processes and products" (p. 232). Researchers from multiple fields take aspects of metacognition and apply them to their particular fields. However, it is still unclear if there is an umbrella concept with one major factor that can be labeled metacognition or whether metacognition has clear and distinct factors upon which researchers can base their future research. Expressing the same concept using multiple terms (e.g., executive skills, metacognitive beliefs, and judgments of learning (Veenman, Van Hout-Wolters & Afflerbach, 2006) can confuse the construct keeping it consistently fuzzy and vague. Research has shown that metacognitive skills are indeed connected to positive academic outcomes (e.g., Everson & Tobias, 1998;Isaacson et al., 2006;Tobias, Everson & Laitusis, 1999). Thus, providing a clearer picture of the nature of metacognition will help grow the existing knowledge base surrounding the educational implications of this concept. The current study seeks to determine whether metacognition is a single construct or made up of multiple factors that can easily be differentiated and applied to students' educational experience. According to Flavell (1979), metacognition can be broken down into four categories: 1) metacognitive knowledge, 2) metacognitive experiences, 3) goals (or tasks), and 4) actions (or strategies). Metacognitive knowledge is conceptualized as the knowledge that has been accumulated over time about humans as cognitive beings and that humans have goals, experiences, take action, and perform tasks. The concept of metacognitive knowledge can be further broken down into three specific classes: person, task, and strategy. First, the person class is everything that one knows about oneself as a cognitive processor and the knowledge that other people are also cognitive in nature. Second, the taskoriented class incorporates the knowledge of how the nature of the information one encounters affects and constrains how one should deal with it (Flavell, 1979). Lastly, the strategy class is the knowledge of which strategies are appropriate to use in any specific situation. Flavell (1979) went on to explain that these three levels of metacognitive knowledge always interact with one another. That is, one's knowledge about people as cognitive beings influences one's understanding of the nature of information within tasks and how to handle that information with the appropriate strategies. Additionally, Flavell made explicit that metacognitive knowledge is "not fundamentally different from other knowledge stored in long-term memory" (Flavell, 1979, p. 907). That is, metacognitive knowledge is not removed from the general information processing model and the knowledge one has about one's own thinking is stored just as any other type of knowledge. Metacognitive experiences are conceptualized as any conscious experience (cognitive or emotional) that accompanies any intellectual activity. Flavell (1979) explains that: "First, they can lead you to establish new goals and to revise or abandon old ones. Experiences of puzzlement or failure can have any of these effects, for example. Second, metacognitive experiences can affect your metacognitive knowledge base by adding to it, deleting from it, or revising it…Finally, metacognitive experiences can activate strategies aimed at either of two types of goals-cognitive or metacognitive." (p. 908). These experiences lead people to stronger metacognitive abilities across all categories, including goals and actions. Goals (or tasks) refer to the objectives of a cognitive activity and action (or strategies) refers to the cognitions or other behaviors employed to meet those goals. Therefore, each time one has a metacognitive experience their metacognitive knowledge, goals and actions are affected. This reasoning makes it very difficult, if not impossible, to study each aspect of metacognition individually without accounting for the others. If one were to create a model of the multiple aspects of metacognition, she would need to allow for all the variables to be correlated with one another. One of the reasons that this categorization of metacognition is important is because it influenced researchers for decades and lead to an interest in dissecting the concept of metacognition and many valiant attempts in the literature to make the term less all-encompassing and "fuzzy." By categorizing metacognition into subcomponents, Flavell also opened the door to the training of specific metacognitive aspects in the classroom. In order to help clarify the concept of metacognition, multiple other definitions have been offered. For example, Schraw (2001) defined metacognition as knowledge and regulation of cognition. Knowledge of cognition is further defined as awareness and what students know about their own cognition or about cognition in general.
Regulation of cognition is defined as a set of activities that help students control their learning, attentional resources, use of strategies, awareness of comprehension breakdowns, planning, monitoring, and evaluating their own thinking. Schraw's approach differs from that of Flavell for several reasons but most importantly he focused on the educational implications of metacognition and on simplifying the categorization of metacognition to two levels. Granted, each level does incorporate a breadth of processes, but Schraw focused his attention on just two categories. As a means of better understanding the difference between cognition and metacognition, Schraw (1998) agreed with Gamer's (1987) position that 1) cognitive skills are necessary to perform a task, and 2) metacognition is necessary to understand how the task was performed. Another definition of metacognition was offered by Alexander, Carr and Schwanenflugel (1995) who subdivided metacognition into three parts: 1) declarative metacognitive knowledge, 2) cognitive monitoring and 3) regulation of strategies. Although this definition is similar to Flavell's, the focus has been placed directly on knowledge, monitoring and regulation of cognition. In fact, Flavell's concepts of the three aspects of metacognitive knowledge, metacognitive experiences, and goals, all very broad categories, are not found in Alexander et al.'s definition. By extracting these concepts, Alexander et al. (1995) have been able to specify more exact processes that account for overall metacognition. Although more subcomponents of metacognition have been accepted by the field (e.g., attention (Miller & Jordan, 1982), procedural metacognitive knowledge (Schraw, 2001), planning (Zimmerman, 1989), knowledge of cognition and regulation/monitoring of that knowledge seem to be the two main components that have been studied thoroughly. We believe that incorporating all respected components of metacognition into a single definition would be beneficial to the research community as a whole. One of the most influential problems in metacognitive research is the lack of clarity in the definition of metacognition and its components (White, 1988). Although most research on metacognition breaks the construct down into two components: knowledge of cognition and regulation of cognition (e.g., Brown, 1978;Flavell, 1979;Schraw & Moshman, 1995), these two categories further consist of several subcomponents. It involves knowledge of one's own and others' cognitive processes; planning prior to performing a task; monitoring one's own thinking, learning and understanding while performing a task; regulating one's thinking by making the proper adjustments; controlling thinking to optimize performance; and evaluating cognitive processes after a solution has been found. Table 1 summarizes the definitions of each of the aspects of the working definition of metacognition. A related aspect of metacognition, metacognitive accuracy, is one's ability to accurately predict his outcome on a particular task.

Knowledge
What individuals know about their own cognition and cognition in general.

Planning
Recognizing the existence of a problem, defining the nature of the problem, and deciding on a strategy for solving the problema.

Monitoring
The assessment of the progress of one's current thinking and work on a particular task.

Regulation/Control
The conscious and nonconscious decisions that one makes based on the output of one's monitoring processes.

Evaluation
The process of appraising one's work that has since been completed After Research on metacognition has mainly focused on metacognitive knowledge or regulation. Flavell (1979) put forward that the types of metacognitive knowledge in his definition could not stand alone; there is a constant interplay among them. We believe this concept is true for all components of metacognition. That is, the components of metacognition (e.g., metacognitive knowledge, metacognition regulation) should not be examined alone due to the interactions among them. The factor structure of metacognition is unclear due to the contradictions in the literature (e.g. Flavell, 1979;Schraw, 2001;Alexander, Carr & Schwanenflugel, 1995). However, it is expected that after performing an exploratory factor analysis (EFA) and a follow-up confirmatory factor analysis (CFA), either two or five factors will have the best fit with the data.

2.1.
Participants Participants totaled 644 undergraduate students from a large Midwestern university. Students participated within their regular final exam time in five subject areas: chemistry (1 class), biology (2 classes), astronomy (1 class), history (2 classes) and education (2 classes). The average self-reported high school GPA for the sample was 3.59, and the sample consisted of 53.6% female and 46.4% male students.

Metacognition Questionnaire
A questionnaire was designed for this study to measure five components of metacognition, which are prevalent in the literature but rarely studied concurrently (i.e., knowledge, planning, monitoring, regulation/control, and evaluation). Although there are many options for measuring metacognition (e.g., think aloud protocols, one on one interviews, online measurement, etc.), a questionnaire was most relevant for this particular study given the need to perform a factor analysis to determine the factor structure of metacognition as a construct. The survey was created using a state-trait model, and items were written from a state metacognitive standpoint. The questionnaire consisted of 30 total items, with roughly six items per construct. Future research can take the results from this study as a baseline for the componential view of metacognition and utilize various methods for replication and validation. The questionnaire used in the current study was a compilation of three existing sources: 1) Metacognitive Awareness Inventory (MAI). Schraw and Dennison (1994) set out to create a questionnaire that confirmed the theoretical existence of eight subcomponents of metacognition: 1) declarative knowledge, 2) procedural knowledge, 3) conditional knowledge, 4) planning, 5) information management strategies, 6) monitoring, 7) debugging strategies, and 8) evaluation of learning. However, the final factor structure was best represented by two factors: knowledge of cognition and regulation of cognition, accounting for 65% of the sample variance. The resulting questionnaire consisted of 52-items on a Likert scale. The internal consistency for the Knowledge of Cognition scale was .93 and for the Regulation of Cognition scale was .88. In two experiments, Schraw et al. (1994) found a significant relationship between knowledge and regulation of cognition (r=.54 and .45, respectively). 2) Inventory of Metacognitive Self Regulation (IMSR). Howard, McGee, Shia and Hong (2000) developed the IMSR from two existing measures: 1) the junior MAI (Sperling, Howard, & Murphy, 2002), and the 2) How I Solve Problems survey (Fortunato, Hecht, Tittle, & Alvarez, 1991). An exploratory factor analysis was run and produced a five factor solution, accounting for 56.3% of the sample variance. The resulting measure (after removing items that did not load well on any factor) consisted of 23 items measured on a Likert scale. Because Howard et al. (2000) were interested in creating a new measure specific to metacognition in the context of problem-solving, they examined the remaining 23 items and revised or rewrote them to increase reliability, and wrote additional items to clearly demonstrate the existence of the five factors found in the initial analysis. The final version of the measure consisted of 37 items with a five point Likert scale. In a second study, Howard et al. (2000) conducted another exploratory factor analysis with the new measure again revealing a five factor structure with eigenvalues over 1.12, accounting for 51.6% of the variance. The overall reliability for the measure was alpha=.935, and the reliability for each factor ranged from alpha=.720 to .867. The five factors were labeled as: 1) knowledge of cognition, 2) objectivity, 3) problem representation, 4) subtask monitoring, and 5) evaluation. .

3)
O'Neil's Self-Assessment Questionnaire (SAQ). The SAQ was created to measure four components of metacognition (planning, monitoring, cognitive strategies, and awareness). The measure was based on a state-trait model for metacognition, and the SAQ was written as a state metacognitive measure. That is, the items were written to elicit responses from students about a particular test they had just taken. This is in stark contrast to the first two measures (MAI and IMSR), which were designed for responses for general metacognitive thinking. For 12th graders, the reliability for each component subscale (consisting of five items) ranged from .73 to .78. A factor analysis confirmed only one factor per subscale (O'Neil & Abedi, 1996). The overall reliability of the measure was not provided. Each of these existing measures offered items that matched with a variety of metacognitive components. However, none of the existing measures encompassed all of the components that have been theoretically derived and reported consistently in the literature (knowledge, attention, monitoring, etc.). Therefore, items were extracted from each of the three existing measures to create a more complete measure that theoretically contains the five aforementioned components (see Table  1). Items from the MAI and IMSR were modified where students would respond about a specific task they had just performed (state-based), rather than general statements. Items were chosen for their relevance to five-theoretical components of metacognition: 1) metacognitive knowledge, 2) monitoring, 3) planning, 4) evaluation, and 5) regulation/control. The resulting questionnaire consisted of 30 items based on a 5-point Likert-type scale from strongly agree to strongly disagree.

2.3.
Procedure Students in each class were introduced to the study by their instructor via email or in class 2-3 weeks prior to data collection. Data collection occurred during the students' regularly scheduled final exam period. Before they began their exam, all students were told that there was a consent form to be signed and questionnaire to be completed. It was also made clear that participation was voluntary and they had the option to complete the questionnaire after they finished their exam. An incentive was offered to the students who participated; one person who completed the survey from each class would be randomly chosen to win a monetary prize of $50. After finishing their final exam, students completed the informed consent form and filled out the questionnaire. The survey included metacognitive items related to the final exam, and this process took [126] approximately 5-10 minutes. Any questions that the students had about the survey were addressed by raising their hands and the researcher helped them individually with comprehension issues.

Characteristics of Questionnaire Items
After an examination of the Q-Q plots for each of the 30 items on the questionnaire, all items appeared to follow a normal distribution.
There were no outliers for any of the items because the responses were restricted from 1 to 5 (Likert-type scale). However, item 28 from the original questionnaire was removed due to the fact that it asked students if they asked for help when they did not understand something on their final exam. This could have been interpreted as asking a student next to them, which would be considered cheating by instructors. Thus, the following analyses were performed with 29 items. Missing Data. On each of the 29 items, there were between zero and six missing data points. No pattern could be discerned; thus it was determined that the data were missing at random. Because dropping all participants with any missing data would have decreased the sample size by 36, a multiple imputation procedure, through the LISREL 8.0, was carried out on all 29 items. Multiple imputation is a preferred method for dealing with missing data even if the data is not missing at random or completely at random (Tabachnick & Fidell, 2007).

Exploratory Factor Analysis
An exploratory factor analysis was performed on the survey data using SPSS 15.0; there were 640 participants with complete data for the analysis. Principal axis factoring was used to reveal the underlying structure of the data. Oblique rotation (promax method) was used to rotate the data due to previous research findings indicating that metacognitive components are typically moderately to strongly correlated with each other. Variable communalities were examined to determine the variability that the individual items were accounting for in the factors. Three communalities were low (below .30), suggesting that those items did not explain much variance within the factors. However, these items were not removed from the analysis without examining the factor structure and factor loadings. Utilizing the 29 items, the index of goodness of fit (Kaiser-Meyer-Olkin test) was calculated, yielding a coefficient of .92. This established the data as suitable for factor analysis according to the .80 criterion put forth by Hair, Anderson, Tatham and Black (1998). Three methods were used to determine the factor structure of the data. First, all factors with eigenvalues over 1.0 were extracted. Second, the interpretability of the factors was assessed. Lastly, the scree test (Cattell, 1966) was used to finalize the suitability of the factor structure. Using the first criteria, five components with eigenvalues over 1.0 were extracted, accounting for 52.6% of the variance. Inspection of the five components, however, revealed that the last three components were not easily interpretable. The items that were originally included to load on each of the five factors did not do so with any consistency. Thus, a twofactor model, as cited in the literature (e.g., Brown, 1978;Flavell, 1979;Schraw & Moshman, 1995), was tested by only extracting two factors in the subsequent analysis.
The two-factor model was most appropriate and interpretable, and the scree test confirmed this conclusion, which indeed suggested a two-factor model. The examination of the items loading on each of the factors did lend themselves to the constructs of metacognitive knowledge and metacognitive regulation; these two subcomponents of overall metacognition are well documented in the literature. The two-factor model accounted for 40.2% of the variance, with eigenvalues of 8.58 and 3.09 for the two factors, respectively. Comrey and Lee (1992) established that factor coefficients of .71 were excellent, .63 were very good, .55 were good, .45 were fair, and .32 were poor. Thus, a conservative threshold was decided to be between "good" and "fair" for this particular analysis, set at .50. Examination of the pattern matrix revealed that eight items had factor coefficients below .50. These items were removed from the subsequent confirmatory factor analysis model and the future use of the factors as outcome measures in multiple regressions.

Confirmatory Factor Analysis
Two confirmatory factor analyses (CFA) based on the previous exploratory factor analysis and the theoretical five-factor model were performed through LISREL 8.0. Although the five-factor model did not emerge from the EFA, it was still important to examine the differences between the models to assess the best fitting model. The five-factor model based on the original theoretical conception (utilizing all 29 items) was estimated using the default of maximum likelihood. All factors were hypothesized to be moderately correlated. The results for the adequacy of the five-factor model were mixed. First, the chi-square results were significant, which indicates a poor fit of the model, χ2 (367, N=640)=1790.64, p<.01. Second, the goodness of fit of the model was tested through the ratio of χ2 to degrees of freedom, which was 4.88. Ideally, the ratio should be 3.0 or below, thus suggesting a moderately poor fit, though this threshold is debated in the literature (Delandshere, 4/3/08, personal communication). Last, the indices of goodness of fit revealed a relatively good fit, with the non-normed fit index (NNFI)=.93, the comparative fit index (CFI)=.93, and the root mean square error approximation (RMSEA)=.08. These three indices of goodness of fit are a subset of a great many indices but are the recommended indices in the current literature (e.g., Schrieber, Stage, King, Nora & Barlow, 2006). Thus, the final interpretation of the fit of the model was a low to moderate fit with the data. Post-hoc model modifications were performed to find a better fitting model. The analysis suggested that two items' error variance should be correlated. After examining the two items, it was found that the items were more similar to each other than similar to the other items in the factor. This provided the necessary evidence to follow the suggestions put forth as modification indices and correlate the error variance between items 11 and 12. The resulting model was a better fit than the original. Again, the chi-square results were significant, χ2 (366, N=640)=1594.44, p<.01. However, the ratio of χ2 to degrees of freedom was better than the original model, 4.35. The fit indices remained fairly stable with the NNFI=.93, CFI=.94, and the RMSEA=.08. Also, the original model had a model AIC of 2146.00 and the modified model had a model AIC of 1955.37, a difference of 190.37. Schreiber et al. (2006 suggest that the model AIC can be a comparison between models, with lower scores indicating a better fit. Thus, it can be concluded that the modified five-factor model is a better fit than the original five-factor model. A two-factor confirmatory factor analysis was conducted to assess whether the fivefactor or two-factor model was a better fit to the data. The two-factor model was based on the results from the exploratory factor analysis, utilizing the final 21 items. As with the EFA, the two latent factors were hypothesized to be moderately correlated. The default of maximum likelihood estimation was used to estimate the model. The results for the adequacy of the model were mixed. First, the chi-square result was significant, which indicates a poor fit of the model, χ2 (188, N=640) =939.72, p<.01. Second, the goodness of fit of the model was tested through the ratio of χ2 to the degrees of freedom, which was 5.00. Last, the indices of goodness of fit revealed a relatively good fit, with the NNFI=.92, the CFI=.93 and the RMSEA=.08.
Thus, the final interpretation of the fit of the model, like the original five-factor model, was a low to moderate fit with the data. Post hoc model modifications were performed to develop a better fitting model. Based on the modification indices, it was again suggested that items 11 and 12's error variance be correlated to improve the fit of the model. After examining the two items, it was found that the items were more similar to each other than similar to the other items in the factor. This provided the necessary evidence to follow the suggestions put forth as modification indices and correlate the error variance between items 11 and 12. The chi-square for the new model was again significant, χ2 (187, N=640) =714.13, p<.01. However, the fit indices after this modification revealed a better fit, with the NNFI=.95, CFI=.95, and RMSEA=.07. The change in χ2 between the two models was significant, χ2change (1, N=640) =225.59, p<.01. Also, the original model had a model AIC of 1025.72 and the modified model had a model AIC of 802.13, a difference of 223.59. Thus, it can be inferred from these results that the second model is stronger than the original. Table 2 presents the results from the five-factor models and the two-factor models. Although the modified two-factor model had slightly better degrees of freedom to chi-square ratio and slightly better fit indices than the modified fivefactor model, both models are fairly similar. However, most literature has shown a two-factor model of metacognition, the exploratory factor analysis revealed two factors, and the two-factor model is clearly more parsimonious. Also, when the exploratory factor analysis was forced into a five-factor structure, the last three factors did not make conceptual sense and very few, if any, items loaded at an acceptable level. Therefore, the modified two-factor model was accepted. Each of the two factors in the two factor model had moderately strong internal consistency as measured by Cronbach's alpha. The reliabilities for each component of metacognition are presented in Table 3.

Discussion and conclusions
4.1. Discussion The literature is mixed when it comes to the definition and components of metacognition.
Most research acknowledges two main components: knowledge and regulation (e.g., Schraw, 2001), but it remained unclear whether this two-component model was driven by questionnaires and methods that really were only addressing those two components. Thus, we created a survey that incorporated these and other theoretically derived and consistently cited subcomponents of metacognition (planning, evaluation, and monitoring) to determine whether the two-component model stands or the five components emerge as independent factors. The results suggest that a two-factor model does hold up when these other subcomponents are introduced in the data. The exploratory factor analysis produced a convincing structure with items loading on two factors that resembled metacognitive knowledge and regulation. Items from the planning and evaluation subcomponents were split with some loading on knowledge and some on regulation. The monitoring items all loaded strongly on the regulation factor. Regardless of the split of the items, the two-factor model outperformed the fivefactor model in terms of the Scree Plot and interpretability of factors in the EFA and in terms of fit indices from the CFA (See Table 2). The componential view of metacognition should be based on the two-factor model resultant here and in much of the previous literature.

Conclusions
The main limitation for this study is the use of a self-report questionnaire to measure metacognition. Multiple methods can be used to assess metacognition, such as think aloud protocols (e.g., Rosenzweig, Krawec & Montague, 2011), verbal interviews (e.g., Winne, 2010), and computer logs (e.g., Veenman & Spaans, 2005), among others. However, each type of measurement device for metacognition, or any internal construct has both pros and cons. By combining three existing questionnaires to create one comprehensive version, we are remaining consistent with much of the literature and providing a useful tool for easily measuring the two factors of metacognition.
The educational implications from this study are clear. Providing clarity in the definition and measurement of metacognition, educational psychologists and educators can continue their work in understanding the relationship between metacognition and academic achievement. Establishing reliable and clear tools to measure students' metacognitive knowledge and regulation can assist everyday educators in their quest of improving higher order thinking skills that are lacking in today's classrooms.