Effects of Ratemyprofessors.com and University Student Evaluations of Teaching on Students’ Course Decision-Making and Self-Efficacy

This study investigated effects of Ratemyprofessors.com and university student evaluations of teaching on students’ course decision-making and self-efficacy in an ethnically diverse undergraduate sample. It also investigated if these effects were impacted by evaluation positivity. Additionally, the study explored if attitudes toward Ratemyprofessors.com were related to student gender, college class, and age. Participants were 73 undergraduates who were exposed to positive and negative evaluations about fictitious professors; participants were informed that the evaluations originated from Ratemyprofessors.com or university student evaluations of teaching. Evaluation positivity but not type influenced students’ intention to enroll in the professor’s course, but not how seriously they would consider the feedback. Evaluation positivity also influenced self-efficacy. Beliefs about and use of Ratemyprofessors.com were not related to student gender, college class, or age. Evaluation positivity’s effect on student course decision making and self-efficacy has implications for for university students, faculty, and administrators.


Introduction
The characteristics on which students rate professors on RMP have changed over time. Previously, students rated their professors on their helpfulness and clarity; these scores were then averaged to produce each professor's overall quality score. Additionally, students rated professors on their easiness (rather than its current incarnation as the difficulty score) (RMP, n.d.). Until summer 2018, students also rated professors' "hotness;" "hot" professors received a chili pepper icon prominently placed at the top of their profile. Although RMP did not officially operationally define the chili pepper as a rating of professors' physical attractiveness, the symbol was widely regarded as such (e.g., Theyson, 2015). Following intense scrutiny that the chili pepper icon was not only unrelated to teaching competence but also degrading, RMP finally terminated its use in June 2018 (Shannon, 2018). RMP also added "tags" to professors' profiles (Hawkins, 2020). Users may select up to three of 20 tags to highlight specific qualities about the professor. Tags include information about workload (e.g., "so many papers"), the importance of staying abreast of course material (e.g., "beware of pop quizzes"), and professorial characteristics (e.g., "hilarious").

RMP's Validity
Just as RMP is the professor-rating website that has gathered the most attention from students, it is also the one that has gathered the most attention from scholars. Many have questioned the validity of the site's evaluations, pointing to issues such as the absence of verification that evaluators are currently or have ever been students of the professor (Brown et al., 2009;Johnson & Crews, 2013;Manno, 2020;Otto et al., 2008). Bitter coworkers may use the site to deride a professor under the guise of a student (Saccone, 2006) and professors themselves may utilize it to improve their online reputation (Montell, 2006). A larger concern is the potential for bias in RMP evaluations. Some scholars express concern that individuals who use the site do not represent the greater population of a professor's students; rather, they represent a selection of students with more extreme attitudes toward the professor (Legg & Wilson, 2012;Peterson et al., 2011), including those with a desire for revenge (Chiang, 2017;Leong, 2020). Moreover, ample research suggests that RMP's evaluations are biased against some professors due to characteristics unrelated to teaching competence; for example, gender, age, ethnicity, and race. Some studies find that RMP evaluations favor men (Boehmer & Wood, 2017;Fisher et al., 2019;Murray et al., 2020;Storage et al., 2016) or women (Sohr-Preston et al., 2016), younger professors (Clayson, 2016;Murray et al., 2020;Stonebraker & Stone, 2015;Sohr-Preston et al., 2016), White professors (Reid, 2010), and individuals without an accent (Murray et al., 2020;Subtirelu, 2015;Subtirelu & Gopavaram, 2016).
Research findings regarding RMP's former easiness, helpfulness, clarity, and hotness scores also advance reasons for concern regarding its validity. Easy professors, for example, receive higher quality scores (Rizvi, 2015;Rosen 2018) and are more likely to have their classes recommended to others (Yoon, 2015). Moreover, "hot" professors receive higher clarity, helpfulness, and overall quality scores than their "not hot" peers (Felton et al., 2004;Riniolo et al., 2006;Rosen, 2018;Theyson, 2015). These findings have led many to question if RMP evaluations represent anything related to student learning given that they seem to place higher importance on easiness and attractiveness than on being challenged to learn and grow (e.g., Yoon, 2015). Some assert that RMP's evaluations represent student consumerism; moreover, the website serves as a destination for students who wish to shop for professors and a degree (Davison & Price, 2009). Indeed, empirical findings lend support to this assertion; students who place higher importance on grades rather than learning are more likely to utilize RMP (Hossain, 2010).

RMP's Effects on Students
Although questions about RMP's biases and validity have given many scholars pause about its evaluations, its 19 million ratings testify to its popularity with students (RMP, 2020a). Students perceive RMP's ratings and evaluations to be credible resources to inform their course decision-making (e.g., Landry et al., 2010). Moreover, they perceive these evaluations to be just as valuable to inform their course decision-making as their own university academic advisors (Hayes & Prus, 2014).
Given the growing number of students utilizing RMP as well as the weight that they place upon its evaluations, researchers have extended from examining its biases to examining its effects on users. Findings suggest that exposure to RMP evaluations influences students' perceptions of professors, including their competency as educators. Lewandowski et al. (2011), for example, found that students perceive a professor's in-person teaching more favorably following exposure to positive RMP evaluations about the professor. Reber et al. (2017) found similar effects for video-recorded professors as well; students who read positive RMP evaluations prior to watching a video-recorded lecture perceived the professor to be more pedagogically adept than students who read negative RMP evaluations prior to the same lecture.
A growing body of research also finds that RMP evaluations influence students' expectations about aspects of their performance in professors' courses. For example, students exposed to positive RMP evaluations for a fictitious professor reported greater motivation to learn in the professor's course compared to students exposed to negative or no evaluations (Edwards & Edwards, 2013). Moreover, students who viewed positive evaluations for a fictitious professor also perceived greater control over their grade in the professor's course (Kowai-Bell et al., 2011;Kowai-Bell et al., 2012;).

Self-Efficacy
If the positivity of RMP evaluations can influence students' confidence that they can exercise control over an important academic achievement marker (i.e., grade) in a professor's course, these evaluations also stand to sway students' self-efficacy for other aspects of the university experience. Self-efficacy refers to individuals' beliefs that they have or can develop the requisite abilities to achieve a goal (Bandura, 1977). Self-efficacy influences individuals' willingness to attempt a new activity, the degree of effort there are willing to invest in the new activity, as well as the extent to which they are willing to persist in their efforts to accomplish the

Open Access
Higher Learning Research Communications 4 activity when faced with obstacles (Bandura, 1982(Bandura, , 1989(Bandura, , 1997. Self-efficacy is not static or global; rather, it varies from domain to domain. University students, for example, possess varying degrees of university selfefficacy. University self-efficacy refers to students' expectations about their ability to produce positive outcomes in their university courses. Students who are high in university self-efficacy expect that they will be able to successfully negotiate challenges that arrive as they pursue their university goals (Solberg et al., 1993;Solberg & Villarreal, 1997).
Self-efficacy is influenced by multiple factors. Mastery experiences bolster one's self-efficacy for an activity; mastery experiences occur when one's efforts lead to successful completion of a goal (Bandura, 1982). For example, successful completion of a challenging course project could function as a mastery experience to strengthen a student's university self-efficacy. Self-efficacy is also influenced by vicarious experiences; these occur when individuals observe the outcome of others' attempts to achieve a goal (Bandura, 1982). Vicarious experiences exert their greatest influence on self-efficacy when individuals have limited information about their own ability to succeed at a particular task or in a particular setting (Pajares, 1997). The influence of vicarious experiences on self-efficacy seems particularly relevant in the context of RMP evaluations as students often use the site to perform reconnaissance of novel professors. If students are exposed to positive evaluations recounting others' successes in a novel professor's course, their self-efficacy for the course may be improved. However, those exposed to negative evaluations recounting others' perceived failures may deflate students' self-efficacy and in turn, lower their willingness to engage in the requisite behaviors needed for university success before even entering the classroom. Behaviors such as course effort (e.g., class attendance, note-taking, studying) and interpersonal contact with the professor are strongly predictive of university course success (Katrevich & Aruguete, 2017;Svanum & Bigatti, 2006). Given that students' expectations about a course shape their behavior toward that course (Hayes & Prus, 2014), exposure to RMP evaluations that describe perceived failures related to course effort (e.g., "there's too much work for anyone to keep up") or the student-professor interpersonal relationship (e.g., "he's hard to talk to") stand to impact students' self-efficacy in ways that have serious implications for their learning. Therefore, this study investigated if the effects of evaluation positivity extend to students' self-efficacy for course effort and the student-professor interpersonal relationship. Moreover, it investigated if previous findings about evaluation positivity's effects on confidence for learning and academic achievement (Edwards & Edwards, 2013;Kowai-Bell et al., 2011;Kowai-Bell et al., 2012) replicate in a different sample.
Although students use RMP to inform their decisions about which professors and which courses to take, some scholars argue that university students turn to RMP only because they have no alternative source of information about potential professors such as university SET (e.g., Lee & Deale, 2018). Since their beginning in the 1920s, end-of-semester university SET are routinely administered as a means of providing formative and summative feedback to professors (Benton & Cashin, 2014;Boring et al., 2016;Hornstein, 2017). Using university SET, students numerically rate professors and provide narrative evaluations (Adams & Umbach, 2012). Like RMP, university SET are anonymous and now administered online (Risquez et al., 2015). Unlike RMP, their results are typically unavailable to students (Brown et al., 2009); for example, students at the university where this data collection occurred do not have access to results of university SET. Although evidence suggests that university SET are marked by many of the same biases present in RMP (age, Wilson et al., 2014;easiness, Clayson et al., 2006;gender, Valencia, 2020;race and ethnicity, Wallace et al., 2019), they provide validity controls unavailable in RMP, such as reasonable assurances that the individuals completing a professor's evaluations are actually students at the university and enrolled in the professor's course. Many students view university SET favorably (Kite et al., 2015) and see it as a useful way for professors and administrators to gather students' feedback (Chen & Hoshower, 2003). This, coupled with some students' concerns that RMP evaluations represent only students with extreme positive or negative views of professors or those motivated by anger and vengefulness (Kindred & Mohammed, 2005), suggests that students may perceive university SET to have greater validity than RMP. However, it is possible that students would prefer RMP given that they, rather than professors and administrators, are its intended audience (RMP, 2020a).
Moreover, RMP provides information about professors that many students value, such as easiness (Kindred & Mohammed, 2005;Liu et al, 2013). University SET, however, do not.
Research comparing students' attitudes toward university SET and RMP is lacking and, as such, this study explored potential differences between them. Students may prefer one source over the other and also be differentially impacted by them. Given that previous research indicates that older students, first-year students, and women students are more likely to complete university SET (e.g., Hatfield & Coyle, 2013;Kherfi, 2011;Macfadyen et al., 2016), this study also explored if these patterns extend to attitudes toward RMP.

Research Question
Research Question 1 was exploratory in nature and investigated students' attitudes toward university SET and RMP. Specifically, it investigated if students read, complete, and consider RMP evaluations more seriously than university SET.

Research Question 2
Research Question 2 was also exploratory in nature. It investigated if demographic patterns related to university SET extend to RMP. Specifically, it investigated if attitudes toward RMP were related to student age, class, and gender.

Research Question 3
Research Question 3 investigated if evaluation positivity (positive evaluations or negative evaluations) and evaluation type (university SET or RMP) influence students' course decision-making. It was hypothesized that students would consider evaluations more seriously and report greater intention to take a professor's course when the professor was positively evaluated. Investigation of evaluation type's (university SET or RMP) influence on consideration of evaluations and intention to take the professor's course was exploratory in nature.

Research Question 4
Finally, the study investigated if evaluation positivity and evaluation type influence students' self-efficacy for the course. Specifically, it investigated if positivity and type influenced students' self-efficacy for two variables important for university success: course effort and development of an interpersonal relationship with the professor. The study also investigated if they impact students' self-efficacy for academic achievement and learning in a course. It was hypothesized that students exposed to positive evaluations would report greater interpersonal, course effort, academic, and learning self-efficacy. Investigation of the effect of evaluation type on self-efficacy was exploratory.

Study Design
This study used a repeated measures, experimental design. Participants read positive evaluations about one fictitious professor and negative evaluations about another fictitious professor. To control for order effects, positive and negative evaluations were counterbalanced. Participants were randomly assigned to evaluation type; they were informed that the evaluations they received were from either university SET or RMP.

Sampling
Power analysis was conducted using G*Power 3 (Erdfelder et al., 1996;Faul et al., 2007). A power of .80 and an alpha level of .05 were used to calculate the minimum number of participants needed to detect a medium effect size. The analysis indicated that data from a minimum of 66 participants would be needed. Convenience sampling was used to recruit undergraduates enrolled in lower-division and upper-division psychology courses at a medium-sized university in the southwestern United States. These courses were selected for recruitment because they fulfill general graduation requirements; therefore, students from diverse academic majors and college classes would be included in the sample. Potential participants received a verbal and written description of the study. Participants received four extra-credit points to be applied to the next assignment; however, an alternate extra-credit opportunity was available for students who chose not to participate in the study. The university's Institutional Review Board reviewed and approved the study.

Materials Evaluation Stimuli
Participants received a set of positive evaluations about one fictitious professor and a set of negative evaluations about a second fictitious professor; these evaluations originated from RMP. RMP uses the average of professors' quality scores to classify them into one of three categories: good quality (score: 3.5 to 5.0), average quality (score: 2.5 to 3.4), or poor quality (score: 0 to 2.4) (RMP, 2020b). The positive evaluations were copied from profiles of good quality professors and the negative evaluations were copied from profiles of poor quality professors. Moreover, evaluations referred to the qualities that RMP guidelines encourage students to address: helpfulness and clarity (RMP, 2020b). Slight changes were made to the evaluations to remove information about professors' names, ages, genders and course disciplines (ex: mathematics, history); however, no changes were made to the evaluations' writing style or spelling errors. The number of evaluations was held constant between the positive and negative evaluations; five evaluations were included for each condition. Evaluation word count was also held constant.
A sample negative evaluation was: "Professor continuously insisted that assignment guidelines were plain as day even though they were really confusing and could not believe that I did not understand the directions.
Refused to grade an assignment that I had to rewrite 3 times and in turn gave me an F." A sample positive evaluation was: "instructions were always crystal clear. teaches the material very well and easily-it's hard not to learn from in this class. The prof was funny and I always looked forward to going to class. Down to earth, passionate about the subject, always willing to help outside of class."

Instrumentation Manipulation Check
Similar to Kowai-Bell et al. (2011), rating of the professors' likability served as a check that participants perceived the manipulation of evaluation positivity. Participants used a 9-point, Likert-type scale to rate the professor's likability on a scale ranging from 1 (strongly disagree) to 9 (strongly agree).

Course Decision-Making
Participants used a 9-point, Likert-type scale ranging from 1 (strongly disagree) to 9 (strongly agree) to respond to statements regarding their decision-making about the professor's course. They rated 1) how seriously they would consider these evaluations when making a decision to enroll in the professor's course and 2) intention to enroll in the professor's course.

Self-Efficacy
Participants used a 9-point, Likert-type scale ranging from 1 (strongly disagree) to 9 (strongly agree) to rate their self-efficacy for their ability to participate in the professor's class, ask the professor for help outside of class, invest effort into the course, perform academically well on assignments, perform academically well on exams, and learn. Ratings for 1) participate in the professor's class and 2) ask the professor for help outside of class were combined to form the interpersonal self-efficacy scale (Cronbach's α internal consistencies: interpersonal self-efficacy with the positively evaluated professor, α = .68; interpersonal self-efficacy with the negatively evaluated professor, α = .80). Ratings for 1) perform academically well on assignments and 2) perform academically well on exams were combined to form the academic achievement self-efficacy scale (Cronbach's α internal consistencies: academic achievement with the positively evaluated professor, α = .92; academic achievement self-efficacy with the negatively evaluated professor, α = .92).

Attitudes Toward University SET and RMP
Participants used a 9-point, Likert-type scale ranging from 1 (never) to 9 (always) to how often they would read evaluations from 1) university SET and 2) RMP during their course decision-making. They also used this scale to indicate the frequency at which they have completed or plan to complete 1) university SET and 2) RMP evaluations. Moreover, they used a 9-point, Likert-type scale ranging from 1 (not seriously) to 9 (very seriously) to indicate how they would consider 1) university SET and 2) RMP evaluations in their course decision-making.

Demographic Questionnaire
Participants also completed a demographic questionnaire that solicited information about age, gender, race or ethnicity, and college class. These items used an open-ended format.

Procedure
Following recruitment and informed consent, participants received a packet containing the evaluations of the fictitious professors; they completed the packet during class. Evaluation positivity was counter-balanced in the packets. Some participants viewed the positively evaluated fictitious professor first whereas others viewed the negatively evaluated professor first. Participants were asked to imagine that they were using these comments to inform their decision-making and expectations about the professors' course and then complete the rating scales. The vignette and role-play method has been used in other studies investigating the effect of RMP (e.g., Sohr-Preston et al., 2016). This method allows researchers to maintain high internal validity (Atzmüller & Steiner, 2010) while also controlling for potential biases that may occur if a student had previous interactions with a real-life professor. Additionally, it avoids the ethical problems that would emerge by purposefully exposing students to an unhelpful and unclear professor (Gotlieb, 2011). Following completion of the rating scales for the positively evaluated and negatively evaluated fictitious professors, participants completed measures of attitudes toward university SET and RMP as well as the demographic questionnaire.

Statistical Analyses
A series of t-tests were used to explore Research Question 1, if participants differ in their reading, completion, and consideration of RMP evaluations and university SET. Bivariate correlational analyses and multivariate analysis of variance (MANOVA) were used to explore Research Question 2. These analyses explored if attitudes toward RMP were associated with student age and differed between student college class and genders. A repeated-measures MANOVA (RM MANOVA) investigated Research Question 3. The RM MANOVA tested the hypothesis that evaluation positivity would affect participants' course decision-making: how seriously they would consider the evaluations and intend to enroll in the professor's course. Additionally, it explored if evaluation type also affected these variables. A RM MANOVA also investigated Research Question 4. It tested the hypothesis that evaluation positivity would influence participants' interpersonal and academic achievement self-efficacy as well as their self-efficacy for investment of course effort and learning. Moreover, it explored if evaluation type also influenced these variables. Bonferroni-corrected analysis of variance (ANOVA) and repeated measures analysis of variance (RM ANOVA) followed significant multivariate tests.

Participant Characteristics
Of the 83 individuals approached for recruitment, 79 consented to participate (95.18% response rate). Three participants were excluded due to incompletion of study measures; an additional three participants were excluded as outliers (their responses on dependent variables were more than three standard deviations beyond the mean). Therefore, the final sample included 73 participants. The sample was 79.5% women (n = 58); participants ranged in age from 18 to 46 years (M = 20.73, SD = 3.84). The sample was ethnically diverse; 67.1% of participants (n = 49) identified as Hispanic or Latinx, 20.5% (n = 15) identified as White or Caucasian, 5.5% (n = 4) identified as Black or African American, 4.1% (n = 3) identified as Asian, and 1.4% (n = 1) identified as Pacific Islander. One participant (1.4%) provided no information about racial or ethnic identity. The sample's racial and ethnic demographics are representative of the university's overall student population. Finally, the sample was comprised of 20 first-year students (27.4%), 12 sophomores (16.4%), 22 juniors (30.1%), and 19 seniors (26%).

Attitudes Toward University SET and RMP
First, Research Question 1 was investigated. Table 1 presents descriptive statistics for how often students complete or plan to complete RMP evaluations and university SET, how often they read RMP evaluations and how often they would read university SET if it were available, and how seriously they would consider RMP evaluations and university SET in their course decision-making. A series of t-tests (see Table 1 for descriptive information and t-test results) found that when compared to RMP, students complete university SET more often, would read university SET more often, and would consider university SET more seriously in their course decision-making. Note: Response options ranged from 1 to 9; higher scores indicate greater frequency and consideration.

Participant Demographics and Attitudes Toward RMP
Next, Research Question 2 was investigated. Bivariate correlational analyses tested if participant age was related to attitudes toward RMP. Participant age was not significantly related to how often participants read RMP evaluations, how often they complete or plan to complete RMP evaluations, or how seriously they consider them in their decision-making (all p > .05).

Next, MANOVA investigated if attitudes toward RMP differed between college classes and student genders.
Box's test for equality of covariance matrices was not significant; therefore, Wilk's λ was used. The MANOVA yielded a significant multivariate college class by student gender interaction effect with a medium effect size, F(9, 153.48) = 1.99, p = .04, ɳp2 = .09, power = 0.73. Neither the multivariate main effect for college class nor student gender was significant. Bonferroni-corrected RM ANOVAs (alpha = .0167) followed-up the significant multivariate college class by student gender interaction effect; all were p > .0167. There were no differences in the frequency at which participants read RMP evaluations (p = .45), how often they complete or plan to complete RMP evaluations (p = .04), or how seriously they consider them in their decision-making (p = .12). See Table 2 for means and standard deviations of these variables. Given the small sample size, some of these analyses were likely underpowered; this is further addressed in the limitations section. Note. Response options ranged from 1 to 9; higher scores indicate greater frequency and consideration.

Manipulation Check
A RM ANOVA tested the effect of evaluation positivity on likability; RM ANOVA was significant with a large effect size, F(1, 71) = 371.23, p < .001, ɳp2 = .84, power = 1.00). Participants perceived the positively evaluated professor (M = 7.90, SD = 1.28) to be more likable than the negatively evaluated professor (M = 2.58, SD = 1.73). This suggests participants perceived the manipulation of evaluation positivity.

Course Decision-Making
A RM MANOVA investigated the effect of evaluation positivity and evaluation type on course decision-making (Research Question 3). Box's test for equality of covariance matrices was significant; therefore, Pillai's trace was used. As hypothesized, the RM MANOVA yielded a significant evaluation positivity main effect on course decision-making, F(2, 69) = 280.62, p < .001, ɳp2 = .89, power = 1.00, indicating a large effect size. The main effect for evaluation type (p = .93) was not significant. Moreover, the evaluation positivity by evaluation type interaction effect was not significant (p = .96).
Bonferroni-corrected RM ANOVAs (alpha = .025) followed up the significant evaluation positivity multivariate effect. Evaluation positivity did not have a significant effect on how seriously participants would consider the evaluations when making a decision to take the course, F(1, 70) = 2.01, p = .16, ɳp2 = .03, power = .29. Participants would consider both positive (M = 8.08, SD = 1.32) and negative (M = 7.81, SD = 1.77) evaluations equally seriously in their decision-making. Evaluation positivity did, however, have a significant effect on decision to enroll in the professor's course, F(1, 70) = 488.55, p < .001, ɳp2 = .88, power = 1.00; large effect size. Participants reported greater likelihood that they would decide to enroll in the positively evaluated professor's course (M = 8.39, SD = 0.96) rather than the negatively evaluated professor's course (M = 2.29, SD = 1.81).

Self-Efficacy
To investigate Research Question 4, a RM MANOVA tested the effect of evaluation positivity and evaluation type on self-efficacy for 1) the interpersonal relationship with the professor, 2) investment of course effort, 3) academics, and 4) learning. Box's test for equality of covariance matrices was significant; therefore, Pillai's trace was used. As hypothesized, the RM MANOVA yielded a significant evaluation positivity main effect with a large effect size. However, the evaluation type main effect and the evaluation positivity by evaluation type interaction effect were not significant.
Bonferroni-corrected RM ANOVAs (alpha = .0125) followed up the significant evaluation positivity multivariate effect. Evaluation positivity influenced all forms of self-efficacy, all with large effect sizes. As hypothesized, participants reported higher interpersonal self-efficacy, course effort self-efficacy, academic achievement self-efficacy, and learning self-efficacy for the positively evaluated professor.   Note: Interpersonal and Academic Self-Efficacy Scales ranged from 2 to 18. Course Effort and Learn Self-Efficacy Scales ranged from 1 to 9. Higher scores indicate greater self-efficacy.

Discussion
A large body of research points to the questionable validity of RMP evaluations (Legg & Wilson, 2012;Murray & Zdravkovic, 2016) yet many students view them as a useful tool to inform their course decision-making (Hayes & Prus, 2014). Given this, a growing body of research has investigated the effects of RMP evaluations and found that their positivity has significant influence on students' expectations. Some scholars argue that students turn to RMP only because they do not have access to university SET (e.g., Kindred & Mohammed, 2005). Consequently, this study investigated if students differ in their attitudes toward university SET and RMP. Moreover, it investigated if attitudes toward RMP were related to student demographics. Finally, it used a repeated-measures, experimental design to investigate if evaluation positivity (positive evaluations or negative evaluations) and evaluation type (RMP or university SET) influenced students' decision-making and self-efficacy for professors' courses.

Attitudes Toward University SET and RMP
Research Question 1 explored if students read, complete, and consider RMP evaluations more seriously than university SET. Participants reported that they complete university SET more frequently than RMP and they would view university SET results more frequently than RMP, if they had access to them. Additionally, they would consider university SET more seriously in their course decision-making than RMP. Despite university SET having a different audience (professors and university administrators) and a different purpose (feedback to improve teaching), these findings suggest that students would prefer university SET and utilize RMP because they have no alternative source of information to inform their course decision-making. Moreover, these findings suggest that students perceive university SET to be more credible than RMP. This possibility has interesting implications for the findings related to Research Questions 3 and 4, discussed below.

RMP Use, Belief, and Participant Demographics
Given that previous research indicates that older students, first-year students, and women students are more likely to complete university SET (e.g., Hatfield & Coyle, 2013;Kherfi, 2011;Macfadyen et al., 2016), Research Question 2 explored if this pattern extended to RMP; in this sample it did not. This may be related to difference in purpose between university SET and RMP. University SET functions to provide formative feedback about teaching to professors and university administrators (Benton & Cashin, 2014). Ratings for hotness and difficulty, however, suggest a very different purpose for RMP-one that places a premium on easy grades rather than learning (Yoon, 2015). That the demographic patterns related to university SET did not extend to attitudes toward RMP in this sample suggests that those who evaluate professors on RMP do not reflect those professors' overall population of students. Although previous research has separately investigated students' perceptions of and motivations for completing university SET (e.g., Kite et al., 2015) and RMP (e.g., Kindred & Mohammed, 2005), future research investigating these motivations within the same samples is merited to further clarify differences and similarities between the two. The need for this research is underscored by the increased use of RMP in university personnel decisions (Bleske-Rechek & Fritsch, 2011) and calls to incorporate it into university program evaluation (Bergin et al, 2013).

Course Decision-Making
Research Question 3 investigated if evaluation positivity and evaluation type influence students' decisionmaking about a professor's course. Neither of these variables had a significant influence on how seriously participants would consider the evaluations when making a decision to take the course. This is an interesting finding considering that students reported that they would consider university SET more seriously than RMP evaluations. This incongruity between students' explicit attitude and implicit response toward university SET aligns with social psychological findings indicating that individuals frequently possess inconsistent attitudes (Shoda et al., 2014). Although explicit attitudes may be easier to verbalize (e.g., "I would consider university SET more seriously"), inconsistent implicit attitudes still exert a strong influence on behavior (e.g., consider university SET and RMP evaluations equally seriously) (Karpen et al., 2012). This dynamic may also account for university SET and RMP evaluations' equal impact on students' self-efficacy, discussed below.
Evaluation positivity may have had no impact on how seriously students considered the information because they perceived any information to be potentially useful information. Although professors are not consumer products or hospitality services to be purchased, consumers' motivations for viewing online customer reviews prior to purchase may generalize to students' use of online professor evaluations. Consumers are increasingly engaging in online research prior to purchases (Ahrend et al., 2018); the content of online customer reviews is especially important to consumers investigating services (e.g., vacation destinations, restaurants) rather than physical products (Kim et al., 2011). Consumers are motivated to use online customer reviews in a desire to get the greatest value for their money while also avoiding the risk of an unsatisfactory expenditure (Thorsten Hennig-Thurau et al., 2004;Goldsmith & Horowitz, 2006). Therefore, they would value both positive and negative evaluations to inform their decision-making. Additionally, when reviews are provided anonymously, such as they are in university SET and RMP, individuals perceive both positive and negative evaluations to be equally credible and trustworthy (Kusumasondjaja et al., 2012). This dynamic may extend to students' perceptions of university SET and RMP evaluations; student may deem evaluations, good or bad, useful to their course decision-making.
Consumers' motivation for using online customer reviews may also account for students' increased intention to enroll in the positively evaluated professors' courses. Consumers use online reviews to help them avoid risk (Goldsmith & Horowitz, 2006). Students may perceive a negatively evaluated professor to be too much of a risk and in turn, reported decreased intention to enroll in the professor's course. However, they likely perceived the positively evaluated professor to be a worthwhile expenditure of their time and effort and in turn reported greater intention to enroll in the professor's course. Future research investigating how students' reportedly utilize university SET and RMP content in their course decision-making is merited. This need is particularly relevant when considered in this context: students' decisions to enroll in a course affect whether some professors are employed at all.

Self-Efficacy
Evaluation type did not influence any form of students' self-efficacy. Similar to their course decision-making, students may perceive any information to be impactful, regardless of its source. Evaluation positivity, however, influenced every form of students' self-efficacy assessed in the current study (Research Question 4). Students reported greater self-efficacy for their interpersonal relationship with the professor and investment of course effort when the professor was positively evaluated rather than negatively evaluated. Moreover, they reported greater self-efficacy for academic achievement and ability to learn with the positively evaluated professor. Considering that self-efficacy influences individuals' willingness to attempt a new activity, the amount of effort they are willing to invest, and the degree to which they will persist in their efforts in the face of obstacles (Bandura, 1982(Bandura, , 1989(Bandura, , 1997, these effects have significant implications for student learning. Exposure to positive evaluations would be a boon to students' confidence as they enter a course with a novel professor. Improved confidence for their ability to successfully invest course effort and interact with the professor could lead to increases in these behaviors, both of which are associated with academic achievement and learning (Katrevich & Aruguete, 2017;Pascarella, 1980;Svanum & Bigatti, 2006). For students exposed to negative evaluations, however, the risk exists that they will be less willing to invest course effort and interact with the professor before the class ever begins. Reduced confidence in their ability to invest course effort, engage interpersonally with the professor, academically achieve, and learn would likely contribute to student disengagement; any of one these alone stands to influence higher education outcomes. This is particularly concerning when one considers these findings in this context: students may engage in selfdefeating cognition about their academic abilities all due to exposure to anonymous professor evaluations such as those on professor-rating websites. Given RMP's anonymity, students are likely to perceive both negative and positive evaluations to be equally credible (Kusumasondjaja et al., 2012); however, in the absence of reviewers' identity, students have no opportunity to seriously evaluate the credibility of the evaluators that are influencing their self-efficacy. A negative evaluation may be a sincere critique of a disengaged professor; but, it may also be a "revenge review" by a student upset about having to invest effort and earn a good grade.

Implications
This study's findings suggest that either type of evaluation, university SET or RMP, stands to impact students' decision-making, including their choice to ever enroll in a course. From a financial perspective, this has implications for both faculty and administrators. For faculty, particularly contingent faculty whose job duties are entirely instruction related, students' decisions not to enroll in a course could easily mean lowered salary for those paid on a pro-rated basis or no salary at all. For administrators, classes cancelled due to low enrollment also represent lost revenue and may present the need to quickly hire and vet new faculty to offer other coursework in place of classes that were cancelled. These findings also present significant educational implications for faculty, administrators, and students. Some scholars contend that students use university SET and RMP to punish professors for dissatisfying grades and perceived high workloads (Backer, 2012;Chiang, 2017). However, course requirements that students may perceive to be too demanding are often pedagogically useful; for example, consistently attending class, doing course readings, and writing papers. Students who are angry that they did not receive an "easy A" may retaliate against the professor by posting negative reviews; findings of the current study suggest that those negative reviews influence the expectations and self-efficacy of the professors' future students. In sum, the negative reviews of a handful of a professor's disgruntled students stand to sabotage the expectations, and possibly efforts, of the professor's future students. This could impact universities' central goal: to educate students. Given this, additional research on students' course choice is merited. Research about the effect of RMP's tags such as "so many papers" and "skip class, you won't pass" on such decisions would be particularly useful as professors' top tags feature prominently at the top of their profiles. The effects of evaluations on students' decision-making and selfefficacy also underscore the need for guidance for students who do plan to use RMP. Students interested in utilizing RMP to inform their course decision-making may benefit from looking for comments consistent with the qualities of effective professors, such as expertise and willingness to provide multiple explanations of concepts (e.g., Gruber et al., 2010). Comments describing the professor as knowledgeable of a broad range of topics related to course content and adept at providing multiple examples to improve students' understanding would be consistent with these characteristics.

Strengths, Limitations, and Future Research
Many studies of the effects of RMP on students' expectations have been conducted with majority White students (e.g., Edwards & Edwards, 2013, Lewandowski et al., 2011Reber et al., 2017); however, research indicates that some parts of the university experience differ between White and ethnic-minority students (Anderson & Smith, 2005). One of these differences may be self-efficacy (Gloria & Hird, 1999;MacPhee et al., 2015;Peguero & Shaffer, 2015;Vuong et al., 2010). This study contributes to the literature by exploring the effects of these evaluations on expectations in an ethnically diverse group of students.
Although the constituency of the sample creates a strength of the study, it is also a limitation. RMP is utilized internationally; however, the current study was conducted at one American university. The dynamics of this sample may not generalize to those in other geographic regions in which RMP is used. Moreover, these findings may not generalize from RMP to other similar websites (e.g., MeinProf.de Germany, MisProfesores.com Mexico, RateMyTeachers.com Australia, Canada, Ireland, New Zealand). Although these all share anonymous, online professor-ratings in common, the characteristics on which individuals rate professors vary across sites (e.g., knowledge, difficulty). Future research exploring the effects of RMP evaluations on students' self-efficacy and expectations in ethnically and geographically diverse samples is merited.
The size of the sample also presents a limitation. Although statistical power was above the conventional .80 (Cohen, 1988) for several analyses, for others it was not. Future studies with larger samples including similar proportions of men and women in the sample are merited, particularly to explore if gendered patterns in attitudes toward university SET extend to RMP.

Conclusions
This study's results add to a growing body of literature suggesting that anonymous, online professor evaluations have significant influences on students' course-related expectations. Unlike the extant literature, it explored whether these effects differ between university SET and RMP; results suggest that they do not. Findings that anonymous professor evaluations of either type have a significant impact on students' expectations, coupled with the popularity of websites such as RMP, suggest a need for guidelines to aid students in identifying useful information in online professor evaluations while minimizing the impact of evaluations from vengeful students.