Educational Rankings and Adolescent Perceptions of College Cost

Download data and study materials from OSF

Principal investigator:

James Chu

Columbia University

Email: jyc2163@columbia.edu

Homepage: https://sociology.columbia.edu/content/james-chu


Sample size: 800

Field period: 04/21/2020-04/05/2021

Abstract
Public metrics like rankings and ratings are routinely criticized for imposing uniformity in how people think and behave, but rankings may also generate divergent outcomes when people interpret them differently. Here, I investigate the inequality implications of this claim by studying how educational rankings affect adolescent perceptions of college cost, and how these effects vary by socioeconomic status background. I propose that those from lower socioeconomic status backgrounds perceive rankings as a stronger signal for cost than their more advantaged peers, leading them to overestimate the cost of opportunities that they might otherwise benefit from. To test this claim, I conducted a conjoint survey experiment among a diverse sample of U.S. adolescents (n=800). Continuing-generation adolescents underestimate the cost of prestige, while their first-generation peers overestimate how expensive highly ranked schools will be. First-generation students also do not interpret rankings as strong signals of financial aid, even when prestigious colleges offer more aid. Differences in perceived net cost mediate college choices, with first-generation students are three times less likely to prefer top ranked colleges, even when told the college has admitted them. These results imply that the public metrics we use to establish consensus about the prestige of colleges may also be leading those from traditionally disadvantaged backgrounds to overestimate the costs of attendance, potentially sustaining the high degree of stratification observed within the higher education system today.
Hypotheses

Where consumers must choose among different options, characterized by public metrics that designate the relative prestige of these options, we expect the following:

  1. All else being equal, consumers will choose more prestigious options.
  2. More prestigious options will be perceived as costing more than less prestigious options (all else being equal).
  3. Those from lower socioeconomic status backgrounds (e.g. first generation students) will interpret prestige as a stronger signal of cost, relative to those from higher socioeconomic status backgrounds (e.g. continuing generation students).
  4. Those from lower socioeconomic status backgrounds will interpret more prestigious options less favorably than those from higher socioeconomic status backgrounds.

Experimental Manipulations

I use conjoint analysis to measure the effects of specific college characteristics on adolescent perceptions about college cost and how they would select between colleges. Respondents were asked to rate pairs of hypothetical college profiles. They were told that “The following are four-year, bachelor-degree granting colleges based in the United States. After reading the college profiles, please give your best estimate for the questions below. You might feel like you do not have enough information to answer all the questions, but please answer with your best guess.”

Each proposed profile was composed of six attributes describing the college: the college ranking, school ownership, size, graduation rate , location, and student-faculty ratio). Except for the ranking of the college, I randomly drew profiles from the actual distributions of college characteristics. By estimating the population average marginal component effect (pAMCE—de la Cuesta, Egami, and Imai 2021).

Outcomes

Manipulation Check: after viewing profiles side-by-side, respondents were asked which college enrolls more students. 87% of adolescents passed the manipulation check. I include those who failed these checks in subsequent analyses and include a control for whether they passed this check or not. Moreover, I test whether excluding them entirely changes the results. The results are substantively the same.

Tuition and Fees: on a slider scale with no default value (to reduce anchoring effects), and for each individual college, participants were asked to give their best estimate for how much it would cost in tuition and fees each year to attend. The overall mean of their estimates, across all profiles, was $42,900.

Financial Aid: participants were asked to give their best estimate for how much financial aid they would receive each year if they were to attend. On average and across all profiles viewed, respondents thought they would receive $26,600 in financial aid.

Preference: on a 6-point scale where 1 is strongly prefer college A and 6 is strongly prefer college B, respondents were asked to indicate which college they would select, assuming they were admitted at both colleges. The scale forced participants to choose one or the other (there was no neutral midpoint).

Each respondent completed two comparisons of pairs of profiled colleges (four total profiles), each displayed on a new screen. Participants took, on average, 17 minutes to complete the experiment, with a minimum of 5 minutes and maximum of 103 minutes.

Summary of Results

Hypothesis 1:
Colleges that were ranked 75-100 were 11.4 percentage points more likely to be selected relative to colleges where the ranking was unknown (p=0.019). These likelihoods scale with better rankings. Colleges ranked 25-50 were 19.3 percentage points more likely to be selected (p<0.001), and those in the top 25 were 26 percentage points more likely to be selected (p<0.001). In terms of pairwise comparisons, colleges in the top 50 were, on average, 10 percentage points more likely to be selected relative to those ranked in the top 100 but under the top 50 (p=0.07). The other pairwise comparisons are not statistically significant.

Hypothesis 2:
A top 25 college was estimated to cost $8,628 more per year than a college with unknown rankings (p<0.001). Relative to the overall mean of $42,900, this corresponds to a perceived 20% increase in tuition and fees. Interestingly, adolescents did not perceive differences in rank signaled differences in cost among the other ranked colleges, perceiving that they cost approximately $2,158 (ranked 75-100, p=0.395), $3,208 (ranked 50-75, p=0.158), or $3,265 (ranked 25-50, p=0.164) more than colleges without rankings. Pairwise differences among this group were also not statistically significant, suggesting a nonlinear effect where the top-ranked colleges were perceived differently in terms of cost.

Hypothesis 3:
Continuing generation students do not perceive that annual tuition and fees scale with rankings. In fact, colleges that are labelled as being ranked between 75 to 100 are perceived as costing $5,200 less in annual tuition and fees (p=0.081) than colleges with unknown rankings. Colleges in the middle ranks (25 to 75) are perceived as costing approximately $150 more per year to attend (p=0.963). Colleges labelled as being in the top 25 were perceived as costing $3,400 more per year, but this, too was not statistically significant (p=0.255). This suggests that continuing generation students do not rely on rankings as a signal of cost, conditional on the other attributes shown in this study. One concern is that continuing generation students pay little attention to any attributes in assessing tuition and fees, which would suggest that they were not paying attention to the task. This does not seem to be the case, as they thought private schools would cost $17,920 more per year than public schools (p<0.001).

By contrast, the cost estimates of first-generation adolescents are far more responsive to rankings. Colleges ranked in the top 25 are perceived as costing $14,300 more in tuition and fees each year (p<0.001) than colleges with unknown rankings, and the difference between the treatment effects for first-generation and continuing-generation adolescents is statistically significant (p=0.022). Put differently, the perceived tuition and fees of top 25 colleges were four times more for first-generation as compared to continuing generation adolescents ($14,300 versus $3,400). Notably, the increases in perceived costs for other ranking levels are also higher than their continuing generation peers, although these differences are not statistically significant. Profiles ranked 25 to 50 are perceived as costing $6,000 more per year (p=0.096); those ranked 50 to 75 are also perceived as costing about $6,000 more per year (p=0.084). Interestingly, colleges ranked lower (75 to 100) are perceived as costing $8,800 more per year than colleges with unknown rankings (p=0.027), but the pairwise comparisons with the other ranking levels are not statistically significant, and it is possible that this exception to a monotonic increase in cost as a function of improved rankings is due to chance.

Critically, tuition and fees are only one component of the perceived net cost of a college. Annual tuition and fees are offset by financial aid, and it may be that first-generation students perceive that they will receive more financial aid in highly ranked colleges. Results show that continuing-generation adolescents perceive rankings as containing little to no signal for financial aid, whether at the extensive or intensive margin. College profiles that are ranked, relative to having no clear rankings, are not perceived as offering more financial aid. There is also no perception that the higher the ranking, the more financial aid received. By contrast, first-generation students predict that top 25 colleges offer approximately $6000 more in financial aid relative to colleges without rankings (marginally statistically significant at p=0.09), but they do not predict a gradient in financial aid offered as a function of rankings. None of the other coefficients are statistically significant.

A key question is whether the differences in perceived cost remain after accounting for financial aid. When considering net cost (annual tuition and fees subtracting financial aid), the interaction effect comparing first-generation and continuing-generation adolescent perceptions of ranked and unranked colleges is $7,954 (p=0.04). That is, even accounting for financial aid, first-generation adolescents perceive rankings to be a stronger signal of cost than continuing-generation adolescents, and this difference is statistically significant.

Hypothesis 4:
The interaction effect between the indicator for first-generation status and whether a college profile is labelled as being ranked in the top 25 is statistically significant (p=0.021), implying that elite signals for rankings (being in the top 25) led to divergent selections for first-generation and continuing-generation adolescents. Put differently, when comparing a top-ranked college with one with unknown rankings, a continuing generation student is approximately three times more likely to select the top ranked school, as compared to a first-generation student. The interaction effect between the indicator for first-generation status and for colleges ranked between 25 to 50 is also of substantial magnitude (5.8 percentage points), although it is not statistically significant (p=0.102). Colleges with lower rankings than 50 were no more likely to be chosen by first-generation or continuing-generation adolescents, corresponding to interaction effects that are near zero in magnitude and not statistically significant. When taken together, these results suggest that, after controlling for key characteristics of colleges like their size, location, and graduation rates, college rankings still have a strong, independent effect on adolescent college choice, and that this signal is stronger for continuing-generation adolescents.

***

Standard errors in all models below are adjusted for clustering by respondent, as each individual i is asked to repeat the same task (k) twice.