sierraclub.org - sierra magazine - sept/oct 2011 - coolschools - methodology
Sierra magazine's "Coolest Schools" ranking is open to all four-year undergraduate colleges and universities in the United States. In March 2011, Sierra sent a 12-page questionnaire to 940 schools, a list based on the widest collection of university contacts that the magazine's researchers could amass; schools that requested a survey were provided one and encouraged to participate. Sierra received 118 complete responses from schools, which the magazine's researchers scored and ranked. There was no cost for participation, but schools that did not respond were not included in the ranking. Furthermore, no affiliation or relationship between a school and the Sierra Club or its employees, past or present, influenced the ranking.
Members of Sierra's editorial team constructed the survey with guidance from the Sierra Club's Conservation department, as well as from campus sustainability experts. Its questions center on environmental goals and achievements, with priority given to achievements. Its 10 categories include energy supply, efficiency, food, academics, purchasing, transportation, waste management, administration, financial investments, and a catch-all called "other initiatives." Each category was worth a maximum of 10 points, for a possible grand total of 100; individual questions were worth between .5 and 6 points to reflect their relative importance.
Where appropriate, standards such as LEED certification were referenced. Sierra acknowledges that some schools choose not to invest in expensive certifications so that money is available for other greening programs. From an evaluator's perspective, however, certifications are useful for measuring practices across a diversity of schools.
Evaluation was based primarily on schools' responses to the survey, but when appropriate, Sierra's researchers made follow-up inquires by phone and e-mail, and used outside sources to verify and complement the information in the survey. Final ranking decisions, however, were based on the scoring key.
Because of differences in schools' abilities to use the drop-down answer function in the interactive PDF, some schools provided additional materials or URLs. All submitted materials were considered, though no extra credit was given for longer answers, multiple files, or added graphics. At least one school in the top 10 used only the space provided without drop-downs.
The scoring key, produced in consultation with the Sierra Club's Conservation department, was created before evaluation began and is publicly available. Graders were primarily interested in schools' actual practices, not in their ability to fill out a survey. If an answer to one question was found elsewhere on the survey, that answer was considered without regard to its location. The surest way to have been awarded full credit, however, was to provide a clear answer in the appropriate space.
The resultant "Coolest Schools" ranking is an index that provides comparative information about the most important elements of campus sustainability.
A school that scored a zero on the index would be one that had not begun to pursue any environmental initiatives, and a school that scored 100 would be at or near sustainability. While many universities are making admirable progress, at present, no school has attained complete sustainability. The top-rated universities scored in the low 80s, indicating much work completed, but also room for improvement.
For each question on the survey, at least one school received zero points, and at least one school received full credit, demonstrating both the challenge that was set for schools, and that Sierra asked about realistic benchmarks. Some schools excelled at specific elements of the survey. However, the top schools were consistent across the categories. Overall, schools' scores were lower in 2011 than they were in 2010. This is due, in large part, to survey revisions made in regard to questions about schools' financial investments.
More than 2,000 four-year colleges and universities exist in the United States, so there are, of course, schools that care about the environment that don't appear on Sierra's list. That said, the magazine's ranking acts as a guide for prospective students who seek a way to compare colleges based on commitment to environmentalism. The ranking also serves to spur competition, create aspirational standards, and publicly reward the institutions that work hard to protect the planet.