Methodology


The addition of the Covid-19 Response and Community Supports sections adds eight new measures, bringing the total to 6 categories and 27 measures on both the survey and the Report Card.  The 6 categories are Student Outcomes, Covid-19 Response, Community Supports, Academic Engagement, Financial Management, and Intervention.  Each category is weighed equally making them worth approximately 16.67% of the final score.

The measures are based on ScholarShot “Best Practices,” derived from our experience working with FGLI Scholars, extensive research in the field, and collaboration with higher education partners.  These measures grasp a majority of the supports that universities use to engage their FGLI students.  However, the Student Outcomes category measures which schools’ supports are affecting the performance for their FGLI students through and post-graduation.

Based on discussions with higher education partners, we chose to only publish the Report Card with universities who achieved an A or B final score.  Our goal is to highlight the positive ways that universities engage FGLI students and still motivate all to improve their practices.  We offer, however, individual reviews with all universities that participated in the survey and encourage them to use this opportunity to create better outcomes for their students.  As with similar reports, we chose to curve our scores.  In this case, we curved the results by 10 points.

 

Categories

The Student Outcomes category is quantitative, using measures solely from the Department of Education data.  These range from total cost of attendance to post-graduation earnings, but ultimately all measure how well FGLI students fare after graduating from the universities. 

The remainder of the categories are qualitative measures surveyed directly from the universities.  While there is a margin of error on the accuracy of the universities self-reporting, ScholarShot found no abnormalities or questionable responses and believes the universities answered the survey questions accurately.

Each category contains the measures that are most relevant to that category.  Below you can see how the categories and measures are organized and how we define each measure:


Graduate Outcomes

Loan repayment rate The percentage of first-generation students who are on track to pay off their loans 5 years after completing
Median Debt The median debt for students who have completed
Total Cost The total price for in-state students (average of on and off campus living)
Graduation The percentage of first-generation students who completed within 6 years at their original institution
Median Earnings The median earnings of students working and not enrolled 10 years after entry

Covid-19 Response

Covid-19 Communication

How often and how many types of communication schools are using to communicate with students about Covid-19

Covid-19 Surveys

How often schools survey about Covid-19 and what is being covered in the surveys as well as the response time to any student need

Work Study

Whether work study jobs have increased, decreased, or remained the same during Covid-19

Covid-19 Supports

How many supports are available virtually, whether support availability has increased, decreased, or remained the same, and which supports are free to students

Covid-19 study areas

Whether schools have quiet spaces available on campus for students to study and whether they are using proper Covid-19 protocols in those areas

Healthcare during covid-19

Whether all students have the ability to access student health during Covid-19, only students with insurance, or no students

Community Supports

Summer Bridge Program

Whether the school has a summer bridge program, the cost for students, and how many FGLI students can participate

Living-Learning Community

Whether the school has FGLI and/or racial/ethnic based living-learning communities that support the students both academically and socially

Text Message Communication

Does the school utilize text message communication to alert parents and students about important information

Mentor Program

Does the school have a mentor program for FGLI students and the percent of FGLI lowerclassmen being mentored

Academic Engagement

Students per academic adviser

Ratio of students per academic adviser at each university

Early Alert System

Whether the school has a university-wide Early Alert System that automatically alerts student and adviser when there is a problem with the student’s alert factors i.e. failed test, unpaid tuition, etc.

Response to early alert

Whether the adviser, student, or both are required to respond to early alerts and how they are required to respond

Time limit for professors

Do the universities have a mandatory time limit in place for professors to return grades and if so, what is that time limit

Financial Management

Debt Program

Whether universities have a debt management, debt counseling program, or both and whether the programs are mandatory for all students, students taking out loans, or no students

Emergency Funding

Do the universities have non-loan emergency funding available for students due to a temporary or unexpected hardship

Financial Gap

Do the schools track and analyze each incoming student’s financial gap and if so, what is the average annual financial gap

Student Services and Interventions

Probation interventions

Does the university have a universal response when a student is placed on probation and if so, what is that response

DROPOUTS in good academic standing

Do the universities track students who drop out but are in good academic standing and if so, what percent of students dropping out are in good academic standing

Mental Health to student ratio

The number of undergraduate students per mental health professional at the university

Mental Health Visits

How many visits, on average, a student makes to on-campus mental health services before being referred to an outside provider

General Healthcare

Is student healthcare available to all students, regardless of health insurance status

Comparing Other University Reporting Methods

There are numerous organizations that also report on how colleges compare to one another, most notably the US News and World Report and the Chetty Report on student economic mobility.  These reports collect a range of historical data to produce their reports.  The ScholarShot University Report Card has some similarity in that our Student Outcomes category (16.67% of the total grade) uses historical data on university performance.  The other five categories, however, (Covid-19 Response, Community Supports, Academic Engagement, Financial Management, and Student Services and Interventions) are quite different from any other report because they come directly from our experiences working with FGLI students at ScholarShot.  The data for these qualitative categories was acquired through a survey completed by the participating universities.  These categories measure practices and policies at each university that directly affect FGLI students and their chances to succeed, or not.  See our Best Practices section for more discussion of these measures.

 

University Participation

During our research, we requested that all 35 public universities in Texas participate in our survey.  Due to the Covid-19 Pandemic, only 22 were able to participate this year:

Angelo State University
Midwestern State University
Prairie View A&M University
Sam Houston State University
Stephen F. Austin State University
Sul Ross State University
Tarleton State University
Texas A&M International University
Texas A&M University-Central Texas
Texas A&M University-Commerce
Texas A&M Corpus Christi
Texas A&M-Kingsville
Texas A&M-San Antonio
Texas Tech University
Texas Woman’s University
The University of Texas at Arlington
The University of Texas at San Antonio
The University of Texas Permian Basin
The University of Texas at Tyler
University of Houston-Clear Lake
University of North Texas
University of North Texas at Dallas

We are grateful for each of these universities’ willingness to participate and help create a better future for all students in Texas even during this difficult time.  The following 13 schools declined to participate and thus are not included in the Report Card:

Lamar University
Texas A&M University-College Station
Texas A&M University-Texarkana
Texas Southern University
Texas State University
The University of Texas at Austin
The University of Texas at Dallas
The University of Texas at El Paso
The University of Texas Rio Grande Valley
University of Houston- Main Campus
University of Houston-Downtown
University of Houston-Victoria
West Texas A&M University

These schools will have the opportunity to participate in future years so that Texas students, parents, and high school counselors can have the opportunity to see what they have to offer FGLI students.