Measuring what matters: How much should we really care about retention?

In his February 2010 TED talk, CEO and author Chip Conley tells a fascinating story about the nation of Bhutan and their transformation from an isolated, undeveloped nation to a modern, technologically rich nation that still manages to maintain the essence of their original culture and traditions.  Bhutan's story is one of striking the balance between progress and innovation, while stilll maintaining core elements of an identity (a rare feat for any country, organization, or school in today's rapid-paced world).  One of the most interesting parts of Conlee's telling of the story is his reason for Bhutan's ability to transorm in these ways.  Quite simply, he believes that it is because Bhutan has learned to "count" the right thing, gross national happiness.  Forty years ago, Bhutan's King coined the term rather off-handedly to describe his commitment to building an economy that would allow for growth, while staying true to Bhutan's Buddhist roots.  Bhutanese officials ran with the concept, developed sophisticated instruments to measure the concept, and used it as a model for the development plan that brought Bhutan into the 21st century.  This all stands in stark contrast to most nations' preoccupation with Gross Domestic Product and their belief that it stands as the supreme indicator of a nation's well-being.

There seems to be a lesson for higher education in all of this, particularly the first-year experience movement.  Like anyone else, we count what is easily countable.  So, in many ways "retention" has become our GDP.  We work hard to measure it, argue over how it should be measured, showcase (or hide) it in reports to our administrations, and tout it at conferences.  This isn't to say that we shouldn't care about retention--the reality is that enrolled students bring money to the institution and that money keeps us running.  But, there seems to be some danger in retention becoming what Conlee describes as a "misplaced metric," an easy to count measure that gives little indication as to the real health of an institution.

So, what should we be counting?  In many ways this question hinges on how we define success in the FYE movement and the factors that we believe contributes to a vibrant campus community.  What does a "successful" student look like at the end of their first year?  What skills, habits, and attitudes would they possess?  While the National Survey of Student Engagement (NSSE) has contributed greatly to institutions' ability to measure certain behaviors and attitudes of students,  we don't seem to do much on our individual campuses to measure the equivalent of the "gross national happiness" for our campuses.  

What would these "intangibles" on our campuses be?  While they will vary slightly across institutions, some possibilities might include

A personal reason for being at a particular institution.  Do students know why they decided to come to your school?  Their purpose and commitment to the educational ideals and objectives at your institution will make a huge difference in their engagement and persistence.  If they don't have a set of fairly good reasons for choosing a particular campus, there is likely to be trouble down the road.

Understanding of and investment in an institutional mission.  This seems strongly correlated with the idea above, but it seems important for institutions to not only orient students to their physical surroundings, but to help them understand the culture and ideals of the institution they have enrolled in so they might become a part of the community and fulfill their role in it.  So, if you are a faith-based institution that espouses character development, do students believe in that mission and pursue that growth?  For liberal arts institutions, do your students value a well-rounded education and recognize the importance of breadth in their learning?  

Passion for learning.   What students believe and feel about learning are important.  We want life-long learners that continue to grow and make contributions to society after they leave our institutions.  Can the first-year experience nurture this passion?  How would it be measured?

This is obviously not an all inclusive list.  But, these would seem to be key indicators of the success of a FYE program.  There are others including deep learning behaviors, formation of supportive mentoring relationships, and the development of grit and persistence.  We probably can't measure them all, but what for you are the key indicators on your campus?  What could we start measuring on our campuses that would be meaningful and give us real insight into the success of our FYE programs?  

Universities need leaders who know what to count.