Why and How: Thoughts From Researchers and Practitioners on Data Collection

Gary Rhodes
Associate Dean, International Education & Senior International Officer,
College of Extended & International Education
Director, Center for Global Education
California State University at Dominguez Hills

It’s important to provide data to reinforce your office’s efforts to support your institution, its faculty, and its students.  Colleges and universities use data to document the impact of programs across campus.  Each institution and each study abroad office may use different measures and highlight different outcomes as a part of their data measurement process.

The March 2014 Terra Dotta Newsletter discussed research focused on community college study abroad.  This month’s article focuses on why and how four year colleges and universities in the field are collecting data. These statements are organized in alphabetical order by the name of the institution.  You can find additional background on study abroad retention and success research at:  https://globaledresearch.com/study-abroad-impact.asp.

One of the most exciting parts of this list of quotes is how it signals the professional advancement of the study abroad field.  When I started in the field in 1989, the amount of research being done, by both practitioners and faculty/researchers about study abroad was limited.  It is exciting to see the many study abroad practitioners who are integrating research into their work and informing practice as well as others in academic departments who are doing research focused on study abroad.

California State University at Los Angeles, Tasha Y. Willis, EdD, MSW, Assistant Professor CSU Los Angeles School of Social Work

We have growing research evidence that study abroad is a powerful catalyst for change and growth among students who most often participate.  What we are largely missing is data on how the experiences of students whose social identities complicate their journeys abroad may differ: students of color, students with disabilities, first generation students, students from lower SES, community colleges, and students who identify as part of the GLBTQ community.  Armed with data, proponents of study abroad are better equipped to advocate not only for increased access, but also to support students before, during and after their trips, thereby enhancing their experiences and learning outcomes.

For example, I am working to disseminate data I collected from African American community college women who studied abroad: while they benefited exponentially, they also experienced racial and gender micro-aggressions, leading us to reflect upon our orientation, in-country and re-entry processes.  We are currently analyzing data on Latina, Asian American and African American social work students who studied on a short term program in Thailand.  In just 10 days, they grew personally and professionally, getting involved locally on the international issue of human trafficking.  We are also undertaking a study on Spanish-speaking, Latina/o social work students who will participate in a 6 week internship in Costa Rica: we are interested in exploring the professional, ethnic and gender identity development of non-Costa Rican Latino/a sojourners.  The implications are yet to unfold but may yield valuable insights for social work education abroad.

Advice for others interested in collecting data on SA?

I certainly encourage others to explore underrepresented and understudied student populations.  Personally, I'm beginning to consider how to incorporate participatory action research methods into my work, in addition to more traditional research methods. I think this lends itself well to study abroad practitioners, who seek to help deepen student learning upon re-entry.  I look forward to hearing from others who may already be doing this or would be interested in discussing the possibilities!

Marquette University. Gail Gilbert, MBA, Assistant Director, Office of International Education

How are you using data to evaluate the levels of global learning taking place through study abroad?

Many of the methods used to assess student learning in study abroad programs are indirect measures (i.e. students are self-reporting through an inventory or survey tool).  One way to use Terra Dotta to assess students more directly is through reflection questions that can be measured against the AAC&U VALUE rubrics, specifically intercultural knowledge and competence (https://www.aacu.org/value/rubrics/pdf/InterculturalKnowledge.pdf ) and global learning (https://www.aacu.org/value/rubrics/globallearning.cfm).  Marquette University uses questionnaires in the "while abroad" and "returnee" phases to gather student reflections.  A sampling of these responses are evaluated against the VALUE rubrics as one method of assessing the impact of study abroad on student intercultural knowledge and competence as well as global learning.

Michigan State University, Brett Berquist, Executive Director, Office of Study Abroad

What is the importance of data and what are methods to get data about the impact of study abroad on retention and success and career?

Diversity, under-representation, retention and completion rates, repeat study abroad rates ­ there is a wealth of data that institutions should be examining.  Research is emerging that measures the powerful impact of early study abroad participation on students that are already of key concern to our institutions.  Retention and completion rates do not demonstrate causality but are an important initial step to situate learning abroad in the toolkit of early student success strategies.

In the pre-conference workshop last week, it was all about 1. Find out who owns the data you're seeking.  You don't need to reinvent the wheel and in many cases, don't even need to run the data yourself.  Then scan the landscape to determine what alliances already exist to these areas within international writ large.  Finally, approach these conversations within the context of larger university goals, e.g. Early student success, etc.

University of Connecticut, Jeffrey O. G. Ogbar, Ph.D., Vice Provost for Diversity, Professor of History, Office of the Provost

The research on the impact of study abroad has been essential in providing support and enthusiasm for our own efforts to expand the opportunity to study abroad to a diverse cross section of our student body at the University of Connecticut. Without these data, we would clearly be unable to convincingly measure the efficacy of study abroad. After reviewing all Student Support Services students (first generation, low income) who studied abroad since 1999, we confirmed our suspicions: there were double-digit increases in retention and graduation rates of SSS students who participated in international education, even when controlling for GPA. We are always seeking ways to erase achievement gaps with our student body; and numerous studies across the country affirm that study abroad is an incredibly successful tool.

University of San Diego, Jessica Luchessi, Associate Director, International Studies Abroad

1. How has data analysis helped study abroad at USD?  Over the past two years, the International Center has collaborated with the Office of Financial Aid to identify freshmen Pell Grant recipients.  Financial Aid then contacts these students to let them know that as Pell recipients, they are eligible for USD International Center scholarships that can be applied to USD faculty-led programs.
2. Advice to Other Study Abroad Staff on Data Collection: It is most helpful to gain a better understanding what type of data is already collected at the institution.  One place would start would be Institutional Research, and they can then recommend which office to reach out to.  After comparing what data is available and what is not, a more strategic approach to collecting additional data can be developed.
3. How can Terra Dotta Software help in data analysis? USD is currently exploring the possibility of granting Terra Dotta Software access to key individuals in various offices across campus to streamline data analysis.  We hope to begin with the Office of Student Conduct, which will allow analysis of on-campus conduct trends of study abroad applicants prior to going abroad.  This information will inform both the International Center and Student Conduct on best practices related to study abroad and how to prepare students for the experience.
4. Data to Confirm Impact of Second Year Study Abroad Program:  To measure the impact of this program, we compared retention rates of sophomore students (77%) to retention rates of those who participated in the SYE Abroad program (96%).  Since the aim of the program was to engage sophomore students through this experience, this data gives us insight on the program’s impact on retention and success.

I hope that these statements have provided some insights into why data on study abroad is important as well as some suggestions on ways to collect the data.  This can include practical data on housing and student satisfaction with courses, professors, and excursions to enhance programming.  It can be focused on special issues like a safety incident survey or to respond to Clery Act requirements.  However, it can also go deeper and look at international and intercultural learning, the impact of study abroad on retention, success, and career.

As highlighted in the November 2013 Terra Dotta Newsletter Article, study abroad administrators have many responsibilities.  Some institutions will have staff in the study abroad office who have the expertise and time to take on the research to analyze data.  At other institutions, collaborations with others in the Institutional Research Office or other departments on campus can help support the development of goals for data collection and analysis.

Either way, as you work to obtain additional support for your efforts, integrating the collection and analysis of data can make an important impact on improving the quality of programs, obtaining greater respectability for your work, getting greater support from other departments and central administration, and professional development for study abroad staff.

Thanks to my colleagues at universities across the U.S. who provided their insights on data collection and analysis at their institutions for this article.