10 Leadership Lessons from Higher Education - BY CHRISTOPHER L. WASHINGTON
In ancient mythology, Janus was the Roman god of change and transition. Artistically depicted as having two faces positioned in opposite directions, Janus possessed the ability to see both into the past and into the future. Today, data analytics, which encom- pass the processes of extracting, compiling and modeling data, enable modern man to discover truths about the past and to render forecasts about the future.
I have found that learning analytics, the educational application of data analytics, hold the potential to magnify the view into how teachers teach and how students learn. They also illuminate the environmental conditions under which learning occurs. With learning analytics, I am able to debunk myths, supplant hunches, and confirm or disconfirm intuitions about teaching and learning. Decisions informed by learning analytics have led to a substantial rethinking of instructional methods and their benefits. Additionally, there has been a change in organizational culture from one in which quality is implied by inputs such as faculty credentials, to one that supports systemic assessment, continuous improvement, and greater accountability to stakeholders based on learning outcomes. I present 10 leadership lessons learned from my experience as a Learning! Champion.
LESSON 1: There is a symbiotic and co-evolutionary relationship between e-learning and learning analytics.
While nearly every other profession outside of the academics is required to prove their effectiveness, up until the turn of the 21st century, higher education was largely exempt from external accountability. With an increase in public demands for greater access, lower cost and higher quality education, there was an increase in institutional pressure to demonstrate accountability.
To determine if e-learning methods are as effective as traditional face-to-face modes of instruction, circa 2000 the U.S. Department of Education (DOE) established a pilot project. At the time, there were strict rules limiting colleges and universities from offering more than 50 percent of a program’s courses in any form of distance education. To remove its restrictions on distance education programs, there had to be sufficient justification.
Franklin University was one of the higher education institutions selected to participate in the pilot project. Pilot program participants gathered and analyzed the data, reported it publicly, and noted how the results were used to improve educational processes and practices. This expectation is now a standard for academic quality review in higher education. Based on the data presented as evidence of instructional equivalency, colleges and universities are now able to offer distance education programs and to disburse federal financial aid to students who enroll in them.
Fast forward to 2017. A lot has changed in the past 15 years since the DOE’s pilot project established a foundational framework for the use of data as evidence in determining the effectiveness of e-learning methods compared to traditional face-to-face modes of instruction. Figure 1 reflects some of the contemporary data sources used today to shed light on the effectiveness of curriculum and instruction, students, faculty and the learning environment.
Academic leaders today recognize that student learning experiences both influence and are influenced by factors in and outside of the classroom. Consequently, data is now being collected across multiple systems and treated and analyzed in a more integrated way. At Franklin University, we’ve moved beyond student attitude surveys of faculty members and courses, to an examination of student “clicks” on media, time spent viewing videos through the LMS, or pages read of assigned e-text through the library. We can examine spikes in tutoring requests and send early alerts to academic advisors when students are falling behind on assignments. We can see if faculty have participated in faculty development workshops, and begin to correlate faculty development data with student success data. The activity of our students in relation to interactive media now signal needed improvements to our curriculum design. The result of the relationship between learning analytics and instructional practices is a continuous refinement of questions and analysis techniques, and a resultant evolution of instructional practices.
The adoption and expansion of e-learning methods in higher education continues to this day. According to the “2015 Online Report Card: Tracking Online Learning in the United States," conducted by the Bab- son Survey Research Group, more than 25 percent of the more than 20 million college students in the United States enrolled in at least one course online. Overall growth rates for online course enrollments grew at a rate of more than 7 percent from 2012 to 2014.
LESSON 2: The price of light is less than the cost of darkness.
Higher education institutions (HEIs), places of both progress and tradition, present a special case study for educational leaders who aim to overcome resistance to incorporating new methods and technologies. According to the Babson Study, in 2014, 29.1 percent of Chief Academic Officers believe that members of their faculties accept the value and legitimacy of online education. Many leaders of HEIs perceive value in technology-enhanced instruction but struggle to get faculty members to adopt learning technologies, develop the talent to use it, or to develop the administrative processes to capture the value from learning analytics.
Perhaps the greatest challenge for leaders of HEIs is getting faculty buy-in. The deep traditions of higher education and significant skepticism of e-learning methods require an honest assessment of the effectiveness of current practices, leaving open the possibility of alternatives to traditional approaches to teaching and learning. Educational leaders who can use information to shed light on a culture that reinforces mediocrity are well positioned to develop a strategy that focuses on learner success, continuous improvement, and the use of learning analytics to make data-informed decisions. Existing data elements are essential in establishing a culture where individuals leverage new technologies to in- form teaching practice and develop a level of comfort with learning analytics. For example, existing industry and research reports on the effectiveness and increasing popularity of e-learning methods set the tone for our institutional conversations.
LESSON 3: Strategy comes before measurement.
A clear educational strategy should drive the system of measurement and not the other way around. Measurement tells educators if our strategy is successful or not and where there may be opportunities for improvement. Data is enormously valuable in analyzing the teaching and learning processes. However, when one emphasizes metrics without the proper strategy in place, the result can drive behaviors that lead to data manipulation and other misuses of informa- tion. Lastly, in making the point that data informs rather than drives practice, it is important to clarify the limitations of data and to express a desire to honor the experience and intuitions of faculty and staff members.
LESSON 4: “Quality is not an act; it is the result of intelligent effort.” —John Ruskin
One point often taken for granted in HEIs is that faculty members all have the same definition of academic quality. In fact, members of our faculty had very different ideas about quality, and if and how it could be measured. In defining quality, definitions ranged from the presence of a qualified faculty member, covering subjects, meeting the tradition of the discipline, fidelity to the standards of excellence set by experts, student satisfaction, meeting students goals, meeting faculty members’ goals, the learning process added value, and continuous improvement of the teaching and learning process based on assessment data. In some professional fields, the definition of academic quality should include a larger group of stakeholders that may include employers, associations and professional organizations.
Based on our quality conversations, we shifted from a focus on subject and content coverage to a focus on determining how students can apply knowledge learned in real-world settings. Early conversations also considered questions such as, “What do we dream our students will learn from us in our courses?” and “What would you want graduates to say about their learning experience?” and “What kinds of learning experiences would you want for them in order to succeed after college?”
We identified our goals to: assure high quality instruction across all academic programs; clarify valuable and rigorous learning outcomes for students; assure activities and assignments align to learning outcomes; allow students to experience meaningful and relevant learning activities and assignments; and make instructional materials support the needs of the instructors and learners.
LESSON 5: “Start with the ending; it’s the best way to begin.” —David Wilcox
In academic settings, inputs have long been treasured more than outcomes. Academic ranking services such as the annual U.S. News and World Report college rankings, have a long history of measuring academic quality — not based on student learning success, but based on a myriad of inputs to the learning process. These input measures include but are not limited to admissions selectivity, standardized admissions test scores and admissions rates, alumni donations, student-to-faculty ratio, class size and faculty credentials.
Today, learning outcomes are the currency of higher education, affording transferability of learning and courses between institutions, enabling educators to communicate what is to be learned, and supporting learners’ ability to communicate what they have learned. Learning outcomes are informed by a variety of inputs including but not limited to the educational goals of institutions and learning and performance tasks of employers. Faculty members must therefore agree on basic learning outcomes for each course, and how those course out- comes fit within the overall curriculum.
The adoption of a learning outcomes approach with the aim of identifying the right outcomes expressed at the appropriate level of rigor, revealed a great deal about the teaching and learning process at Franklin University. For example, many faculty members were well versed in subject categories and the topics they wanted to cover but not in writing measurable learning outcomes for learners. An evaluation of our syllabi across all programs and courses revealed inconsistencies in introducing, reinforcing and evaluating course outcomes through learning activities and assignments. Other concerns included unintended redundancies of course materials in some programs, hidden prerequisites, and a skill deficit among our faculty to address evolving manifestations of some rapidly evolving disciplines.
LESSON 6: The bait needs to be attractive to the fish — not to the fisherman.
Most e-learning experiences offered at colleges are organized by faculty members in the same way as the face-to-face version of the course. They are often presented and delivered within the same parameters and schedule, and are evaluated using the same student satisfaction methods. Technology- enhanced curriculum provides opportunities to truly rethink how education is delivered. A clear understanding of learner needs, learning requirements, and of the potential ways learners and educators interact with learning technologies factor heavily in the success of technology-enhanced learning experiences. Some scholars suggest that success in digital learning is more likely if students serve as learning designers and engage in formative evaluation activities; i.e. an evaluation that takes place by the students before learning projects occur, with the aim of improving the project’s design and performance. This approach is quite different from a traditional lecture method where faculty members maintain total control of instruction.
To make education meaningful to the learner, the process of selecting instructional materials is also important. This process is often unmanaged at HEIs, with faculty members teaching each section of a course — often offering different resources to students at different price points. The instructional materials, technologies and virtual learning materials should: be accessible to and used by students; support learning outcomes; and contribute to student success. The selection of appropriate educational technologies is essential to successful teaching and learning. They should match the requirements of learning tasks, and be accessible and easy to use by students. Student surveys can be a place where data is collected on students’ reactions to instructional materials. Increasingly, instructional materials generate their own data, informing faculty about the use and effectiveness of the material in contributing to student learning. Another important data collection consideration is the cost of instructional material relative to other options and relative to perceived instructional benefit.
LESSON 7: “Every line is the perfect length if you do not measure it.” —Marty Rubin
There has always been a way to examine the effectiveness of courses. Prior to the introduction of e-learning methods, faculty members at Franklin University measured student attendance and retention, course completion, and student grades. However, today the capabilities of our learning management system (LMS) allow faculty members to examine student behaviors in relation to their academic achievement. Faculty members can now examine data related to the time students accessed course information, whether they watched all or part of an assigned video, answered questions correctly, responded to and posted to discussion boards, clicked on a lecture, or opened up an email with assignment instructions. The interrelationship between student actions and student success measures (such as course grades or nationally normed exams) allows us to uncover patterns and formulate predictions. Where data is informing student support practices, interventions for supporting students outside of the classroom and within virtual environments are evolving rapidly.
LESSON 8: “Before anything else, preparation is the key to success.” —Alexander Graham Bell
Faculty members are not born with an innate knowledge of how to teach or how to assess student learning. To ensure widespread understanding, we offered faculty members training and development opportunities designed to build a level of comfort and familiarity with e-learning and the use of learning analytics. Training and development allows faculty members to practice the range of teaching and learning methods. The workshops lead to conversations about assessment, encourage faculty to use the language of assessment, and help them gain competence and confidence as teachers using a variety of instructional approaches. A number of historic measures remain important. These include student surveys of faculty and faculty observations based on teaching effectiveness rubrics. In addition, modern LMS technology allows an analysis of faculty behaviors and engagement with the course and with students. The results of measuring teaching effectiveness allows professional development planning and other HR decisions.
LESSON 9: “What gets measured, gets managed.” —Peter Drucker
In many organizations, after the effort to gather and make sense of data, it can be summarized and placed on bookshelves to collect dust. Data collected should be used to make improvements to the course outcomes, instructional methods and ma- terials, or the assessment methods used. Figure 2 illustrates this relationship. In the end, data should inform improvements to student learning. Based on review of the data, our faculty members have achieved a number of the following goals:
>> Redesign of the entire program’s curriculum to better fit market requirements and to avoid irrelevancy;
>> Inform hiring plans for additional faculty;
>> Target improvement of certain student learning outcomes for transferability;
>> Change assignment requirements, supporting materials, and grading criteria;
>> Change student feedback and faculty development practices;
>> Change outcome assessment criteria; and
>> Add learner support services.
Based on student and faculty feedback on courses, university-wide decisions have led to an increase in the perceived value and attractiveness of courses and programs.
LESSON 10: Our future is more data driven.
A number of trends suggest that the future of education will be more data driven. These trends include: (1) advances in technology; (2) looking at the softer side of learning; (3) greater interoperability of data systems; and (4) adaptive learning technologies.
First, analytical software is becoming more advanced and more broadly available. LMS software is becoming more advanced as designers of the software respond to the increasingly sophisticated user by adding new features. Hardware and learning software are also becoming less expensive and more powerful in terms of their computing capabilities.
Second, while economic measuressuch as enrollment and retention, course completion and grades were early indicators as dependent variables, increasingly, faculty are looking at the softer side of learning. These measures include student well-being, their active engagement, and the perceived relevance of the curriculum as it relates to their personal and career aspirations that are believed to be related to their success later in life.
Third, there is a movement toward greater interoperability of data systems. Currently, data silos exist both within and across organizations. As we begin to see the relationship between data sets as predictors of student success, this will drive efforts to have these systems talk to one another. For example, as many community college students enter four-year colleges prior to graduation from their two-year associate programs, data on learning outcomes met or courses taken may be sent from the four-year college back to the community college. This “reverse transfer” may signal the awarding of the two-year degree from the community college, which would positively affect their graduation rates and financial allocations from the community. Another example includes tying faculty development data to student success data. These data sources often reside in different places. Yet, it is believed that good teaching contributes to student learning. Systems that connect these two data sets would more effectively answer questions about the relationship between teaching and learning.
Lastly, adaptive learning is an educational method that uses computers as interactive teaching devices to direct learning tasks and paths based on the users’ competence and their unique needs. Adaptive learning is a form of machine learning that tailors educational experiences based on their responses. These methods produce both activity data and outcome data. As prerequisite knowledge and learning pathways continue to become clearer, adaptive methods will become more effective at individualizing learning.
The data generated by learning technologies such as content repositories, digital learning materials and interactive media objects are magnifying the view into how teachers teach and how students learn. The data also illuminate the environmental conditions under which learning occurs. Decisions informed by learning analytics can influence a culture of assessment and continuous improvement. We have, by no means, perfected our analysis and understanding of quality teaching and student success in higher education. Fortunately, the process of treating teaching and learning as a subject that can be analytically understood is moving forward, nudged by technology and human curiosity. With all this said, individuals and organizations need to constantly consider and develop new measures, new algorithms, and new social processes that enhance our ability to make data informed improvements.
—Dr. Washington is Senior Vice President for Academics at Franklin University. He opened the International Institute of Innovative Instruction, a collective body of learning scientists from across the globe the work to create and teach dynamic and innovative courses. He received the 2016 Learning! Champion Award for exceptional contributions to the learning industry.