On Target: Using a Needs Assessment and Data to Plan Professional Development

Concurrent Session 5

Brief Abstract

Do you want to make faculty professional development meaningful? Understanding whether a skill impacts daily work and if faculty are competent in that skill are key to meaningful development.  In this interactive session, participants will explore a multi-dimensional, competency-based needs assessment instrument and design their own faculty development needs assessment.

Presenters

Lead Presenter: M. Aaron Bond, Virginia Polytechnic Institute and State University

Dr. Aaron Bond has worked in the field of instructional technology, distance education, and professional development for more than 20 years. He has served as a corporate trainer, face-to-face classroom instructor, an online instructor, and as a secondary principal. Currently he serves as the Senior Director for Learning Services in Technology-enhanced Learning and Online Strategies at Virginia Tech where he oversees faculty professional development, learning technologies, and instructional design. 
He has an earned B.A. in History from Mary Baldwin College, a M.A. Interdisciplinary Studies from Fort Hayes State University, an Ed. S. Educational Leadership and Policy Studies from the University of Virginia, and a Ph. D. from Virginia Tech in Curriculum and Instruction. 
 His research includes authentic learning experiences, communities of practice, socio-cognitive factors on learning, and distance learning. He is the co author of two books: 'Building Virtual Communities of Practice for Distance Educators' and 'Systems Thinking for Instructional Designers: Catalyzing Organizational Change.'

Co-presenter: Marc Zaldivar, Virginia Tech

I taught English courses for 10 years, and also worked simultaneously in the Writing Center, culminating in 2 years as the Director of the Virginia Tech Writing Center. As I began my Ph.D. program, I soon began working in what was Learning Technologies at the time, supporting what was the Faculty Development Institute (which is now the Professional Development Network). After completing my Ph.D., I worked with LT, which became TLOS, on ePortfolios, authentic assessment, and other instructional-design-related activities. Currently, I have returned to my 'roots' in the Professional Development Network, helping faculty at Virginia Tech continually improve.

Co-presenter: Hamidreza Taimoory, Virginia Tech

Hamidreza is a Ph.D. student in Engineering Education and has a master's degree in industrial engineering at Virginia Tech (VT). He has worked in the industry as a research and development engineer. He is currently a data analyst in TLOS (Technology-Enhanced Learning And Online Strategies) at VT. His expertise is in quantitative research. His primary research interest is motivation, co-curricular activities, and professional development in engineering education.

Extended Abstract

Introduction 

Professional development staff and instructional designers often give faculty what we think they need without fully considering their opinions when designing training, consultations, and other services. This often overlooked practice may have implications for missed opportunities in individual faculty growth. As higher education institutions grapple with increased use of online teaching and learning to meet strategic priorities, faculty development needs must be considered. Many higher education faculty are facing increased workloads and expectations without the individualized support necessary to meet the increased demand. This express workshop will explore a faculty work-related needs assessment instrument used at a large R1 institution to determine faculty technology-based needs and provide guidance for replicating the instrument in other institutions. The strategies described in this session are meant to be easily replicated and applicable to any higher education setting. Determining a faculty member’s comfort level with specific tools, innovations, or strategies allows for the development of a meaningful, personalized professional development plan.

Theoretical Background

Assessment is a tool that enables educators to develop a strategic framework in course design. Needs assessment instruments can reveal topics and skills in areas where learners need growth (Palomba & Banta, 1999). Needs assessments are a fundamental approach to the creation of workshops and courses for professional development programs. Most needs assessment instruments consider the importance of needs in only one dimension (Ullian & Stritter, 1997). While asking learners about specific needs may be beneficial, it might not be enough. A multi-dimensional assessment might help prioritize the workshops developed. Separating measures into two dimensions: need and importance, leads to a better understanding of professional learner needs.(Quirk et al., 2002). 

Workshop Details

A competency-based needs assessment instrument was developed and piloted with 500 faculty members in the fall of 2017. The needs assessment has now been administered for 5 years and more than 2,500 learners have provided input into their own learning plans. Incorporating 25 competencies developed by a cross-disciplinary committee, the needs assessment informs prioritization of professional learning curriculum areas and helps advise  the development of personalized learning plans for individual learners. 

For each competency, participants rate the importance of the competency to their work and rate their current level of ability with the competency on a five-point Likert scale. The gap or discrepancy score between the two values was used to compute a Mean Weighted Discrepancy Score. The 25 competencies are ranked to help identify "big picture" gaps across different groups of learners. Additionally, each of the 25 competencies are ranked for each individual learner to provide a suggested list of professional development offerings to fill gaps identified by the data.

This session will describe the 25 competencies, the data analysis process, and lessons learned from 5 years of data collection and data-informed decision making. After the initial introduction of the competency-based needs assessment instrument and initial findings, participants will be asked to work in teams to identify competencies that balance their own institution's goals with perceived learner needs, participate in guided practice creating a multi-dimensional assessment, and practice using data to make decisions that impact student learning. 

The workshop will consist of 4 parts:

  • Real examples of how multi-dimensional measures can be used as a needs assessment instrument to find gaps in faculty development programming.

  • Hands on exercises for narrowing categories and competencies for faculty development needs assessment at a participant's given institution.

  • Demonstration and practice building a multi-dimensional measure that makes sense for a given participant’s context.

  • Practice extrapolating the data and making decisions based on the information. 

Most of the session will be group work with teams collaborating to create a set of competencies and strategies for incorporating a multi-dimensional methodology in needs assessment into their planning processes. Each group will be given a framework to guide discussion based on design-thinking exercises. Time will be allotted for group share outs and questions.

 References

Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: planning, implementing, and improving assessment in higher education. Higher and Adult Education Series. ERIC.

Quirk, M., Stone, S., Devaney-O’Neil, S., Mazor, K., Starr, S., & Lasser, D. (2002). Using    differences between perceptions of importance and competence to identify teaching needs of primary care preceptors. Teaching and Learning in Medicine, 14(3), 157–163. https://doi.org/10.1207/S15328015TLM1403_4

Ullian, J. A., & Stritter, F. T. (1997). Types of faculty development programs. Family Medicine, 29(4), 237–241.