An exploratory analysis of major dropdowns in Student Evaluation of Teaching ratings in higher education
Keywords:
Downloads
Type:
Text fileAbstract
We present an exploratory study of Student Evaluation of Teaching (SET) surveys that focused on analyzing the major dropdowns that sometimes occur in SET scores of those university teachers who teach the same subject for at least two consecutive years. Specifically, the purpose of this study was to identify those SET survey items associated with major dropdowns in teacher evaluations. The sample comprised SET surveys ratings from one Spanish University over five years, for a total of 2,284 different teacher-subject pairs under analysis. The 5% percentile of the differences between the teacher and subject SET ratings in two consecutive years was used as the cut-off point to identify major dropdowns. Frequency tables were used for analysis. The results highlighted SET survey items related to teaching methodology as those most associated with major dropdowns in teachers’ ratings. In particular, the item concerning teachers’ compliance with class schedules showed the highest frequency of occurrence. Beyond the specific scores obtained or the specific items in which major dropdowns may take place, in this study we discuss the importance of incorporating major dropdowns analysis to SET surveys reports in the future.
Downloads
References
Aguillo, I., Bar-Ilan, J., Levene, M., & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85(1), 243-256. https://doi.org/10.1007/s11192-010-0190-z
Google Scholar CrossrefASA, American Sociological Association (2019). Statement on Student Evaluation of Teaching. https://www.asanet.org/sites/default/files/asa_statement_on_student_evaluations_of_teaching_sept52019.pdf
Google Scholar CrossrefAsare-Nuamah, P. (2017). International Students’ Satisfaction: Assessing the Determinants of Satisfaction. Higher Education for the Future, 4(1), 44-59. https://doi.org/10.1177/2347631116681213
Google Scholar CrossrefBedggood, R. E., & Donovan, J. D. (2012). University performance evaluations: what are we really measuring? Studies in Higher Education, 37(7), 825-842. https://doi.org/10.1080/03075079.2010.549221
Google Scholar CrossrefBritish Educational Research Association (BERA) (2018). Ethical Guidelines for Educational Research. London: BERA.
Google Scholar CrossrefBoring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. Science Open Research. https://doi.org/10.14293/s2199-1006.1.sor-edu.aetbzc.v1
Google Scholar CrossrefCapa-Aydin, Y. (2016). Student evaluation of instruction: comparison between in-class and online methods. Assessment & Evaluation in Higher Education, 41, 112-126. https://doi.org/10.1080/02602938.2014.987106
Google Scholar CrossrefCasero, A. (2008). Propuesta de un cuestionario de evaluación de la calidad docente universitaria consensuado entre alumnos y profesores. [Proposal for a consensual questionnaire for evaluating the quality of university among students and teachers]. Revista de Investigación Educativa, 26(1), 25-44. https://revistas.um.es/rie/article/view/94091
Google Scholar CrossrefCastro-Morera, M.; Navarro-Asencio, E. & Blanco-Blanco, Á. (2020). The quality of teaching as perceived by students and university teachers: analysis of the dimensionality of a teacher evaluation questionnaire. Educación XX1, 23(2), 41-65. https://doi.org/10.5944/educXX1.25711
Google Scholar CrossrefCentra, J. A., & Gaubatz, N. B. (2000). Is there gender bias in student evaluations of teaching? The Journal of Higher Education, 71, 17-33. https://doi.org/10.1080/00221546.2000.11780814
Google Scholar CrossrefChan, C. K. Y., Luk, L. Y. Y., & Zeng, M. (2014). Teachers’ perceptions of student evaluations of teaching. Educational Research and Evaluation, 20(4), 275-289. https://doi.org/10.1080/13803611.2014.932698
Google Scholar CrossrefChing, G. (2018). A literature review on the student evaluation of teaching: An examination of the search, experience, and credence qualities of SET. Higher Education Evaluation and Development, 12(2), 63-84. https://doi.org/10.1108/HEED-04-2018-0009
Google Scholar CrossrefClayson, D. E. (2009). Student evaluations of teaching: are they related to what students learn? A meta-analysis and review of the literature. Journal of Marketing Education, 31(1), 16-30. https://doi.org/10.1177/0273475308324086
Google Scholar CrossrefClayson, D. E., & Sheffet, M. J. (2006). Personality and the student evaluation of teaching. Journal of Marketing Education, 28, 149-160. https://doi.org/10.1177/0273475306288402
Google Scholar CrossrefGriffin, B. W. (2001). Instructor reputation and student ratings of instruction. Contemporary Educational Psychology, 26(4), 534-552. https://doi.org/10.1006/ceps.2000.1075
Google Scholar CrossrefGriffin, B. W. (2016). Perceived autonomy support, intrinsic motivation, and student ratings of instruction. Studies in Educational Evaluation, 51, 116-125. https://doi.org/10.1016/j.stueduc.2016.10.007
Google Scholar CrossrefGuilbault, M. (2016). Students as customers in higher education: reframing the debate. Journal of Marketing for Higher Education, 26(2), 132-142. https://doi.org/10.1080/08841241.2016.1245234
Google Scholar CrossrefHenard, F. (2010). Learning our lesson: Review of quality teaching in higher education. Institutional Management in Higher Education, OECD Publishing.
Google Scholar CrossrefJohnson, R. (2000). The authority of the student evaluation questionnaire. Teaching in Higher Education, 5, 419-434. https://doi.org/10.1080/713699176
Google Scholar CrossrefKomotar, M. H. (2019). Global university rankings and their impact on the internationalisation of higher education. European Journal of Education, 54(2), 299-310. https://doi.org/10.1111/ejed.12332
Google Scholar CrossrefMarsh, H. W. (2007a). Students’ evaluations of university teaching: A multidimensional perspective. In R. P. Perry and J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence based perspective (pp. 319-384). New York: Springer.
Google Scholar CrossrefMarsh, H. W. (2007b). Do university teachers become more effective with experience? A multilevel growth model of students’ evaluations of teaching over 13 years. Journal of Educational Psychology, 99(4), 775-790. https://doi.org/10.1037/0022-0663.99.4.775
Google Scholar CrossrefMau, R. R., & Opengart, R. (2012). Comparing Ratings: In-class (paper) vs. Out of Class (online) Student Evaluations. Higher Education Studies, 2(3), 55-68. https://doi.org/10.5539/hes.v2n3p55
Google Scholar CrossrefMolero, D., & Ruiz, J. (2005). La evaluación de la docencia universitaria. Dimensiones y variables más relevantes. [The evaluation of university teaching. Dimensions and most relevant variables]. Revista de Investigación Educativa, 23(1), 57-84. https://revistas.um.es/rie/article/view/98341
Google Scholar CrossrefMoreno-Murcia, J. A., Silveira, Y., & Belando, N. (2015). Questionnaire Evaluating Teaching Competencies in the University Environment. Evaluation of Teaching Competencies in the University. Journal of New Approaches in Educational Research, 4(1), 54-61. https://www.learntechlib.org/p/150692/
Google Scholar CrossrefMortelmans, D., & Spooren, P. (2009). A revalidation of the SET37-questionnaire for student evaluations of teaching. Educational Studies, 35, 547-552. https://doi.org/10.1080/03055690902880299
Google Scholar CrossrefNeumann, R. (2000). Communicating Student Evaluation of Teaching Results: Rating Interpretation Guides (RIGs). Assessment & Evaluation in Higher Education, 25(2), 121-134. https://doi.org/10.1080/02602930050031289
Google Scholar CrossrefRichardson, J. T. E. (2005). Instruments for obtaining student feedback: a review of the literature. Assessment & Evaluation in Higher Education, 30(4), 387-415. https://doi.org/10.1080/02602930500099193
Google Scholar CrossrefRindermann, H., & Schofield, N. (2001). Generalizability of multidimensional student ratings of university instruction across courses and teachers. Research in Higher Education, 42(4), 377-399. https://doi.org/10.1023/A:1011050724796
Google Scholar CrossrefScheepers, A. (2019). SET Project: Student Evaluations of Teaching, Measuring and Enhancing Course Quality and Teaching Quality. Rotterdam School of Management. https://equal.network/wp-content/uploads/2019/03/SET_Ad-Scheepers-.pdf
Google Scholar CrossrefSpooren, P., Brockx, B. & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598-642. https://doi.org/10.3102/0034654313496870
Google Scholar CrossrefStark, P. B., & Freishtat, R. (2014). An evaluation of course evaluations. Science Open Research. https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AOFRQA.v1
Google Scholar CrossrefSulis, I., Porcu, M., & Capursi, V. (2019). On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise. Social Indicators Research, 142, 1305-1331. https://doi.org/10.1007/s11205-018-1946-8
Google Scholar CrossrefSurgenor, P. W. G. (2013). Obstacles and opportunities: Addressing the growing pains of summative student evaluation of teaching. Assessment & Evaluation in Higher Education, 38, 363-376. https://doi.org/10.1080/02602938.2011.635247
Google Scholar CrossrefDownloads
Published
Almetric
Dimensions
How to Cite
Issue
Section
License
Copyright (c) 2023 Santiago Leguey, Ana I. Cid-Cid, Rocío Guede Cid, Jaime Prieto
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles are published under Creative Commons copyright (CC BY). Authors hold the copyright and retain publishing rights without restrictions, but authors allow anyone to download, reuse, reprint, modify, distribute, and/or copy articles as the original source is cited.
