Developing a Multi-Perspective Usability Evaluation for integrated study system management Based on Multi-criteria Decision Analysis

The usability and effectiveness of information systems in educational institutions have a high impact on the quality of teaching and learning experiences. Despite the increasing adoption of such systems in educational institution, an inclusive evaluation for the usability that covers all user perspectives (Student, Academic, and IT staff) is subject to further study and assessing. To address this gap, this project work proposed an integrated of a multi-perspective usability evaluation framework based on the Multi-Criteria Decision Analysis (MCDA) methods which developed to assess the usability of InSIS and Moodle as a case study. The proposed comprehensive evaluation framework grounded in ISO 9126 standards. A key focus of the research is identifying critical usability criteria and measuring system performance through structured data collection across the studied institution, using Fuzzy Weighted Zero-Inconsistency (FWZIC) and Fuzzy Decision by Opinion Score Method (FDOSM). The results indicate Security, Understandability, and Learnability was the highest weight values (0.121+) in evaluation from the Academician and Students perspective, while stability and recoverability were taking the highest weight value (0.102) from the IT staff perspective. The variance in the important weight values per perspectives, reflect the necessity of considering all perspective in evaluation and prove the proposed hypothesis of this project. Consequently, the features/tools of the studied systems (InSIS and Moodle) show differences priority as well. The comprehensive results indicate that Moodle outperforms InSIS in terms of operability and user satisfaction and interaction among students and faculty. However, InSIS demonstrates notable strengths in functionality and efficiency, particularly from the IT staff perspective. Given the evolving nature of the field, the framework and findings should be continuously refined through further research. This study insights and recommends future work in applying the framework to larger-scale environments and exploring the integration of emerging technologies for enhanced usability evaluation. Ultimately, this project contributes a multi-perspective framework that not only supports usability assessment but also provides a framework for improving the functionality and user experience of educational information systems.