The Development & Evaluation of a Machine Learning-Assisted Recommendation System for Generic Competencies Development

The term generic competency (GC) refers to skills and abilities essential to succeed in one's work across multiple disciplines. GC includes teamwork, problem-solving, communication, and critical thinking skills. GC development activities (GCDAs) are essential elements of GC development. However, participation in GCDAs is voluntary and much less structured than a formal curriculum. Therefore, students need advisors to help them choose GCDAs that match their personalities and career aspirations. Using machine learning (ML) to help school advisors make sound recommendations based on a dataset of historical data is a promising solution. However, there is a lack of sufficient research in this area. This research aimed to evaluate the effectiveness of a machine learning-assisted recommendation system (MARS) for GCDAs in higher education. We developed a MARS for students in higher education. The machine learning model of the MARS was trained using school advisors' recommendations of GCDAs for more than 400 student profiles in two previous cohorts. The collaborative filtering and content-based recommendation algorithms were compared when training the model. We found that a LightFM, a hybrid model, provided the best performance. Then, the MARS was given to new incoming students to evaluate its effectiveness in making recommendations. The student's acceptance of the MARS was examined using a survey. We found that the students had positive perceptions about the MARS. Finally, we compared the improvements in the GCs of existing students with two previous cohorts which were collected using different assessment methods.