Journal of Nursing and Practice

Application of Attribute Hierarchy Model in Nursing Skills Assessment Scoring Standard

Abstract

Objective : To develop a nursing skills assessment scoring standard using the Attribute Hierarchy Model (AHM) and evaluate its effectiveness in daily teaching and practice.

Methods : Based on the scoring standards for intravenous indwelling needle infusion in the national vocational college nursing skills competition, assessment indicators were identified. AHM was employed to determine the relative weights of these indicators. The developed AHM-based scoring standard was applied to assess the intravenous indwelling needle infusion skills of 290 higher vocational nursing students. Student performance and the effectiveness of the evaluation were analyzed.

Results : The average student score was 77.59 ± 8.86. 9 cases (3.1%) scored ≥ 90, 96 cases (33.1%) scored between 80 and 90, 146 cases (50.3%) scored between 70 and 80, and 39 cases (13.5%) scored below 70. Analysis revealed deficiencies in several secondary assessment indicators, including non-standard disinfection (63%), inaccurate drip rate adjustment (43%), and failure to perform necessary checks during the procedure.

Conclusion : The AHM-based scoring standard provides a robust theoretical foundation for evaluating nursing skills. The evaluation results demonstrate its scientific validity and practicality. By accurately identifying areas of weakness in student performance, this approach facilitates "process-result" based evaluation and offers a valuable reference for assessing the practical abilities of nursing students.

Keywords

Attribute hierarchy model, Nursing, Skill assessment, Scoring criteria

National Nursing Skills Competition advocates a new concept of practical teaching. It is an assessment tool that reflects the core skill level of nursing students and has been included in the talent training quality evaluation system [1-5]. In order to implement the concept of "competitions promote daily teaching, and daily teaching leads to competitions" [6], the author embeds the key points of operation in the nursing skills competition of the National Vocational College Skills Competition into daily practical teaching and assessment. Each operation in the competition involves multiple evaluation indicators [7]. When applying it specifically, teachers and students question the basis for the score of each indicator. In addition, the full score for intravenous indwelling needle infusion operation in the competition is 20.5 points, while the full score for students' daily operation assessment is 100 points. When the full score changes, the scores of each indicator also change. Therefore, it is very necessary to scientifically determine the scores of each indicator and establish a theoretically based scoring standard to comprehensively and accurately present the students' skill level.

The attribute hierarchical model (AHM) is an unstructured decision-making method based on attribute mathematics. It solves the problem of the failure of the maximum membership criterion in fuzzy comprehensive evaluation by solving the weight of indicators through relative attributes. It can evaluate a single indicator or comprehensively evaluate all indicators. The results are accurate and scientific. At present, it is widely used in bank internal control, earthquake disaster assessment, primary medical status assessment, clinical drug evaluation and internship teaching assessment and evaluation [8-13], but its application in nursing skills assessment and evaluation has not been reported. The author applied AHM to nursing skills assessment and scoring to provide a reference for scientifically evaluating the practical ability of nursing students and reflecting the students' skill level impartially and objectively. Because intravenous infusion is the most basic nursing operation in daily nursing work [14], this study took intravenous indwelling needle infusion as an example.

Objects and Methods

Research subjects

The 23 ^rd grade nursing major and the 21 ^st grade five-year nursing major students of the Medical College of Bozhou Vocational and Technical College were selected as the research subjects. Inclusion criteria for the research subjects: complete information, completion of the assessment on time, and detailed records of the scoring results. Exclusion criteria: ① The operation was not completed within the time limit; ② The specific score records in the scoring were not detailed. In the basic nursing intravenous indwelling needle infusion training course in the spring semester of 2024, the teaching was carried out according to the operation procedures and observation points in the intravenous indwelling needle infusion scoring standards of the nursing skills competition. The final skills assessment used the intravenous indwelling needle infusion scoring standards based on AHM to collect student scores and analyze them. Finally, 290 students' operation scores were included as research subjects.

Methods

Establishment of AHM assessment indicators for intravenous indwelling needle infusion

According to the scoring criteria for intravenous indwelling needle infusion in the nursing skills competition of the higher vocational group of the 2023 National Vocational College Skills Competition, 10 first - level indicators and 34 second-level indicators were determined (Table 1).

Calculation of relative weight coefficients of scoring indicators based on AHM

AHM is a practical multi-objective decision analysis method that decomposes complex problems into different factors in an ordered hierarchical structure. The most critical issue is how to obtain the weight value of each influencing factor [15]. Based on AHM, the relative weight of indicators is established, and their relative importance is compared and the weight of each indicator is established. The weight is quantified into a numerical value for comparison. The numerical value has no specific unit and only represents its relative importance relative to another indicator. The larger the value, the higher the importance, and vice versa [16]. μ _ij represents the importance value of the i ^th indicator relative to the j ^th indicator; μ _ji represents the importance value of the j ^th indicator relative to the i ^th indicator ; μ _ii represents the comparison of the i ^th indicator itself. According to mathematical rules, μ _ii = 0, μ _ij + μ _ji =1. Construct a judgment matrix, and perform a consistency test on the judgment matrix according to formula (1). Then, the relative attribute weight vector of the indicator is obtained according to formula (2). w c( i ) : weight coefficient of the i ^th indicator; n : number of evaluation indicators; μ _ij : importance value of the i ^th indicator relative to the j ^th indicator. By constructing a judgment matrix and performing mathematical operations such as weighted average, the importance of each indicator at each level, i.e., relative weight, is determined [17].

$$\begin{array}{l}g(x)=\left\{\begin{array}{l}1,x>0.5\\ 0,x\le 0.5\end{array}\right.\\ Q_i=\left\{\left.j:g({\mu }_{ij})1,1\le j\le n\right\}\right.\\ g({\mu }_{ik})=g[{\displaystyle {\sum }_{jem}g({\mu }_{ik})}]\ge 0,1\le k\le n(1)\\ W_c(i)=\frac{2}{n(n-1)}{\displaystyle {\sum }_{j=1}^m{\mu }_{ij}}(2)\end{array}$$

Calculation skills assessment operation score

Based on the established AHM intravenous indwelling needle infusion operation scoring standard, the students' operation is scored. According to the determined relative weight coefficient, the corresponding scores of the 34 secondary assessment indicators C1-C34 are assigned. If the students do not operate the corresponding assessment indicators in a standardized manner, the corresponding scores will be deducted, that is, the skill assessment score = 100 - ( 0 or 1 ) × C1 - ( 0 or 1 ) × C2 - ( 0 or 1 ) × C3 - ( 0 or 1 ) × C4 - ( 0 or 1 ) × C5 - ( 0 or 1 ) × C6 - ....... - ( 0 or 1 ) × C34, where if the indicator operation meets the standard, the corresponding indicator score is 0, and if the indicator operation does not meet the standard, the corresponding indicator score is 1.

Statistical analysis

The students' age, gender, weight coefficient and score of the assessment index and the test results of intravenous catheter infusion operation were statistically analyzed using Microsoft Excel 2010. The count data were expressed as cases (%) and the measurement data that conformed to the normal distribution were expressed as ±s.

Results

Construction and score of the first-level assessment indicator matrix

A matrix was constructed based on the AHM method, and the weight coefficients of the 10 first-level assessment indicators were calculated. The corresponding scores were assigned to each first-level indicator based on the daily teaching intravenous indwelling needle infusion operation scoring standard with a full score of 100 points (Table 2).

Construction and score of the secondary assessment indicator matrix

A judgment matrix was constructed to calculate the weight coefficients of the 34 secondary assessment indicators, and then the corresponding scores were assigned to each secondary assessment indicator according to the scores of the primary assessment indicators (Table 3).

Students’ operation scores and the achievement of secondary assessment indicators

290 students, 246 were girls (88.62 %) and 33 were boys (11.38 %); the average age of the students was 19.3 ± 1.24 years old. Based on the scoring criteria constructed by AHM, the highest score was 92.75, the lowest score was 56.14, and the average score was 77.59 ± 8.86. The student scores are shown in Table 4. The statistical results of the secondary assessment indicators that met the standards and those that did not meet the standards are shown in Table 5.

Discussion

Advantages of the AHM-based nursing skills assessment scoring standard

Cao Xi [18] pointed out in his research that in addition to reforming the practical teaching method, teachers should also reform the practical assessment model, establish objective, visual and structured scoring rules, and reduce subjective scoring. Based on the attribute hierarchy model, the author established the hierarchical structure and comparison matrix of AHM, sorted and weighted each scoring factor according to the hierarchy, and obtained the relative weight coefficients of 10 first-level indicators and 34 second-level indicators of intravenous indwelling needle infusion operation. This can help determine the weight of different indicators in the scoring and determine the score of each indicator in the scoring rules. It has a quantitative basis, reduces the influence of subjective factors on the scoring results, and theoretically solves the controversy between fuzzy evaluation and experience evaluation [19], making the evaluation results more reasonable.

Furthermore, the weight coefficient of each indicator is calculated based on the AHM scoring standard. No matter how the total score changes, the weight of each indicator will not change. Multiplying the weight of each indicator by the total score can get the first-level indicator and the second-level indicator score. This solves the problem that the intravenous indwelling needle infusion operation in the 2023 National Vocational College Skills Competition Higher Vocational Nursing Skills Competition has a full score of 20.5 points, while the full score for this operation in the students' usual operation assessment is 100 points, which is inconvenient for daily teaching due to different scores. This AHM -based scoring method can be flexibly adjusted according to the characteristics of different competitions, and the scoring rules can be dynamically optimized and adjusted according to the actual situation of daily teaching, which is both scientific and improves its applicability.

Advantages of application in daily teaching assessment

AHM based scoring standard for intravenous indwelling needle infusion in nursing skills competitions was applied to daily teaching and practical assessment. On the one hand, it achieved a deep integration of teaching and competition [20], and on the other hand, the evaluation was more scientific and reasonable. Studies have shown that when teaching based on the assessment elements of the skills competition, the overall level of students' operational skills is good, and 86.5% of students scored 70 points or above. However, there is still a certain proportion of students who need to strengthen training and improvement. In which aspects do these students need to strengthen training? By counting the non-compliance of the secondary indicators, the weak points of students' operations can be found, helping students to find deficiencies and make up for them, so as to achieve precise teaching and improve teaching effects.

Table 5 shows that students failed to meet the standards for 9 secondary assessment indicators, including 63% for non-standard disinfection, 43% for inaccurate drip rate adjustment, and 38% for failure to check during operation. The analysis found that students had mastered the operation requirements for these three indicators in theory, but they were not aware of the three checks and seven comparisons in actual operation. They remembered to check before operation, but were prone to omissions during and after operation. They knew that disinfection should be done twice and the specific requirements, but the actual operation was not standardized and the awareness of aseptic was not strong enough. They knew how to adjust the drip rate, but the adjustment was not accurate. This may be related to the fact that students mostly stayed in passive learning, thinking that they had mastered it in theory but lacked active practice, and were unable to effectively connect and transform theory with practice, resulting in the inability to strictly abide by the operation in behavior. It is also consistent with previous studies on the characteristics of clinical operation of nursing students and new nurses [21]. Through the feedback of the results, the weak points of the operation of 146 students with poor performance and 39 unqualified students were strengthened, and students were urged to practice diligently to transform theory into skills. The students finally met the standards and their operation skills were improved. The AHM-based scoring standard for intravenous catheter infusion operations not only has a quantitative basis, but also realizes the "process-result" two-way evaluation, optimizes the talent evaluation model, and improves the quality of talent training.

Conclusion

The attribute hierarchy model provides a scientific and effective method for the formulation of the scoring standard for higher vocational nursing skills assessment. Since the research has just started, this article only takes the "intravenous indwelling needle infusion" operation in the nursing skills competition as an example. In future research, we will try to apply it to other higher vocational nursing skills operation projects to further test the feasibility of the nursing skills assessment scoring standard based on AHM. We also hope that more nursing educators and managers will jointly explore the application of the attribute hierarchy model in the reform of nursing assessment and evaluation to promote the high-quality training and development of nursing talents.

Funding

2021 Anhui Province College Excellent Young Talents Support Plan Key Project (No.: gxyqZD2021072); 2020 Anhui Province Vocational and Adult Education Society Key Project (No.: azcg22); 2022 Anhui Province Quality Engineering General Teaching and Research Project (No.: 2022jyxm1037).

References

1. Huang Fang, Li Minli, Xiong Liangsheng, et al. (2019) The reform of higher vocational nursing education led by nursing skills competition. Chinese Journal of Nursing Education 16: 511-514.
2. Ni Qiqi, He Guijuan, Li Hangting, et al. (2021) Design and implementation of the Zhejiang University Nursing Competition. Chinese Journal of Nursing Education 18: 20-24.
3. Ho CF, Chao SY (2020) Evaluating the health and social care competences of nursing students using skills competition. Hu Li Za Zhi 67: 61-71.
4. Liang Y, Liu WH, Li X, et al. (2024) 2022 Shandong Province university medical technical skills competition nursing track: An effective project to improve core competencies of nursing students. Heliyon 10: e26208.
5. Gaffney M K (2024) The Skills Olympics: Using Competition to Illustrate Patient Care Challenges and Promote the Transfer of Nursing Knowledge. Nurs Educ Perspect 46: 200-201.
6. Zhou Yinghua, Wu Yixin, Xu Shuqin, et al. (2023) Introduction and experience sharing of the health and social care project of the World Skills Competition. Military Nursing 2023: 100-103.
7. National Vocational School Skills Competition Organizing Committee (2023) National Vocational School Skills Competition Rules. Beijing: Department of Vocational and Adult Education, Ministry of Education.
8. Peng Jun (2015) Evaluation of internal control of commercial banks based on attribute hierarchy model. Commercial Economic Research 17: 78-79.
9. Zhang Gaochan, Zhang Chaoyuan (2016) Earthquake disaster assessment in Yunnan based on attribute hierarchy model (AHM). Journal of Luoyang Normal University 35: 3-8.
10. Wang Yaping (2017) Application of TOPSIS method bassed on AHM in primary medical status evaluation. China Management Informationization 20: 198.
11. Meng Bingbing, Song Jiawei, Chen Huijuan, et al. (2022) Evaluation of the rationality of preventive use of antibiotics in neurosurgery based on attribute hierarchical model. Chinese Journal of Hospital Pharmacy 42: 435-438.
12. Wang Xueqian, Zhang Pengcheng, Song Jiawei (2022) Evaluation of the rationality of rivaroxaban application based on attribute hierarchy model. Clinical Medication Journal 20: 72-75.
13. Wu Yan, Song Jiawei, Zhang Zhetao, et al. (2023) Exploration of the teaching assessment and evaluation model of clinical pharmacy interns based on AHM weighted TOPSIS method. Chinese Journal of Clinical Pharmacy 32: 41-46.
14. Wang Jiaojiao, Tan Qin, Qi Jingwen, et al. (2021) Application of direct observation and evaluation method of operation skills in intravenous infusion teaching for intern nurses. Qilu Nursing Journal 27: 163-165.
15. Meng Bingbing, Song Jiawei, Chen Huijuan, et al. (2022) Evaluation of the rationality of preventive use of antibiotics in neurosurgery based on attribute hierarchical model. Chinese Journal of Hospital Pharmacy 42: 435.
16. Wang Long, Zhu Lingna, Song Jiawei, et al. (2020) Establishment and application of evaluation criteria for the rational use of gemcitabine. Chinese Journal of New Drugs and Clinical Practice 39: 568.
17. Song Jiawei, Wang Yuanyuan, Chen Huijuan, et al. Application of attribute hierarchy model in the evaluation of palonosetron drug utilization. Chinese Journal of Clinical Pharmacy 31: 523-527.
18. Cao Xi, Zeng Jing, He Chunyu, et al. (2022) Introduction and enlightenment of the Southwest and Northwest Division Competition of the National College Students' Medical Technology Skills Competition for Nursing Majors. Military Nursing 39: 98-100.
19. Cheng Qiansheng (1998) Attribute Hierarchy Model AHM: A new unstructured decision making method. Journal of Peking University (Natural Science Edition) 1998: 12-16.
20. Liu Jing, Jiang Guanchao (2021) Summary and enlightenment of the national clinical skills competition for college students in higher medical colleges. Chinese Journal of Nursing Education 18: 12-16.
21. Luo Jian, Yang Liu, Yang Dan, et al. (2017) Application of direct observation of skills in the standardized training skills assessment of new nurses. Chinese Journal of Nursing Management 17: 75-78.