Psychometric testing Strategic Clinical Quality Indicator of Postoperative Pain (SCQIPP); Indonesian version

Arifin Triyanto 1 , Annisa Permatasari 2 , Miftahul Jannah 3 , Ellyana Eka Putri 4 ,           Heny Suseani Pangastuti 5 , Anita Kustanti 6 , Khudazi Aulawi 7
Authors affiliations:
  1. Arifin Triyanto, Department of Medical Surgical Nursing, Faculty of Medicine, Public Health & Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia.Email: arifintriyanto@mail.ugm.ac.id
  2. Annisa Permatasari. School of Nursing, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Email; annisapermata2018@mail.ugm.ac.id
  3. Miftahul Jannah. School of Nursing, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Email; miftahuljannah2018@mail.ugm.ac.id
  4. Ellyana Eka Putri. School of Nursing, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Email; ellyana.eka.putri@mail.ugm.ac.id
  5. Heny Suseani Pangastuti. Department of Medical Surgical Nursing, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Email: heny_pangastuti@ugm.ac.id
  6. Anita Kustanti. Department of Medical Surgical Nursing, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Email: anitakustanti@ugm.ac.id
  7. Khudazi Aulawi. Department of Medical Surgical Nursing, Faculty of Medicine, Public Health and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia; Email: aulawi@ugm.ac.id
Correspondence: Arifin Triyanto; Email: arifintriyanto@mail.ugm.ac.id; Mobile: 62 89667765595

 

ABSTRACT

 

Background & objective: Effective pain management can reduce stress response, facilitate early mobilization, and improve quality of life in the postoperative period. Currently, little instruments used to measure the quality of pain management as a whole. Most pain management evaluation in Indonesia only measure outcomes such as pain intensity. This study was to evaluate the validity and reliability of the Indonesian version of Strategic Clinical Quality Indicator of Postoperative Pain (SCQIPP).

Methodology: This is psychometric testing using cross-cultural adaptation and back translation methods. Conducted in two referral hospital in Yogyakarta, Indonesia, this study involving postoperative patients with inclusion criteria: the patient had undergone treatment for at least 48 hours postoperatively; aged ≥ 18 years old, able to read and write, able to communicate, willing to be involved in this study. The exclusion criteria were: the patient was in severe pain; patients with decreased consciousness. Face validity analysis used percent agreement, content validity used Content Validity Index (CVI), construct validity used r count and r table calculations, and reliability used Cronbach alpha.

Results: Instrument testing was conducted on 4 experts and testing on 164 elective post-operative patients. The test results obtained face validity values between 75% -100%, CVI values of all items = 1, construct validity values between 0.27 - 0.66, and Cronbach alpha values of 0.77.

Conclusions: The Indonesian version of the SCQIPP instrument consisting of 14 valid and reliable items for measuring the quality of postoperative pain management. The test results obtained face validity values between 75% -100%,

Keywords: Indonesia; pain; post-operative; psychometric testing; reliability; validity

Citation: Triyanto A, Permatasari A, Jannah M, Putri EE, Pangastuti HS, Kustanti A, Aulawi K. Psychometric testing Strategic Clinical Quality Indicator of Postoperative Pain (SCQIPP); Indonesian version. Anaesth. pain intensive care 2025;29(9):1171-78. DOI: 10.35975/apic.v29i9.3048

Received: May 09, 2025; Revised: October 26, 2025; Accepted: January 01, 2025

 

1. INTRODUCTION

 

All surgical procedures are associated with acute pain which is defined as an unpleasant sensation, sensory and emotional experience.1 Acute pain resulting from tissue damage during surgery can take a long time, and it became a chronic pain. Prevention and reduction of postoperative pain is the main responsibility of health workers. Many patients have pain experience beyond their expectation. Adequate perioperative pain management is an integral part of nursing care. Every biological, psychological and social dimension of pain must be understood and given optimal treatment.2
Improving pain management is still challenging for the nurse. The nurses during postoperative period need to be improve in with standard protocols and standard services in pain management.3 Inadequate pain management is associated with early mobility, prolonged length of stay, high cost of care and decreased patient satisfaction. Effective pain management can reduce stress responses, facilitate early mobilization, and improve quality of life in the postoperative period.2
Pain management is also a standard assessment required in the National Hospital Accreditation Standards (SNARS) and the Joint Commission International (JCI). This standards requires every hospitals implement the standards in SNARS to achieve the status as a good hospital, so that pain management as one of the component must be good implemented.  Assessment of the quality of pain management can be done by assessing aspects, both structure, process and results. Structural aspects include standard operating procedures (SOP), equipment availability, staff quality, pain team organization; process aspects can be assessed from communication, actions, patient trust, environment; and outcome aspects assessed from the level of pain felt by the patient.4
Pain management in hospital settings has several obstacles including teamwork in a collaborative approach, adaptation to the local culture of the patient, and efforts to improve pain management in the future.5 Previous study shows that pain is the main reason for being hospitalized. Based on the data, only 14.2% of assessments used numeric pain scores recorded in medical records, documentation of pain intensity by health professionals was 41.5%, and 7.7% of patients had to wait 30 min to get treatment. This data indicated that quality of pain managements were still low.6 Understanding the various conditions and mechanisms of pain in postoperative patients, it is necessary to improve the identification of effective treatment strategies to improve patient outcomes. In the future, pain treatment must focus on the development of therapeutic agents, improving pain management and rapid recovery.7
Currently, in Indonesia, there are little instruments used to measure the quality of pain management as a whole. Most pain management measurements only measure outcomes such as pain intensity. Measurement of the quality of pain management can also be done on the process aspect during treatment. Therefore, a comprehensive pain management quality measurement instrument in the Indonesian language version is needed.

 

2. METHODOLOGY

 

The study used a descriptive cross-sectional approach in the adult surgical ward of a hospital in Yogyakarta. Sampling was done using consecutive sampling technique with inclusion criteria: the patient had undergone treatment for at least 48 hours postoperatively; aged ≥ 18 years old, able to read and write well, able to communicate well, willing to be involved in research, while the exclusion criteria were: the patient was in severe pain (patients with VAS pain ≥ 7); patients with decreased consciousness. The minimum number of samples required follows the 1:5 method, the minimum number of samples is 5 times the number of questionnaire items. The SCQIPP instrument has 19 question items, the minimum number of samples is 95 respondents. In this study, we conducted psychometric testing on 14 items only.  This study approved by Ethical Committee of Faculty of Medicine Public Health and Nursing, Universities Gadjah Mada No: KE/FK/0880/EC/2021.

The translation of the SCQIPP questionnaire has been approved by E. Idvall, as the developer of the instrument. The SCQIPP instrument consists of 14 questions about pain treatment with a Likert scale from 1 to 5. The number 1 indicates strongly disagree and the number 5 indicates strongly agree. While 5 additional questions about postoperative pain experiences with a range of answer choices from 0 to 10. The SCQIPP instrument consists of 6 subscales consisting 4 main and 2 additional ones. In this study, psychometric testing was carried out on 4 main subscales (communication, action, trust, and environment) consisting of 14 questions. These fourteen items are main item in the SCQIPP instrument, while the other 5 items are additional only and optional to using it. The distribution of question items for each subscale can be seen in Table 1.

Table 1: Distribution of SCQIPP item
Subscale Question number Total
Communication 1, 11, 14 3
Action 2, 3, 6, 7 4
Trust 5, 8, 12, 13 4
Environment 4, 9, 10 3
Satisfaction 18, 19 2
Pain intensity 15, 16, 17 3
Total 19
 

The cross-cultural adaptation process goes through several stages. First, the original instrument is translated independently by 2 sworn translators. Second, the translation results are reviewed by 3 experts (1 from academics and 2 from clinicians) related to the suitability of the translated content. Third, the review results from the three experts are re-translated by 2 different translators. After the Indonesian version results are obtained, the questionnaire is reviewed by 4 experts consisting of 2 experts from academics and 2 experts from clinicians to assess face validity and content validity. Experts provide an assessment of the face validity of each question item with appropriate and inappropriate categories by considering 5 aspects, namely grammatical accuracy, clarity of question items, correct spelling of words, correct sentence structure, and appropriateness of writing size. Meanwhile, for content validity, experts assess the level of relevance, clarity, simplicity and ambiguity by providing a Likert scale assessment of 1 = not relevant, 2 = less relevant and requires significant changes, 3 = relevant but requires modification and 4 = very relevant. The questionnaire from the expert review is tested on patients according to the criteria. Before filling, the patient gets information related to the research being conducted and signs the consent. The time needed to fill is approximately 10 min.

Face validity was conducted by researchers by asking for expert assessments on each SCQIPP item. The experts were asked to evaluate the suitability of the questionnaire items. The face validity assessment used a dichotomous scale, namely appropriate and inappropriate. An item is reliable if the percent agreement value ≥ 75%.

Content validity is assessed using the Content Validity Index (CVI). The CVI value is calculated from the sum of the values 3 and 4 of each item divided by the number of experts. The CVI result is generally acceptable if CVI ≥ 0.8, with 1.0 indicating perfect agreement. The internal consistency of each question item is measured using Cronbach's alpha. The Cronbach's alpha value is good if the coefficient value is above 0.70. The reliability of the questionnaire is measured through r count. R count refers to the calculated Pearson correlation coefficient. It represent the strength and direction of the linear relationship between variables typically item / question in questionnaire. Higher r count indicates a stronger correlation. Meanwhile r table refers to critical value of the Pearson correlation coefficient from statistical table. Item to be reliable if r count > r table (r table with 164 respondents is 1.53).

 

3. RESULTS

 

Demographic characteristics
This study involved 164 respondents who underwent various types of surgical procedures. The majority of respondents were male (54.3%), with an average age of 46.9 ± 15.1 years and most of them had senior high school education background. Most respondents underwent gastroenterology surgery (29.9%) with an average duration of 78.6 ± 53.9 min. complete respondent characteristic data can be seen in Table 2.

Table 2: Data of respondents in post-operative patients (n = 164)
Variables N (%)
Gender
Male 89 (54.3)
Female 75 (45.7)
Education
Elementary School 37 (22.5)
Junior High School 14 (8.5)
Senior High School 80 (48.8)
College 33 (20.2)
Work/job
Civil Servants/Retirees 18 (10.8)
Private Sector 58 (35.3)
Farmer 21 (34.3
Student 7 (4.3)
Housewife 29 (17.5)
Others 31 (18.8)
Operation type
Gastroenterology 49 (29.9)
Orthopedics 42 (65.6)
Oncology 16 (9.8)
Neurology 6 (3.6)
Thoracic 13 (7.9)
ENT 2 (1.2
Urology 16 (9.7)
General 20 (12.3)
Age (years) mean ± SD 46.9 ±15.1
Surgical Duration (minute) 126.5 ±104.5
 

3.1. Face Validity
We conducted face validity with expert judgement. This process ensures that the items in the instrument are relevant and comprehensible to the target respondent, which is essential for accurate data collection and interpretation. The face validity assessment by 4 experts showed that all items had a suitability of more than or equal to 75%. This shows that the face validity of each item in this questionnaire is valid. The results of the assessment of each item by the experts can be seen in Table 3.

Table 3: Results of the 4 expert agreement test
Items Expert 1 Expert 2 Expert 3 Expert 4 Score % agreement
1 1 1 1 1 4 100
2 1 1 1 1 4 100
3 0 1 1 1 3 75
4 0 1 1 1 3 75
5 1 1 1 1 4 100
6 0 1 1 1 3 75
7 0 1 1 1 3 75
8 1 1 0 1 3 75
9 1 1 1 1 4 100
10 1 1 1 1 4 100
11 0 1 1 1 3 75
12 0 1 1 1 3 75
13 0 1 1 1 3 75
14 1 1 1 1 4 100
 

3.2. Content Validity
Content validity from the four experts has assessed relevance, clarity, simplicity and ambiguity. The results of the expert assessment obtained all questionnaire items have a CVI value of 1 in all aspects. This shows that all SCQIPP questionnaire items have good content validity. The results of content validity can be seen in Table 4.

Table 4: Results of CVI analysis of 14 SCQIPP items
Expert Relevance Clarity Simplicity Ambiguity
Expert 1 1 1 1 1
Expert 2 1 1 1 1
Expert 3 1 1 1 1
Expert 4 1 1 1 1
 

3.3. Validity of construct
The construct validity is assessed by comparing the calculated r with the table r with an error rate of 5%. The calculated r results are known by comparing the results of each item compared to the overall total. The results show that the questionnaire has a calculated r count greater than the r-table so it can be concluded that each item of the questionnaire is valid with a validity value between 0.373 and 0.587. The calculated r for each item can be seen in Table 5.

Table 5: Results of r-count and r-table of the SCQIPP instrument
No. Item r-count r-table Result
1 Before my operation I was told about the type of pain treatment I would be offered after surgery

Sebelum operasi, saya diberitahu mengenai jenis
perawatan nyeri yang akan ditawarkan kepada saya setelah operasi.		 0.424 0.153 Valid
2 After my operation I talked with nurse about how I wanted my pain to be treated

Setelah operasi, saya berbicara dengan perawat
mengenai bagaimana saya ingin nyeri saya dirawat.		 0.373 0.153 Valid
3 I received support or help in finding a comfortable position in bed to help avoid pain

Saya mendapat dukungan atau bantuan dalam
mencari posisi yang nyaman di tempat tidur untuk menghindari nyeri.		 0.517 0.153 Valid
4 I was given the opportunity for peace and quite so I could sleep at night

Saya diberikan kesempatan untuk merasakan kedamaian dan ketenangan sehingga saya dapat tidur di malam hari.		 0.568 0.153 Valid
5 Even if I did not always ask for it, I was given pain medication

Meskipun saya tidak selalu minta, saya diberikan
pengobatan untuk rasa nyeri		 0.322 0.153 Valid
6 The staff asked me about the pain I had when I breathed deeply, sat up, or move around

Staf bertanya apakah saya merasakan sakit saat mengambil napas dalam-dalam, duduk, atau berpindah tempat.		 0.461 0.153 Valid
7 To determine my level of pain, a member of the staff asked me to pick a number between 1 and 10 (or make a mark on a straight line) at least once every morning, afternoon, and evening

Untuk menentukan tingkat rasa nyeri, anggota staf meminta saya untuk memilih angka dari 1 hingga 10 (atau membuat tanda pada garis lurus) paling tidak satu kali setiap pagi, siang, dan malam.		 0.496 0.153 Valid
8 The nurse helped me with pain treatment until I was satisfied with the effects of pain relief

Para perawat membantu saya dengan perawatan nyeri hingga saya puas dengan efek pereda rasa nyeri.		 0.529 0.153 Valid
9 I have a pleasant room

Saya memiliki kamar yang nyaman.		 0.541 0.153 Valid
10 There have been enough nurses on duty for someone to respond quickly to my request for pain relief

Terdapat cukup perawat yang bertugas untuk dapat merespon dengan cepat permintaan saya akan pereda rasa nyeri.		 0.507 0.153 Valid
11 When nurses come on duty, they know ‘everything’ about how much pain I have had and the pain treatment I have received

Saat perawat bertugas, mereka mengetahui semua hal tentang seberapa banyak rasa nyeri yang saya rasakan dan perawatan nyeri yang saya terima.		 0.478 0.153 Valid
12 The nurses are knowledgeable about how to relieve my pain

Para perawat memiliki pengetahuan yang baik tentang bagaimana meredakan rasa nyeri saya.		 0.533 0.153 Valid
13 The nurses believe me when I tell them about my pain

Para perawat mempercayai saya saat saya memberitahu mereka tentang rasa nyeri saya.		 0.587 0.153 Valid
14 The nurses and doctors have cooperated in treating my pain

Para perawat dan dokter telah bekerjsama dalam merawat rasa nyeri saya.		 0.578 0.153 Valid
Note: italic items are question in bahasa (indonesian version)
 

3.4. Reliability
The reliability value is calculated using Cronbach alpha. The calculation results of this questionnaire obtained a Cronbach alpha of 0.77. This shows that the questionnaire is reliable.

 

4. DISCUSSION

 

The SCQIPP questionnaire is a multifaced tool designed to evaluate and improve the quality of care provided to patients experiencing postoperative pain. This tool can evaluate the quality of care from both patients and healthcare provider perspectives. Its first develop by Idvall in Sweden 2001. This tool has been validated in multiple languages, including Turkish, Greek, and Chinese, demonstrating its reliability and applicability across different cultural contexts.8,9,10 The tool typically includes 14 items, although some studies have modified the number of items based on specific validation results.9 Quality indicators for postoperative pain management include measures of structure, process, and outcomes. These indicators help in benchmarking and improving care practices.10 Studies using the SCQIPP questionnaire have consistently identified areas needing improvement in postoperative pain management, such as communication, action, trust, and environment.11 Regular assessment of pain intensity using numerical rating scales (NRS) is a common practice. However, it is important to note that pain intensity alone may not provide a comprehensive picture of pain management effectiveness.12
In this psychometric process, researchers carry out cross-cultural adaptation. Cross-cultural adaptation of instruments is a complex and multi-faceted process that ensures the validity and reliability of research tools when used in different cultural contexts.13 Addressing cultural nuances and ensuring that the instrument is relevant and understandable in the target culture. This may involve modifying items or response options to achieve cultural equivalence.14
The explanation of the step is discussed below:
  1. First, translating the instrument from the source language to the target language by bilingual experts.
  2. Translating the instrument back to the source language to check for consistency and accuracy. Combining the forward translations into a single version, resolving discrepancies through discussion or consensus among translators.
  3. An expert committee reviews the translations to ensure cltural relevance and conceptual equivalence. This step often involves professionals from various fields such as linguistics, subject matter experts, and statisticians.15 Experts should be selected based on their competence in the subject matter and their independence to avoid conflicts of interest.16 Experts should be given the opportunity to provide qualitative feedback on each item, suggesting revisions to improve clarity and relevance. Validation is often an iterative process, involving multiple rounds of review and revision to refine the instrument.17
  4. Face validity in this psychometric testing goes through a review process from 4 experts. The results of the 4 experts Fare processed into percent agreement. Percent agreement is the reliability statistic obtained by dividing the number of observations agreed upon to the total number of observations. It is easy to compute and interpret, and 70% agreement is considered as the minimum acceptable level. However, percent agreement is criticized due to chance agreement i.e., the proportion of agreements when the observers' ratings are unrelated. Cohen's Kappa is perceived as a better index because it accounts for the chance agreement.18 Agreement between measurements refers to the degree of similarity between two measurements. Various statistical methods can be used to measure the level of agreement. The agreement results in this validity test show a value of 75% - 100%. These results indicate that all items in the instrument are valid. Face validity results are said to be valid if they have an expert agreement value of more than or equal to 75%.19
Testing of the questionnaire that has gone through an expert review process was carried out on 60 postoperative patients. The total number of respondents who took part in this psychometric testing was sufficient based on the provisions of the number of respondents in psychometric testing of 3 times the number of question items tested. Respondents in this psychometric test were patients with postoperative conditions. This is in accordance with the concept developed by the inventor of the instrument using postoperative respondents. In other studies that also conducted psychometric tests, researchers also used postoperative patients in determining indicators of the quality of pain management.9,10
The majority of respondents were female with a majority of senior high school education and an average age of 46.9 years. The majority of respondents underwent surgery in the gastroenterology area with an average duration of surgery of 78.6 min. Respondent characteristics can be factors related to pain in postoperative patients. Gender and type of surgery contribute to differences in postoperative pain intensity.20 However, these differences did not affect the results of the psychometric test, basically the psychometric test was conducted to measure the accuracy in measuring the quality of postoperative pain management regardless of variations in respondents.21
The results of testing respondents obtained the r-count of the Indonesian version of the SCQIPP instrument ranging from 0.27 - 0.66 and this result is greater than the r-table (0.25) which shows that all items of the instrument are valid. Coefficient correlation can be categorized into several groups, a value of 0.9 - 1.0 very high correlation, 0.7 - 0.9 high correlation, 0.5 - 0.7 moderate correlation and 0.3 - 0.5 low correlation.22 The results show that the question items are at low and moderate correlation. This result is lower than the validity of the SCQIPP reliability conducted in Turkey and Greece. This difference can occur due to differences in sample characteristics and the number of samples involved in this validity test.20
The results of this psychometric test indicate that the Indonesian version of SCQIPP is valid and reliable for use in measuring the quality of pain management in postoperative patients. SCQIPP has previously been widely adapted in various countries through cross-cultural adaptation. In Turkey, SCQIPP has been tested on 113 elective surgery patients. Meanwhile, in Greece, SCQIPP has been tested on 210 elective surgery patients. Psychometric testing in both countries showed valid results.8,23 This confirms that the SCQIPP instrument is suitable for use in various countries including Indonesia. The indicators in SCQIPP are very comprehensive because they not only measure outcomes, but also measure during the postoperative pain management process. This is in line with the standard indicators of pain management quality in various countries that indicators of pain management quality include structure, process and outcome.24 This comprehensive measurement supports the achievement of good pain management. Measurement of the quality of pain management must be precise, the quality of the indicators must be applicable, planned and can measure comprehensively.25
 

5. CONCLUSION

 

SCQIPP adapted to Indonesian version trough cross-cultural adaptation with 14 item showed good validity and reliability in all item of questionnaire. This questionnaire acceptable to assess pain management quality in postoperative patients. This study will make change in measuring pain management not only outcome based but also all aspects including structure and process.

6. Acknowledgements
We thanks to all participants for their participation during this study.

7. Author contribution
AT: Conception, data collection, data management, data analysis, draft writing, manuscript writing, submission
  1. AP, EKP: data collection, data management, manuscript writing
HSP, AK, KA: Conception, manuscript review

8. Conflicts of interest
All authors declare no conflict of interest.

 

9. REFERENCES

 
  1. Malek J, Sevcik P. Postoperative Pain Management. third edit. Praha: Mlada Fronta; 2017. 1–112 p. [FullText]
  2. Small C, Laycock H. Acute postoperative pain management. Br J Surg. 2020;107(2):e70–80. [PubMed] DOI: 1002/bjs.11477
  3. Rawal N. Current issues in postoperative pain management. Eur J Anaesthesiol. 2016;33(3):160–71. [PubMed] DOI: 1097/EJA.0000000000000366
  4. Meissner W, Huygen F, Neugebauer EAM, Benhamou D, Betteridge N, Coluzzi F, et al. Management of acute pain in the postoperative setting : the importance of quality indicators. Curr Med Res Opin. 2018;34(1):187–96. [PubMed]DOI: 1080/03007995.2017.1391081
  5. Akbar N, Teo SP, Artini Hj-Abdul-Rahman HN, Hj-Husaini HA, Venkatasalu MR. Barriers and Solutions for Improving Pain Management Practices in Acute Hospital Settings: Perspectives of Healthcare Practitioners for a Pain-Free Hospital Initiative. Ann Geriatr Med Res. 2019;23(4):190–6. [PubMed] DOI: 4235/agmr.19.0037
  6. Tawil S, Iskandar K, Salameh P. Pain management in hospitals: Patients’ satisfaction and related barriers. Pharm Pract (Granada). 2018;16(3):1–9. [PubMed] DOI:18549/PharmPract.2018.03.1268
  7. Pogatzki-Zahn EM, Segelcke D, Schug SA. Postoperative pain—from mechanisms to treatment. Pain Reports. 2017;2(2). [PubMed] DOI: 18549/PharmPract.2018.03.1268
  8. Vatansever NA, Akansel N. Validation study of the strategic and clinical quality indicators in postoperative pain management questionnaire in Turkish surgery patients. Pain Manag Nurs [Internet]. 2014;15(4):871–80. Available from: http://dx.doi.org/10.1016/j.pmn.2014.01.003 [PubMed] DOI: 1016/j.pmn.2014.01.003
  9. Hreńczuk MK, Zadrona D. Pain and functioning of patients after endovascular treatment of thoracoabdominal aortic aneurysms during the first five days after the procedure. Pielegniarstwo XXI Wieku. 2022;21(4):224–9. [FullText]
  10. Chen J, Tong Y, Cheng Y, Xue Z, Liu M. Establishment and Empirical Evaluation of a Quality Indicator System for Postoperative Pain Management. Pain Med (United States). 2020;21(12):3270–82. [PubMed] DOI:  1093/pm/pnaa221
  11. Piotrkowska R, Jarzynkowski P, Mędrzycka-Dąbrowska W, Terech-Skóra S, Kobylarz A, Książek J. Assessment of the Quality of Nursing Care of Postoperative Pain in Patients Undergoing Vascular Procedures. J Perianesthesia Nurs. 2020;35(5):484–90. [PubMed] DOI: 1016/j.jopan.2020.03.010
  12. Komann M, Baumbach P, Stamer UM, Weinmann C, Arnold C, Pogatzki-Zahn E, et al. Desire to Receive More Pain Treatment – A Relevant Patient-Reported Outcome Measure to Assess Quality of Post-Operative Pain Management? Results From 79,996 Patients Enrolled in the Pain Registry QUIPS from 2016 to 2019. J Pain. 2021;22(6):730–8. [PubMed] DOI: 1016/j.jpain.2021.01.002
  13. Cruchinho P, López-Franco MD, Capelas ML, Almeida S, Bennett PM, da Silva MM, et al. Translation, Cross-Cultural Adaptation, and Validation of Measurement Instruments: A Practical Guideline for Novice Researchers. J Multidiscip Healthc. 2024;17(May 2024):2701–28. [PubMed] DOI: 2147/JMDH.S419714
  14. Lei F, Lee E. Cross-cultural modification strategies for instruments measuring health beliefs about cancer screening: Systematic review. JMIR Cancer. 2021;7(4):1–12. [PubMed] DOI: 2196/28393
  15. Castillo-Sierra DM, González-Consuegra R V. Cross cultural adaptation of “Florida patient acceptance survey” instrument that measures acceptance of patients of cardiac devices. Salud Uninorte. 2018;34(2):383–92. [FullText]
  16. Hoang DT, Nguyen NT, Hwang D. A group recommender system for selecting experts to review a specific problem [Internet]. Vol. 11055 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer International Publishing; 2018. 270–280 p. Available from: http://dx.doi.org/10.1007/978-3-319-98443-8_25 [FullText]
  17. Vveinhardt J, Gulbovaitė E. Expert Evaluation of Diagnostic Instrument for Personal and Organizational Value Congruence. J Bus Ethics. 2016;136(3):481–501. [FullText]
  18. Bryer J. Relationship Between Intraclass Correlation (ICC) and Percent Agreement. IRRsim 1.0.1. 2023. p. 1–18. Available from:[Full Text]
  19. Shweta, Bajpai RC, Chaturvedi HK. Evaluation of inter-rater agreement and inter-rater reliability for observational data: An overview of concepts and methods. Vol. 41, Journal of the Indian Academy of Applied Psychology. 2015. p. 20–7. [FullText]
  20. Mi X, Zou B, Rashidi P, Baharloo R, Fillingim RB, Wallace MR, et al. Effects of Patient and Surgery Characteristics on Persistent Postoperative Pain: A Mediation Analysis. Clin J Pain. 2021;37(11):803–11. [PubMed] DOI:  1097/AJP.0000000000000979
  21. Ahmed I, Ishtiaq S. Reliability and validity: Importance in Medical Research. Vol. 71, Journal of the Pakistan Medical Association. 2021. p. 2401–6. [PubMed] DOI:  47391/JPMA.06-861
  22. Calkins K. Correlation Coefficients. 2023 [cited 2024 Jan 5]. Available from: [PubMed]
    DOI: 1002/jmri.28337
  23. Sarakatsianou C, Baloyiannis I, Perivoliotis K, Kolonia K, Georgopoulou S, Tzovaras G. Validation and Scoring of the Greek Version of the Strategic and Clinical Quality Indicators in Postoperative Pain Management (SCQIPP) Questionnaire. J Perianesthesia Nurs [Internet]. 2022;37(6):918–24. Available from: https://doi.org/10.1016/j.jopan.2022.03.005 [PubMed] DOI: 1016/j.jopan.2022.03.005
  24. Glarcher M, Kundt FS, Meissner W, Osterbrink J. Quality Indicators (QI) of Acute Pain after Surgery in European Countries. Pain Manag Nurs [Internet]. 2021;22(6):730–9. Available from: https://doi.org/10.1016/j.pmn.2021.01.012 [PubMed] DOI: 1016/j.pmn.2021.01.012
  25. Köse Tamer L, Sucu Dağ G. The Assessment of Pain and the Quality of Postoperative Pain Management in Surgical Patients. SAGE Open. 2020;10(2). [FullText]