Breaking Ground in Skill Assessment: The Construction and Validation of a Problem-Solving Scale for University Students

: Students today is expected to possess 21st-century skills, one of which is Problem Solving Skills (PSS). The importance of mastering this skill needs to be accompanied by the availability of a measuring tool. This study addresses this need by developing and validating the Problem-Solving Skill Scale. The development of this scale is focused on guidance and counseling students. The research and development of this instrument follow the educational design research method according to Plomp and Nieveen. Four hundred students were involved to test the construct validity. Problem-solving skills were measured based on three aspects: (1) Problem-Solving Confidence, (2) Approach-Avoidance Style, (3) Personal Control. Data were analyzed using the Rasch model by the WinSteps software. The research results describe the instrument's quality, the person-item interaction , the item difficulty level (item measures), fit and misfit items, and the distribution of respondents' abilities and the distribution of item levels on the same scale, and the scale’s ability to measure what it intends to measure. The Rasch analysis results show that the PSS Scale is good, precise, and has item fit with the model. The PSS Scale is a valid and reliable measuring tool for accurately measuring students' problem-solving skills. This research discusses implications and recommendations for further research efforts to facilitate the improvement of problem-solving skills in students using this measurement tool.


INTRODUCTION
Education is a crucial aspect of human life, serving as a means of empowering individuals to face global challenges.Through education, individuals can develop potential, intelligence, skills, personality, and noble characters (Fajri et al., 2021).Currently, education is situated in the knowledge age, marked by an extraordinary acceleration of knowledge growth.In the 21st century, education has become increasingly vital to ensure students possess learning and innovation skills, the ability to use technology and information media, as well as the capacity to work and persevere using life skills (Sole & Anggraeni, 2018).The 21st-century learning demands a technology-based approach to balance the demands of the millennial era, aiming for learners to become accustomed to 21st-century life skills.The 21st century, associated with the era of the fourth industrial revolution, exerts a wide-ranging influence on education (Bernie Trilling, 2009).
¨Corresponding author: Ma'rifatin Indah Kholili.Department of Guidance and Counseling, Faculty of Education, Universitas Sebelas Maret.Jl. Ir. Sutami No.36,Jebres,Kec. Jebres,Kota Surakarta,Jawa Tengah 57126,Indonesia. Email: marifatin.ink@staff.uns.ac.idAccording to Trilling & Fadel (2009), key skills essential in the context of the 21st century include learning and innovation skills, encompassing critical thinking and problem-solving skills, communication and collaboration skills, and creativity and innovation skills.Problem-solving skill has become a focal point and a key objective of education, particularly in the 21st century (Gunawan et al., 2020).An individual's skills for living in the 21st century can be categorized into three aspects: skills for life and career development, skills for learning and innovation, and skills for using information and technology media (Fitarahmawati & Suhartini, 2021).One of the skills required in learning and innovation is critical thinking and problem-solving skills (Suardi et al., 2019).Problem-solving skills are crucial for individuals facing challenges and demands in an increasingly complex life, contributing positively to their surroundings (Bariyyah, 2021).These skills help individuals builld new knowledge and support scientific thinking activities in the learning process (Wahyuni et al., 2015).Problem-solving is a skill related to mathematical knowledge, general intelligence, creativity, and verbal ability (Harefa & Purba, 2020;Lestari & Munahefi, 2023).It involves conceptual understanding and procedural knowledge.
Problem-solving skills are grounded in the process of identifying problems, exploring alternative solutions, and applying the best solution in relatively new situations (Araiza-Alba et al., 2021;Bariyyah, 2021).Dörner & Funke (2017) define problem-solving skills as the ability to define problems, determine their causes, set priorities, select various solution options, and implement those solutions.Based on the opinions of several experts, problem-solving skills can be defined as the ability to identify problems, search and select alternative solutions, and make decisions in resolving issues.Furthermore Schoppek et al., (2018) state that problem-solving skills involve the ability to address all problems and make difficult decisions.Meanwhile, according to Palennari et al., (Palennari et al., 2022), problem-solving skills are an effort to find solutions, especially in practical issues that require economically viable, timely, and implementable considerations, making them crucial in various aspects of life (Rahayu & Adistana, 2018).
Problem-solving enhances critical thinking skills and developmental learning strategies (Harefa & Purba, 2020).The improvement of problem-solving skills is essential for individuals to contribute effectively to solving real-life problems (Mudhofir et al., 2022).Possessing these skills helps student make all activities more meaningful, higher in quality, and closer to real-world experiences (Wulandari et al., 2019).Within the scope of education, problem-solving skills do not develop spontaneously; they require stimuli through the implementation of learning models such as project-based learning or the development of other skills (Mandina & Ochonogor, 2018).Problem-solving is a cognitive process of finding ways to achieve goals that depend on the ability to organize and process information.Therefore, this skill is crucial in learning to enhance students' learning outcomes (Irawan et al., 2022).Competence and credibility of individuals are not solely determined by their educational achievements but also by their problem-solving skills (Jamari, 2016).Problem-solving skills can be cultivated when individuals become university students, as they are considered intellectual individuals expected to become leaders and bring about change in the future.One of the soft skills that propels students' intellectual abilities is problem-solving skills.Therefore, in this research, the intention is to develop a problem-solving scale for students to assess the extent of their problem-solving skills.
According to Mourtos et al., (2004), problem-solving is defined as the process used to obtain the best answers to the unknown or decisions subject to various constraints.Heppner (Heppner & Petersen, 1982) and Kourmousi et al.(Kourmousi et al., 2016) suggest that problem-solving skills encompass cognitive, affective, and behavioral processes, as well as a set of specific skills individuals use to find solutions to everyday life challenges.Problem-solving skills consist of three aspects: (1) Problem-Solving Confidence (PSC), which assesses self-confidence and self-assurance in effectively solving problems (e.g., "I can usually think of creative and effective alternatives to solve problems.").Higher scores on PSC are associated with a higher level of confidence in problem-solving.(2) Approach-Avoidance Style, which evaluates whether individuals tend to confront or avoid problems (e.g., "When a solution to a problem doesn't work, I don't examine why it didn't work.").Higher scores indicate a tendency to avoid rather than approach problems.(3) Personal Control (PC), which assesses self-control elements in emotions and behavior (e.g., "I make quick judgments and then regret them.").Higher scores on PC reflect a more negative perception of personal control over one's issues.
According to Woods et al., (1997) and Mourtos et al., (2004) students with strong problemsolving abilities exhibit the following attributes: (1) Willingness to invest time in reading, gathering information, and defining problems, (2) Use of processes and various tactics to address problems, (3) Ability to monitor their problem-solving processes and reflect on their effectiveness, (4) Emphasis on accuracy over speed, (5) Writing down ideas and creating charts/figures while solving problems, (6) Organized and systematic approach, (7) Flexibility, maintaining open options, and viewing situations from different perspectives, (8) Utilization of relevant subject knowledge and objectively and critically assessing the quality, accuracy, and relevance of knowledge/data, (9) Willingness to take risks and handle ambiguity, welcoming change, and managing stress, (10) Adoption of a holistic approach that emphasizes fundamentals rather than attempting to combine various memorized sample solutions.
Asiye & Bilge, ( 2016) assert that problem-solving is a key skill that develops students' ability to think about situations, issues, and problems in new and different ways.It also enables them to address these challenges by employing creative, systematic, and analytical strategies.Problem-solving skills are measured by considering the following subscales (Asiye & Bilge, 2016) Woods (1997), Mourtos et al. (2004), Kourmossi (2016), and Asiye (2016), it is concluded that problem-solving skills involve the process used to obtain the best answers to the unknown, allowing the resolution of life's problems or challenges.This skill is applied to think about situations, problems, and their solutions in new and different ways, and to face them using creative, systematic, and analytical strategies.
This study addresses this need by developing and validating the Problem-Solving Skill Scale.The development and validation of the Problem-Solving Skill Scale is a crucial undertaking for several reasons, highlighting its importance in both academic research and practical applications.Problemsolving skills are integral to navigating the complexities of the modern world, enabling individuals to effectively address and overcome challenges in various domains, including personal life, education, and the workplace.These skills encompass a broad range of cognitive processes, such as identifying problems, generating solutions, making decisions, and evaluating outcomes.
In the educational sector, the ability to solve problems is increasingly recognized as a core competency that students must acquire.Educators and policymakers are keen on developing curricula that not only impart knowledge but also foster critical thinking and problem-solving abilities.By accurately assessing these skills, educational institutions can tailor their teaching strategies to better prepare students for the demands of the 21st century.
In the professional realm, problem-solving skills are a key determinant of job performance and career advancement.Employers across industries seek individuals who can adapt to changing environments, innovate, and resolve issues efficiently.A validated Problem-Solving Skill Scale provides organizations with a reliable tool to assess and develop the problem-solving capabilities of their workforce, leading to improved productivity and innovation.Furthermore, in psychological research and practice, understanding an individual's problem-solving abilities can offer insights into their cognitive functioning and psychological well-being.It can aid in the diagnosis and intervention of cognitive impairments and emotional disorders, where problem-solving skills may be compromised.
The development of a scientifically validated scale to measure problem-solving skills addresses the need for a standardized assessment tool.Such a scale ensures consistency and accuracy in measuring these complex cognitive abilities, facilitating research, educational assessment, and human resource management.Moreover, it contributes to the body of knowledge in cognitive psychology, education, and organizational behavior, supporting further research and practical applications aimed at enhancing problem-solving skills across different populations and contexts.

METHOD
The research and development of this instrument employed the educational design research method.Plomp & Nieveen (Plomp & Nieveen, 2013) define educational design research as "a research design appropriate to develop research-based solutions to complex problems in educational practice or to develop or validate theories about learning processes, learning environments and the like."This study aimed to develop the Problem Solving Skills Scale, conducted in three main stages following Plomp and Nieveen: Preliminary research, Prototyping stage, and Assessment phase.

Participants
Data were collected from May to July 2023.The main participants were guidance and counseling students at Universitas Sebelas Maret, Surakarta, Central Java, with additional participants from various regions in Java.The total participants were 400 students, meeting the participation criteria of guidance and counseling students.Data collection utilized a Google Form.

Procedures
This research begins with the preliminary research stage.During this stage, the researcher gathered the 21st-century skill needs required by students.Findings from classroom learning implementation identified that students did not possess optimal problem-solving skills.The researcher proceeded by designing the problem-solving skills instrument, specifically addressing conceptual and operational definitions.
The second stage carried out is prototyping stage.In this stage, the researcher developed the instrument based on theoretical concepts.The problem-solving skills scale was developed for guidance and counseling students (prospective teacher).Referring to DeVellis's model (DeVellis, 2017), the researcher ensured the aspects of problem-solving skills to be utilized in the prototype instrument.
The last stage is the Assessment phase.During this phase, the instrument underwent feasibility test by experts and users.Expert input considered three aspects: validity, practicality, and effectiveness.User-based instrument feasibility was analyzed using the Rasch model to identify items aligning with the instrument development model's needs.

Data Analysis
Data in this study were analyzed using Rasch analysis with Winstep software.The analysis was grounded in two fundamental theorems: individual ability/agreement levels and item difficulty levels to be approved.Psychometric tools for data analysis included summary statistics, respondent and instrument quality, and person-item interaction (Saputra et al., 2023).Item fit was determined based on outfit mean-square values, standardized outfit z-scores, and point-measure correlations, following Boone et al.'s description (Boone, 2016) If a test item failed to meet these criteria, it indicated poor quality, requiring improvement or replacement.This approach ensured that students' understanding would be tested through suitable and high-quality test items.

RESULTS
The results of the development and validation of the PSS Scale constitute a theoretical study and research findings.The research results describe the instrument's quality, the person-item interaction, the item difficulty level (item measures), fit and misfit items, and the distribution of respondents' abilities and the distribution of item levels on the same scale, and the scale's ability to measure what it intends to measure.The analysis of data is derived from the identification of construct validity using Rasch analysis.
The PSS Scale comprises three aspects: Problem-Solving Confidence, Approach-Avoidance Style, and Personal Control.Problem-solving skills were measured using 4-point Likert scale, with responses ranging from Very Suitable (VS), Suitable (S), Not Suitable (NS), to Not Suitable at All (NSA).The aspect, indicator, and item examples are presented in Table 1.Based on Rasch analysis data, the authors obtained summary statistics information in Figure 1.This section provides comprehensive information about the quality of respondents and the interaction between individuals and items.The mean measure on person 1.16 is greater than 0, indicating that the tendency of subjects' abilities is greater than the difficulty level of items.The Cronbach's Alpha value (KR-20) of 0.89 indicates that the test reliability or the overall person-item interaction is very good (notes: < 0.5: poor; 0.5-0.6:weak; 0.6-0.7:fair; 0.7-0.8:good, > 0.8: excellent).With Person reliability at 0.90 and Item reliability at 0.99, based on the Fisher table, the consistency of respondent answers falls into the good category, and the item reliability is in the excellent category (notes: < 0.67: weak; 0.67-0.880:fair; 0.8-0.90:good, 0.91-0.94:excellent; > 0.94: outstanding).Based on the summary data of infit MNSQ and outfit MNSQ for the person table, the consecutive values are 1.02 and 1.01, where ideal Infit MNSQ & Outfit MNSQ values approach 1.The infit zstd and outfit zstd values, with average values on the person table being -0.28 and -0.32, respectively, and the ideal infit Zstd and outfit ZSTD values approach 0. Therefore, it can be concluded that the quality of this instrument is ideal.
Based on the item measure data, the standard deviation of this instrument is 0.99.The standard deviation logit item values are useful for identifying item groups (separation).According to the table, there are four items that do not meet the three criteria, namely items 33, 34, 35, and 36 (outliers or misfit items).These four items were eliminated from the instrument.Two items did not meet the criteria for outfit Z-standard and point measure correlation (items 15 and 21), but the outfit mean square criterion was met, so these two items were revised.Items that only did not meet one criterion were retained without revision.The criteria used to check the suitability of misfit items were  2, items 33, 34, 35, and 36 are considered misfitting items and need to be revised.However, based on the analysis of item difficulty levels, these items were considered outliers, so these four items were eliminated from the scale.
One advantage of Rasch modeling is its ability to generate a map illustrating the distribution of respondent abilities and item difficulty levels on the same scale, known as the Wright Map (Person-Item Map), as presented in Figure 3.The left side of the Wright Map depicts the distribution of respondent abilities, showing some respondents with high abilities at +3 logits and +4 logits.These respondents are beyond the boundaries of two standard deviations (T), indicating significantly different high intelligence (outliers).The right side of the Wright Map illustrates the distribution of item difficulty or the distribution of logit values for items.If logit values are found outside the range of +2SD to -2SD, this condition can be termed as outliers.Item no.36 was considered to have the highest difficulty level, making its meaning very difficult for respondents to understand.Items 1 and 3 were considered to have the lowest difficulty levels.Other items still meet the standard deviation distance requirements and can be well answered by respondents.Wright Map analysis can provide information to identify respondent abilities while simultaneously analyzing the quality of the tested items.Figure 4 demonstrates the scale's unidimensionality.This section aims to evaluate whether the tool can measure what it is intended to measure, in this case, the PSS Scale.Unidimensionality interpretation compares the measurement variance results with the minimum requirements for unidimensionality.The condition that the instrument can measure what is intended is a minimum unidimensionality value of 20% (Saputra et al., 2023).Figure 5 shows the measurement variance results of raw data at 39.9%.The measurement results indicate that the minimum unidimensionality requirement is met at 20%.This analysis means that the instrument falls into the category of excellent in measuring what it is intended to measure.

Figure 5 Unidimendionality
The Rasch model analysis results indicate that the PSS Scale is a valid, reliable measuring tool that can be used to accurately measure the skill levels of students.The results of this measurement tool can be utilized by lecturers to assess the skill levels of each guidance and counseling student.The implementation of problem-solving skills is crucial for students to face 21st-century challenges.

DISCUSSION
Problem-solving skills are vital for prospective educators as they not only develop critical thinking skills but also train prospective educators in managing learning to foster critical thinking skills (Mubarok et al., 2023).Efforts to enhance teachers' competence in solving learning and non-learning problems may have a positive impact.Problem-solving is essential for prospective educators as a focused cognitive operation used to adapt to internal/external demands or challenges (Heppner & Petersen, 1982).Heppner believes that applied problem-solving skills are crucial for solving real-life problems.It is essential for prospective educators to improve problem-solving skills and other critical thinking skills.Prospective educators need problem-solving skills to make their thinking more critical and analytical (Araiza-Alba et al., 2021).Providing opportunities for teachers to solve professional and collaborative learning problems is one strategy to improve education quality (Haryanto, 2016).
Currently, pre-service teachers at universities need strong experiences and skills to face 21stcentury challenges and future education (Ibrahim & Ahmed, 2023).The new generation, especially students, needs to use technology in daily life and should do the same for educational purposes.Students need to develop core competencies, such as problem-solving and computational thinking (BARANA et al., 2020).Educational focus on enhancing problem-solving skills for prospective teachers can be achieved through classroom learning practices, as problem-solving skills can also influence other skills.Deniz (Deniz, 2013) states that problem-solving skills and emotional intelligence are closely related, meaning that if the learning process is focused on improving problem-solving skills, the emotional intelligence of prospective educators can also increase.Mastery of problem-solving skills can also enhance and sharpen an individual's critical thinking ability (Savitri, 2022).Problem-solving ability is closely related to resilience skills; the higher the level of problem-solving ability, the higher the resilience indicator (Coşkun et al., 2014).Furthermore, Yelkin explains that individuals with good problem-solving skills can have a better life than others because they are more successful in finding the best solutions and know how to behave in problematic situations.
The level of problem-solving skills possessed by each individual cannot be observed through the observation process; a scientifically justifiable measuring tool is needed.The urgency of developing a measuring tool/urgency of developing a skill scale is to develop and improve a measuring tool so that its use can be justified (K.T. Marselina et al., 2021).The development of this instrument is important because measurement results are highly influenced by the quality of the measuring tool used, and the quality of the measuring tool is determined by the validity and reliability of the instrument obtained through several validation steps and empirical tests on a limited group (Wahyudi et al., 2018).The development of a measuring tool/skill scale is important because the more instruments there are in identifying skills, the more complex the data obtained from various instruments that can be used (Syahputra et al., 2022).
Instrument development needs to undergo validity and reliability testing.One analysis that can be utilized to obtain a valid measuring tool is using the Rasch model (Maryati et al., 2019).Rasch analysis is a psychometric technique developed to improve the precision of researchers building instruments, monitor the quality of instruments, and calculate respondent performance (Sumintono, 2018).Rasch analysis allows researchers to construct alternative forms of measurement instruments, opening the door to transforming instruments in the growth and change of students.Rasch analysis also helps researchers think in a more sophisticated way regarding the construction (variable) they want to measure.

CONCLUSION
Guidance and counseling students need to take on a role in addressing the challenges of the 21st century.The demand for mastering skills must be accompanied by the availability of a measuring tool.The results of the research and development of the Problem-Solving Skill Scale aim to meet this need.This scale consists of three aspects: Problem-Solving Confidence, Approach-Avoidance Style, and Personal Control.The validation results using the Rasch analysis model indicate that the PSS Scale can be used validly and reliably to measure the problem-solving skills of guidance and counseling students.The findings of this research can be followed up by instructors or future researchers to conduct implementation trials of learning models focused on improving students' problem-solving skills, such as problem-based learning or project-based learning.

Figure 2 Figure 3
Figure 2 Item Difficulty Levels (item measure) of PSS

Figure 4
Figure 4 Wright Map Analysis (Person-Item Map)

Table 1 .
Aspects, Indicator, and Item sample of PSS Scale