The Development of Questionnaire to Investigate Students' Attitudes and Approaches in Physics Problem Solving

This study aims to develop physics problem solving survey utilized to obtain students' attitude and approach while solving physics problem. Several steps are conducted to develop the survey: validating, computing reliability, and testing. This research involves Physics Education students-Tanjungpura University who study at Fourth and Sixth semester. Furthermore, after questionnaire administered to the students, data are analysed through determining average score of the students and computing average percentage of students who are agree, neutral, and disagree based on semester and gender. The results show that the sixth semester students' responses are more expertlike than those of the fourth semester students and female students are less expert than those male students. Based on semester and gender, majority of students have the same attitudes and approaches while solving problems. Students and experts have different attitudes about the role of mathematics in problem solving.


INTRODUCTION
Physics is one of science subjects taught from high school to college. During the study of physics, most students have difficulty under-right answer. For example, when a question is presented in verbal form, students tend to use the formula immediately without thinking of a strategy to understand the problem (TMS & Sirait, 2016). As a result, most students are unsuccessful in determining the correct answer.
The ability to solve problems is very important in learning science especially in physics (Docktor, Strand, Mestre, & Brian, 2015;. From a pedagogical point of view, problem solving can be used as a tool for evaluating student learning (Heller, Ketih, & Anderson, 1992). These issues are usually displayed at the end of each chapter of physics textbooks and other disciplines of science. In addition, teachers can also provide questions to evaluate students' understanding at the end of the lesson and at the end of a course or lecture.
To help students successfully solve physics problems, strategies for solving physics problems have been developed in physics education research.  has developed a problem solving strategy that includes the following steps: 1) visualizing the problem, 2) explaining the problem based on the physics concept, 3) planning a solution, 4) executing the plan, and 5) checking and evaluating. Another physics problem solving strategy is a qualitative solution (Leonard, Dufresne, & Mestre., 1996). This strategy has three main components, namely 1) sets out the main principles or concepts that can be applied to solve problems, 2) justifies principles or concepts to apply, and 3) applies principles or concepts to achieve an answer. Therefore the operational definition of a qualitative settlement strategy is what, why, and how the answer to the question.
The next strategy is an explicit solving strategy. This strategy is part of the learning process by directly teaching students how to use higher techniques to solve problems (Huffman, 1997). Solving problems in textbooks often only presents steps in general and usually emphasizes quantitative aspects. Explicit problem solving tends to emphasize both qualitative and quantitative aspects. The explicit problem solving steps are as follows 1) focusing on the problem, 2) explaining the problem in the physical context, 3) planning the solution, 4) executing the plan, and 5) evaluating the answer. Furthermore, these steps are almost similar to those made by .
The latter strategy is a concept-focused strategy. Docktor et al. (2015) developed a modified physics-problem-solving strategy from Leonard et al. (1996) called the conceptual breakdown. This strategy includes three important parts of the principles (principles or concepts that are appropriate for the problem), justification (explanation of why principles or concepts are appropriate), and plans (a number of steps that present ways to solve problems and include mathematical equations).
In addition to strategy, assessment rubrics are also developed. Kuo, Hull, Gupa, & Elby (2012) and Hull et al (2013) create a rubric that combines conceptual and mathematical reasoning. This rubric can evaluate the work of students who do not follow formal strategies. In addition, Docktor et al. (2016) designed an assessment rubric containing several items including 1) useful descriptions, 2) physics approach, 3) more specific physics applications, 4) mathematical steps, and 5) logical answers.
Research on physics problem solving generally focuses on improving students' abilities through strategies or approaches, giving scaffolding (Lin & Singh, 2015), and also how students can solve problems effectively (Mason & Singh, 2010a;2016). The study of students' attitudes or approaches to physics is limited. The attitude and approach of students to learning has a significant influence on the objects being studied by students. Mastering physics not only develops the knowledge structure of the physics concept but also develops productive attitudes about knowledge and physics learning (Balta, Mason, & Singh, 2016).
A team of researchers from the University of Maryland developed an instrument to explore students' attitudes and expectations toward learning physics. The questionnaire is the Maryland Physics Expectation (MPEX) (Redish, Saul, & Steinberg, 1998). Questionnaire consists of 34 items with answers agree or disagree and given before and after learning. Furthermore, Colorado Attitudes about Science Survey (CLASS) is a similar questionnaire with MPEX that explores students' attitudes about physics teaching (Adams et al, 2006). Based on the data analysis shows that qualitatively the results obtained are the same as the results obtained using MPEX. Then, Attitudes toward Problem Solving Survey (APSS) consists of 20 statements used to explore students' attitudes toward the completion of physics problems (Cummings, Lockwood, & Marx, 2004). Indicators developed from MPEX by focusing on how to solve physics problems in textbooks such as the role of formulas or mathematical equations, the importance of physics concepts, strategies or approaches used in solving physics problems. The questionnaire given to students before and after lectures at three different universities. Furthermore, Attitudes and Approaches to Problem Solving (AAPS) were developed from APSS consisting of 33 statements with five answers that strongly agree, agree, neutral/do not know, disagree, and strongly disagree. The questionnaire was given to physics lecturer, physics graduate student, and physics undergraduate student (Mason & Singh, 2010b).
Inspired by the results of research at several universities abroad, this study aims to develop a Physical Problem Solving Questionnaire (PPSQ) that has been validated and ready to be used to explore the attitude and approach of physics teacher candidates when doing physics. This questionnaire was developed from several findings that students' view of physics learning influenced success in physics learning (Roth, 1994;May & Etkina, 2002;Lising & Elby, 2005). The indicators in this questionnaire consist of several aspects such as concepts, formulas or equations, representations, strategies, and interests in solving physics problems (Redish et al., 1998;Adams et al., 2006;Balta et al., 2016). Development of this questionnaire is very important to do because there is still limited questionnaire about how students solve the physics problem in the Indonesian format. In fact, lecturers can use this questionnaire in their learning as a way to assist students in solving physics problems.

METHOD
The questionnaire developed in this study was adapted from Attitudes and Approaches to Physics Problem Solving (Mason and Singh, 2010b;Balta et al., 2016). Questionnaire is translated first from English to Indonesian.
Furthermore, the questionnaire was then validated by five physics lecturers to determine the suitability of the questionnaire statement with the content of physics. In addition to the physics lecturer, the questionnaire was also validated by five physics semester final students who are working on the final project. Students are involved to know the legibility of each statement in the questionnaire. This validation step follows a strategy by Mulford and Robinson (2002) by involving students to evaluate the readability of the chemical concept test. The validation of the content, which measures the ease of understanding the statement (Barniol & Zavala, 2014) consisting of four options that are very easy to understand (4), easy to understand (3), elusive (2), and very elusive (1). The next step was calculated the average score of each item statement and overall. The average criterion score of each statement fit of use is greater than or equal to three.
After the questionnaire was validated, the questionnaire was given to 10 students as a test to find out whether there was still a confusing statement and also to know the time needed to fill in the questionnaire. Gall, Gall, & Borg, (2005) stated that an initial test or evaluation of the instrument could involve six to 12 participants.
where K = the number of questions = the value of variance of answer to question j = the value of variance of total scores The expected reliability value (α) of the questionnaire and appropriately used is greater than or equal to 0.7. Physical Problem Solving Questionnaires (PPSQ) consist of 30 statements with five choices of answers: strongly agree, agree, neutral or ignorant, disagree, and strongly disagree. The indicator of the questionnaire is the use of representation, mathematical roles and formulas, re-examine answers, the role of concepts, discuss and ask questions, problem solving strategies, and interest in solving problems.
To find out the attitude and approach of students to solving the physics problem, the questionnaire was given to Physics Education students of Tanjungpura University in the fourth and sixth semesters, amounting to 150 people with 45 men and 105 women. This student will be a physics teacher at Junior High School and High School after completing their studies. Questionnaires are given at the time the students take the course. The student's answer is given a score of 1 if it corresponds to an expert answer, a score of 0 for a neutral pick, and a score of -1 for answers that do not match the expert's answer (Mason & Singh, 2010b). Answer very agree and agree given the same score and also vice versa to disagree and strongly disagree. Then the student scores (semesters IV, VI, Male, and Female) are calculated for each item. Next, calculate the average percentage of students corresponding to the expert's answer, neutral/ignorant, not according to the expert, and not answer (Marx & Cummings, 2007).

RESULTS AND DISSCUSSION
The indicators and examples of the PPSQ statements are shown in Table 1. Before the questionnaire was given to the students, the questionnaire was validated first by a physics lecturer as well as a student. The validation is validation of the content to measure the compatibility between the statement with the physics learning and also the degree of ease of understanding the statement. Validators are required to score from score 1 to 4 (very elusive, elusive, easy to understand, and very easy to understand).
Based on the calculation, the average validity questionnaire score conducted by the lecturer (five persons) is 3.43 (easily understood category). For more details, the average validity score for each questionnaire is shown in Table 2. However item 2 has a score of 2.8 so it needs to be discussed with the validator before being further validated by the student. The word "visualize" is changed to "sketch".
Furthermore, the questionnaire was also validated by students with five validators. The average score of 3.65 is easily understood but there are still two statements (9 and 20) scoring below 3. Therefore interviews with validators are used to improve a more understandable word or phrase. Statement number 9 slightly confuses the validator against the word "way and strategy". As for item 20 distinguish between "context" and "situation". Based on the interview results obtained that the word "strategy" and "situation" better known by students. After the score of each statement is greater than or equal to 3 (easy to understand category), the next step is to test the students to find out the time required to fill the questionnaire and also whether there is still a sentence or word that is still confusing. Based on the results of the experiments obtained that the average student answered the questionnaire 10 to 25 minutes, so it was decided the time to fill the questionnaire is 20 minutes. Then, the questionnaire reliability is also calculated using Alpha Cronbach. Results obtained for 0.73 which means 0.1 points lower than the questionnaire of physics solving by Mason and Singh (2010) and Balta et al (2016). Nevertheless this questionnaire is still accep-table and worthy of use (Cortina, 1993;Wilcox & Lewandowski, 2016) Students in general, on average have taken over 60 semester credit units and also they have taken the core courses of physics courses such as Basic Physics, Mechanics, Thermodynamics, Waves and Optics, and others. Thus, it is assumed that students have often solved the problems of physics both in the form of assignment and during the midterm exam or the final exam of the semester.
PPSQ consists of 30 statements with five answer options. The nine items have disagreeable or strongly disagreeable answers that correspond to the expert answers experienced in solving the physics problem and 21 items have the answer agree or strongly agree. Students are required to fill the questionnaire according to the time specified. Questionnaires were collected and then analyzed based on semester taken as well as gender.
Student responses corresponding to expert answers are given a score of 1, which is neutral or or unknown given a score of 0, and which is not in accordance with the expert's answer given a score of -1. The next step is to calculate the average score of each item based on semester and gender. According to the data analysis, it is found that the average score of the students in the fourth semester is 0.44 while the average score of the sixth semester students is 0.46, slightly higher than the fourth semester. This means that the attitude and approach of completing the problems of sixth semester students more appropriate with experts or experienced. For both men and women, the mean scores were 0.48 and 0.44, respectively. The average student score for each item is shown in Table 3. There are five items (1, 3, 10, 11, and 29) that have negative scores (shown in Table 3), meaning that for all of these items students generally conflict with attitudes and approaches by experts or people who have experience in solving physics problems. The lowest score obtained by students is in the statement number 3 and number 11. In point 3, 95% of students assume that mathematics is the most important thing in solving the problem. Data obtained by Mason and Singh (2010) show that approximately 50% of physics graduate students agree with this statement. Then, 50% of undergraduate students in Turkey also agree with item no 3 (Balta et al, 2016). This proves that when students enter physics class or study physics, mathematics becomes the main capital. Though understanding concepts, principles, physics law is important in learning physics (Docktor et al, 2015). Furthermore, to point 11, 95% of students assume that equations or physical formulas can only be used to solve certain problems. Furthermore, for item 29, about 70% of students more easily complete the number calculation compared to symbols or formulas. This shows that students tend to solve problems relying on formulas or mathematical equations. Then, for these five points, students can be categorized at the beginner level or lack of experience, supported also by point 1 that students tend to end their effort when they do not know the right steps to solve the problem. The number of students for each group that has an appropriate, neutral, and inconsistent attitude with experts is also calculated and converted into percentages. The average percentage of students is shown in Figure 1. The histogram shows that the average score is over 50% of the students voted in favor of the expert attitude in solving the physics problem. The percentage of sixth semester students whose answers correspond to the experts is higher than the fourth semester students. That's because the sixth semester student has  Table 4. The indicators of the use of representations (pictures, diagrams, graphs, etc.) are items 2, 14, 16, 17, and 18, scored an average of 80% for students in the sixth semester while semester students fourth around 65% utilize multi representation when solving the problem. By gender, the average percentage of men using multi representation is higher than that of women, which is 78% and 73% respectively. Kohl and Finkelstein (2005) and Rosengrant, Etkina, & van Heuvelen (2006) suggest that students who often use good representation when completing homework as well as exams have higher problem solving abilities.
Furthermore, Savinainen, Makynen, Nieminen, (2013) states that interaction diagram representation can help students to identify kind of force and draw the force diagrams. The TMS and Sirait (2016) study showed that students trained using motion diagrams have better conceptual mastery skills. Students with good vector representation skills tend to successfully identify the force diagram correctly (Sirait, Hamdani. & & Octavianty, 2017). In addition, the use of representations greatly assisted students in studying electricity and magnetism (Kustusch, 2016) and illustrating the diagram greatly helped students learn electric force and electric fields (Cao & Brizuela, 2016).
Indicators of the role of mathematical equations and formulas in the completion of physics problems are listed in 3, 5, 10, and 11. Around 33% of students of the fourth semester agree that working on the problem of physics is simply entering numbers into formulas. Meanwhile, the sixth semester students are more understanding in the use of equations or formulas, only 16% of students agree to the statement. Mathematics is needed as a language to solve the problem of physics, but not just use the formula rather than understand the meaning of the equation in the physical context (Redish & Kuo, 2015). Then, the percentage of men corresponding to the attitude of experts or experienced in terms of use of formulas or equations is higher than that of women ie. 62% and 50% respectively.
In addition, the practice of solving the problem with various strategies is also important for successful completion of the problem. For example, in points 12 and 27 on strategies for solving problems, the majority of students (more than 90%) have an attitude consistent with scientists or experts. This indicates that students need to be equipped with various problem solving strategies such as qualitative problem solving (Leonard et al, 1996), conceptual (Docktor et al, 2015), and explicit (Huffman, 1997). In addition to strategy, other factors that may help the student in solving problems or problems are beliefs of his or her own ability (Yuliarti et al, 2016) and reflective thinking skills (Ellinawati, Rusdiana, Sabandar,  Students' attitudes to ask or discuss with peers when having difficulty when working on physics are shown in the numbers 1, 6 and 23. The average percentage of fourth semester students who ask colleagues is higher than the students of sixth semester that is 80% and 74%. Heller and Hollabaugh (1992) stated that students who taught the completion of physics problems in groups have higher problemsolving abilities than the individual classes. In addition, Mason and Singh (2010a) research results also show that feedback with peers can help students to solve physics problems.
The student's response about re-examining the results of work on physics both assignments and exams have a high percentage. For each item shown at numbers 19, 21, 24, and 28, over 70% of students responded positively. That is, students take the time to check the answers and also investigate where the mistakes they are doing. Zimmerman (1998) said that one of the characteristics of students who succeed in learning is to re-examine the task before submitting to the lecturer.

CONCLUSION
Based on the results of the research can be concluded that the sixth semester students more experienced than the fourth semester in solving the physics problem. This means that attitudes will change with age and experience. Then in terms of gender, men respond more positively or in accordance with experts than women.
Questionnaire completion of physics is expected to be one tool to obtain information about attitudes and approaches of students at the time of completing the matter of physics so that teachers can strive in helping students improve the ability of completion of physics problems. Given the attitudes and approaches of students to problem solving can affect success in answering questions. Furthermore, further research needs to be done to find correlation between attitude and approach of student with ability of physics problem solving.