The Sign Test
© 1998 by Dr. Thomas W. MacFarland -- All Rights Reserved
************ sign_tes.doc ************ Background: The Sign Test is a nonparamteric test that may be of use when it is only necessary (or possible) to know if observed differences between two conditions are significant. That is to say, with appropriate use of the sign test, it would be possible to determine if X is really "more" than Y, or however the conditions are arranged. The sign test is structured so that plus (+) and minus (-) "signs" are used to denote change in magnitude, as opposed to a any attempt at quantitative measurement. Scenario: This study involves a typical situation in educational research where is is desirable to know if an intervention program is of benefit: -- This study determines if involvement in an intervention program promotes cooperation among students in a Fortran programming class when working on group projects. -- However, because the instrument used to measure cooperation has not been refined and subjected to estimates of reliability and validity, the research is only confident that the test measures "gain" or "loss." -- Because the instrument only measures change in either direction, the non-parametric sign test is correctly selected to be the most appropriate test for this study. Ms. Guthrie teaches Frotran programming to a class of 40 post-secondary students at Egg Harbor Technical Institute. Although students receive individual grades for their homework and other class activities, Ms. Guthrie is concerned that her students have not yet learned how to work on programming assignments in a cooperative manner when working on group projects. This concern is important since "real-world" programming requires cooperation among and between team members. To address this concern, Ms. Guthrie unobtrusively judged the individual level of cooperation by her students using a ten-item observation instrument, with the scale structured so that: -- 1 = No cooperation -- 5 = Moderate cooperation -- 10 = Fully cooperative This initial observation was viewed as the pretest. Ms. Guthrie then offered a two-hour lecture on the need for cooperation by members of a programming team. She then again unobtrusively measured the individual level of cooperation of her students on the next group programming assignment. This measure served as the posttest. A summary of the study is presented in Table 1. Data represent individual judgment by Ms. Guthrie, knowing that scores at best show change in direction. Ms. Guthrie fully knows that it would be incorrect to view the data with any measure of precision (i.e., the data are at most ordinal, they are not interval). Table 1 Pretest and Posttest Scores for Fortran Programming Students Regarding Level of Cooperation on In-Class Programming Activities =================================================== Student Number Pretest Posttest --------------------------------------------------- 01 05 08 02 04 06 03 03 03 04 06 05 05 08 09 06 10 09 07 08 07 08 08 08 09 04 08 10 05 05 11 08 09 12 03 05 13 05 04 14 06 05 15 04 04 16 07 08 17 07 09 18 09 09 19 08 07 20 05 08 21 05 06 22 08 08 23 03 04 24 05 06 25 06 07 26 04 08 27 07 08 28 09 10 29 10 10 30 08 09 31 08 08 32 04 06 33 04 05 34 07 08 35 05 07 36 07 09 37 08 10 38 03 06 39 05 06 40 07 08 --------------------------------------------------- Ho: Null Hypothesis: There is no difference in cooperative behavior (as measured by pretest and posttest scores) after Fortran programming students are exposed to a special intervention lecture designed to promote cooperation when working on group programming assignments (p <= .05). Files: 1. sign_tes.doc 2. sign_tes.dat 3. sign_tes.r01 4. sign_tes.o01 5. sign_tes.con 6. sign_tes.lis Command: At the Unix prompt (%), key: %spss -m < sign_tes.r01 > sign_tes.o01 Note: A more robust methodology, using a more precise assessment instrument, may be desirable. Yet, there are times when increased precision in measurement is not feasible, especially for pilot work such as this study. ************ sign_tes.dat ************ 01 05 08 02 04 06 03 03 03 04 06 05 05 08 09 06 10 09 07 08 07 08 08 08 09 04 08 10 05 05 11 08 09 12 03 05 13 05 04 14 06 05 15 04 04 16 07 08 17 07 09 18 09 09 19 08 07 20 05 08 21 05 06 22 08 08 23 03 04 24 05 06 25 06 07 26 04 08 27 07 08 28 09 10 29 10 10 30 08 09 31 08 08 32 04 06 33 04 05 34 07 08 35 05 07 36 07 09 37 08 10 38 03 06 39 05 06 40 07 08 ************ sign_tes.r01 ************ SET WIDTH = 80 SET LENGTH = NONE SET CASE = UPLOW SET HEADER = NO TITLE = Sign Test COMMENT = This file examines if involvement in an intervention program promotes cooperation among students in a Fortran programming class when working on group projects DATA LIST FILE = 'sign_tes.dat' FIXED / Stu_Code 20-21 Pretest 35-36 Posttest 50-51 Variable Labels Stu_Code "Subject Code" / Pretest "Level of Cooperation - Pretest" / Posttest "Level of Cooperation - Posttest" NPAR TESTS SIGN = Pretest WITH Posttest ************ sign_tes.o01 ************ 1 SET WIDTH = 80 2 SET LENGTH = NONE 3 SET CASE = UPLOW 4 SET HEADER = NO 5 TITLE = Sign Test 6 COMMENT = This file examines if involvement in an 7 intervention program promotes cooperation 8 among students in a Fortran programming class 9 when working on group projects 10 DATA LIST FILE = 'sign_tes.dat' FIXED 11 / Stu_Code 20-21 12 Pretest 35-36 13 Posttest 50-51 14 This command will read 1 records from sign_tes.dat Variable Rec Start End Format STU_CODE 1 20 21 F2.0 PRETEST 1 35 36 F2.0 POSTTEST 1 50 51 F2.0 15 Variable Labels 16 Stu_Code "Subject Code" 17 / Pretest "Level of Cooperation - Pretest" 18 / Posttest "Level of Cooperation - Posttest" 19 20 NPAR TESTS SIGN = Pretest WITH Posttest ***** Workspace allows for 26214 cases for NPAR tests ***** - - - - - Sign Test PRETEST Level of Cooperation - Pretest with POSTTEST Level of Cooperation - Posttest Cases 6 - Diffs (POSTTEST LT PRETEST) Z = 3.3588 26 + Diffs (POSTTEST GT PRETEST) 8 Ties 2-Tailed P = .0008 -- 40 Total ************ sign_tes.con ************ Outcome: - - - - - SIGN TEST PRETEST Level of Cooperation - Pretest with POSTTEST Level of Cooperation - Posttest Cases 6 - Diffs (POSTTEST LT PRETEST) Z = 3.3588 26 + Diffs (POSTTEST GT PRETEST) 8 Ties 2-Tailed P = .0008 -- 40 Total When interpreting the above SPSS printout, it is often best to use the following methodology to interpret the meaning of the calculated P value. 1. n - changes = 06 (posttest < pretest) 2. n + changes = 26 (posttest > pretest) 3. n 0 changes = 08 (posttest = pretest) 4. p = .00008 which is <= .05 Conclusion: After careful analysis of the data, Ms. Guthrie rejected the Null Hypothesis and instead observed that there was a significant difference (p <= .05) between pretest scores and posttest scores: -- Behavior was influenced by the intervention activity. -- To be more exact, students were more cooperative after the lecture on the need for cooperation: -- 26 students showed increased cooperation -- 06 students showed less cooperation -- 08 students had no change in cooperation -- The probability that there is no difference in pretest and posttest scores is p = .00008 which is certainly less than p = .05. Behavior was changed (positive gain) because of the lecture. Note. Ms. Guthrie cannot claim that there was 65 percent (26/40) positive change because of the lecture. The sign test does not support this level of measurement. Instead, Ms. Guthrie can only claim that there was positive change. More precise methodologies and inferential tests would be needed to offer this level of judgment. Even so, for initial attempts to examine a problem, the sign test may be a very appropriate test to gain a sense of general direction in trends. ************ sign_tes.lis ************ % minitab MTB > outfile 'sign_tes.lis' Collecting Minitab session in file: sign_tes.lis MTB > # MINITAB addendum to sign_tes.dat MTB > read 'sign_tes.dat' c1 c2 c3 Entering data from file: sign_tes.dat 40 rows read. MTB > name c1 'Stu_Code' c2 'Pre' c3 'Post' MTB > tally 'Pre' 'Post' Pre COUNT Post COUNT 3 4 3 1 4 6 4 3 5 8 5 5 6 3 6 6 7 6 7 4 8 9 8 11 9 2 9 7 10 2 10 3 N= 40 N= 40 MTB > describe 'Pre' 'Post' N MEAN MEDIAN TRMEAN STDEV SEMEAN Pre 40 6.150 6.000 6.111 2.020 0.319 Post 40 7.125 8.000 7.167 1.856 0.293 MIN MAX Q1 Q3 Pre 3.000 10.000 4.250 8.000 Post 3.000 10.000 6.000 8.750 MTB > let c4 = c3 - c2 MTB > print c2 c3 c4 ROW Pre Post C4 1 5 8 3 2 4 6 2 3 3 3 0 4 6 5 -1 5 8 9 1 6 10 9 -1 7 8 7 -1 8 8 8 0 9 4 8 4 10 5 5 0 11 8 9 1 12 3 5 2 13 5 4 -1 14 6 5 -1 15 4 4 0 16 7 8 1 17 7 9 2 18 9 9 0 Continue? y 19 8 7 -1 20 5 8 3 21 5 6 1 22 8 8 0 23 3 4 1 24 5 6 1 25 6 7 1 26 4 8 4 27 7 8 1 28 9 10 1 29 10 10 0 30 8 9 1 31 8 8 0 32 4 6 2 33 4 5 1 34 7 8 1 35 5 7 2 36 7 9 2 37 8 10 2 38 3 6 3 39 5 6 1 40 7 8 1 MTB > stem-and-leaf c4 Stem-and-leaf of C4 N = 40 Leaf Unit = 0.10 6 -0 999999 10 -0 0000 14 0 0000 14 0 (14) 1 00000000000000 12 1 12 2 0000000 5 2 5 3 000 2 3 2 4 00 0 4 MTB > stest 0 c4 SIGN TEST OF MEDIAN = 0.00000 VERSUS N.E. 0.00000 N BELOW EQUAL ABOVE P-VALUE MEDIAN C4 40 6 8 26 0.0005 1.000 MTB > # And be sure to notice that the p value is indeed MTB > # less than .05. MTB > stop -------------------------- Disclaimer: All care was used to prepare the information in this tutorial. Even so, the author does not and cannot guarantee the accuracy of this information. The author disclaims any and all injury that may come about from the use of this tutorial. As always, students and all others should check with their advisor(s) and/or other appropriate professionals for any and all assistance on research design, analysis, selected levels of significance, and interpretation of output file(s). The author is entitled to exclusive distribution of this tutorial. Readers have permission to print this tutorial for individual use, provided that the copyright statement appears and that there is no redistribution of this tutorial without permission. Prepared 980316 Revised 980914 end-of-file 'sign_tes.ssi'