The Sign Test

© 1998 by Dr. Thomas W. MacFarland -- All Rights Reserved


************
sign_tes.doc
************

Background:  The Sign Test is a nonparamteric test that may
             be of use when it is only necessary (or possible)
             to know if observed differences between two 
             conditions are significant.  That is to say, with
             appropriate use of the sign test, it would be 
             possible to determine if X is really "more" than
             Y, or however the conditions are arranged.

             The sign test is structured so that plus (+) and
             minus (-) "signs" are used to denote change in
             magnitude, as opposed to a any attempt at 
             quantitative measurement.


Scenario:    This study involves a typical situation in 
             educational research where is is desirable to 
             know if an intervention program is of benefit:

             -- This study determines if involvement in an
                intervention program promotes cooperation
                among students in a Fortran programming class
                when working on group projects.  

             -- However, because the instrument used to 
                measure cooperation has not been refined
                and subjected to estimates of reliability
                and validity, the research is only confident
                that the test measures "gain" or "loss."

             -- Because the instrument only measures change
                in either direction, the non-parametric sign 
                test is correctly selected to be the most
                appropriate test for this study.

             Ms. Guthrie teaches Frotran programming to a 
             class of 40 post-secondary students at Egg Harbor 
             Technical Institute.  Although students receive
             individual grades for their homework and other
             class activities, Ms. Guthrie is concerned that
             her students have not yet learned how to work on
             programming assignments in a cooperative manner
             when working on group projects.  This concern is 
             important since "real-world" programming requires 
             cooperation among and between team members.
             
             To address this concern, Ms. Guthrie unobtrusively
             judged the individual level of cooperation by her 
             students using a ten-item observation instrument,
             with the scale structured so that:

             --  1 = No cooperation

             --  5 = Moderate cooperation

             -- 10 = Fully cooperative 
             
             This initial observation was viewed as the pretest.

             Ms. Guthrie then offered a two-hour lecture on the
             need for cooperation by members of a programming
             team.  She then again unobtrusively measured the
             individual level of cooperation of her students on 
             the next group programming assignment.  This measure 
             served as the posttest.

             A summary of the study is presented in Table 1.  
             Data represent individual judgment by Ms. Guthrie, 
             knowing that scores at best show change in direction.
             Ms. Guthrie fully knows that it would be incorrect    
             to view the data with any measure of precision (i.e.,
             the data are at most ordinal, they are not interval).
             
             

             Table 1

             Pretest and Posttest Scores for Fortran Programming
             Students Regarding Level of Cooperation on In-Class
             Programming Activities
             ===================================================

             Student Number    Pretest        Posttest
             ---------------------------------------------------

                   01             05             08 
                   02             04             06 
                   03             03             03 
                   04             06             05 
                   05             08             09 
                   06             10             09 
                   07             08             07 
                   08             08             08 
                   09             04             08 
                   10             05             05 
                   11             08             09 
                   12             03             05 
                   13             05             04 
                   14             06             05 
                   15             04             04 
                   16             07             08 
                   17             07             09 
                   18             09             09 
                   19             08             07 
                   20             05             08 
                   21             05             06 
                   22             08             08 
                   23             03             04 
                   24             05             06 
                   25             06             07 
                   26             04             08 
                   27             07             08 
                   28             09             10 
                   29             10             10 
                   30             08             09 
                   31             08             08 
                   32             04             06 
                   33             04             05 
                   34             07             08 
                   35             05             07 
                   36             07             09 
                   37             08             10 
                   38             03             06 
                   39             05             06 
                   40             07             08 
             ---------------------------------------------------


Ho:          Null Hypothesis:  There is no difference in 
             cooperative behavior (as measured by pretest and 
             posttest scores) after Fortran programming students
             are exposed to a special intervention lecture 
             designed to promote cooperation when working on
             group programming assignments (p <= .05).

 
Files:       1.  sign_tes.doc 
 
             2.  sign_tes.dat

             3.  sign_tes.r01

             4.  sign_tes.o01

             5.  sign_tes.con

             6.  sign_tes.lis


Command:     At the Unix prompt (%), key:

             %spss -m < sign_tes.r01 > sign_tes.o01


Note:        A more robust methodology, using a more precise
             assessment instrument, may be desirable.  Yet,
             there are times when increased precision in
             measurement is not feasible, especially for
             pilot work such as this study. 


************
sign_tes.dat
************
                   01             05             08 
                   02             04             06 
                   03             03             03 
                   04             06             05 
                   05             08             09 
                   06             10             09 
                   07             08             07 
                   08             08             08 
                   09             04             08 
                   10             05             05 
                   11             08             09 
                   12             03             05 
                   13             05             04 
                   14             06             05 
                   15             04             04 
                   16             07             08 
                   17             07             09 
                   18             09             09 
                   19             08             07 
                   20             05             08 
                   21             05             06 
                   22             08             08 
                   23             03             04 
                   24             05             06 
                   25             06             07 
                   26             04             08 
                   27             07             08 
                   28             09             10 
                   29             10             10 
                   30             08             09 
                   31             08             08 
                   32             04             06 
                   33             04             05 
                   34             07             08 
                   35             05             07 
                   36             07             09 
                   37             08             10 
                   38             03             06 
                   39             05             06 
                   40             07             08 


************
sign_tes.r01
************
SET WIDTH      = 80
SET LENGTH     = NONE
SET CASE       = UPLOW
SET HEADER     = NO
TITLE          = Sign Test
COMMENT        = This file examines if involvement in an
                 intervention program promotes cooperation
                 among students in a Fortran programming class
                 when working on group projects
DATA LIST FILE = 'sign_tes.dat' FIXED
     / Stu_Code   20-21
       Pretest    35-36
       Posttest   50-51

Variable Labels
       Stu_Code   "Subject Code"
     / Pretest    "Level of Cooperation - Pretest"
     / Posttest   "Level of Cooperation - Posttest"

NPAR TESTS SIGN = Pretest WITH Posttest


************
sign_tes.o01
************
   1  SET WIDTH      = 80
   2  SET LENGTH     = NONE
   3  SET CASE       = UPLOW
   4  SET HEADER     = NO
   5  TITLE          = Sign Test
   6  COMMENT        = This file examines if involvement in an
   7                   intervention program promotes cooperation
   8                   among students in a Fortran programming class
   9                   when working on group projects
  10  DATA LIST FILE = 'sign_tes.dat' FIXED
  11       / Stu_Code   20-21
  12         Pretest    35-36
  13         Posttest   50-51
  14

This command will read 1 records from sign_tes.dat

Variable   Rec   Start     End         Format

STU_CODE     1      20      21         F2.0
PRETEST      1      35      36         F2.0
POSTTEST     1      50      51         F2.0

  15  Variable Labels
  16         Stu_Code   "Subject Code"
  17       / Pretest    "Level of Cooperation - Pretest"
  18       / Posttest   "Level of Cooperation - Posttest"
  19
  20  NPAR TESTS SIGN = Pretest WITH Posttest

***** Workspace allows for 26214 cases for NPAR tests *****



- - - - - Sign Test

     PRETEST   Level of Cooperation - Pretest
with POSTTEST  Level of Cooperation - Posttest


           Cases

               6  - Diffs (POSTTEST LT PRETEST)              Z =     3.3588
              26  + Diffs (POSTTEST GT PRETEST)
               8    Ties                            2-Tailed P =      .0008
              --
              40    Total

************
sign_tes.con
************

Outcome:      - - - - - SIGN TEST


     PRETEST   Level of Cooperation - Pretest
with POSTTEST  Level of Cooperation - Posttest


           Cases

               6  - Diffs (POSTTEST LT PRETEST)              Z =     3.3588
              26  + Diffs (POSTTEST GT PRETEST)
               8    Ties                            2-Tailed P =      .0008
              --
              40    Total


             When interpreting the above SPSS printout, it 
             is often best to use the following methodology to 
             interpret the meaning of the calculated P value.

             1.  n - changes = 06 (posttest < pretest)

             2.  n + changes = 26 (posttest > pretest)

             3.  n 0 changes = 08 (posttest = pretest) 

             4.  p = .00008 which is <= .05


Conclusion:  After careful analysis of the data, Ms. Guthrie
             rejected the Null Hypothesis and instead 
             observed that there was a significant difference 
             (p <= .05) between pretest scores and posttest 
             scores:

             -- Behavior was influenced by the intervention 
                activity.
            
             -- To be more exact, students were more cooperative
                after the lecture on the need for cooperation:

                -- 26 students showed increased cooperation

                -- 06 students showed less cooperation

                -- 08 students had no change in cooperation

             -- The probability that there is no difference in 
                pretest and posttest scores is p = .00008 which 
                is certainly less than p = .05.

             Behavior was changed (positive gain) because of the 
             lecture. 

Note.        Ms. Guthrie cannot claim that there was 65 percent
             (26/40) positive change because of the lecture.  The
             sign test does not support this level of measurement.
             Instead, Ms. Guthrie can only claim that there was
             positive change.

             More precise methodologies and inferential tests 
             would be needed to offer this level of judgment.
             Even so, for initial attempts to examine a problem,
             the sign test may be a very appropriate test to 
             gain a sense of general direction in trends.


************
sign_tes.lis
************

% minitab
 
 MTB > outfile 'sign_tes.lis'
 Collecting Minitab session in file: sign_tes.lis
 MTB > # MINITAB addendum to sign_tes.dat
 MTB > read 'sign_tes.dat' c1 c2 c3
 Entering data from file: sign_tes.dat
      40 rows read.
 MTB > name c1 'Stu_Code' c2 'Pre' c3 'Post'
 MTB > tally 'Pre' 'Post'
 
      Pre  COUNT       Post  COUNT
        3     4           3     1 
        4     6           4     3 
        5     8           5     5 
        6     3           6     6 
        7     6           7     4 
        8     9           8    11 
        9     2           9     7 
       10     2          10     3 
       N=    40          N=    40 
 
 MTB > describe 'Pre' 'Post'
 
                 N     MEAN   MEDIAN   TRMEAN    STDEV   SEMEAN
 Pre            40    6.150    6.000    6.111    2.020    0.319
 Post           40    7.125    8.000    7.167    1.856    0.293
 
               MIN      MAX       Q1       Q3
 Pre         3.000   10.000    4.250    8.000
 Post        3.000   10.000    6.000    8.750
 
 MTB > let c4 = c3 - c2
 MTB > print c2 c3 c4
 
 
  ROW   Pre   Post   C4
 
    1     5      8    3
    2     4      6    2
    3     3      3    0
    4     6      5   -1
    5     8      9    1
    6    10      9   -1
    7     8      7   -1
    8     8      8    0
    9     4      8    4
   10     5      5    0
   11     8      9    1
   12     3      5    2
   13     5      4   -1
   14     6      5   -1
   15     4      4    0
   16     7      8    1
   17     7      9    2
   18     9      9    0
 Continue? y
   19     8      7   -1
   20     5      8    3
   21     5      6    1
   22     8      8    0
   23     3      4    1
   24     5      6    1
   25     6      7    1
   26     4      8    4
   27     7      8    1
   28     9     10    1
   29    10     10    0
   30     8      9    1
   31     8      8    0
   32     4      6    2
   33     4      5    1
   34     7      8    1
   35     5      7    2
   36     7      9    2
   37     8     10    2
   38     3      6    3
   39     5      6    1
   40     7      8    1
 
 MTB > stem-and-leaf c4
 
 Stem-and-leaf of C4        N  = 40
 Leaf Unit = 0.10
 
 
     6   -0 999999
    10   -0 0000
    14    0 0000
    14    0 
   (14)   1 00000000000000
    12    1 
    12    2 0000000
     5    2 
     5    3 000
     2    3 
     2    4 00
     0    4 
 
 MTB > stest 0 c4
 
 SIGN TEST OF MEDIAN = 0.00000 VERSUS  N.E.  0.00000
 
                N  BELOW  EQUAL  ABOVE   P-VALUE     MEDIAN
 C4            40      6      8     26    0.0005      1.000

 MTB > # And be sure to notice that the p value is indeed
 MTB > # less than .05.
 MTB > stop
 
--------------------------
Disclaimer:  All care was used to prepare the information in this 
tutorial.  Even so, the author does not and cannot guarantee the 
accuracy of this information.  The author disclaims any and all 
injury that may come about from the use of this tutorial.  As 
always, students and all others should check with their advisor(s) 
and/or other appropriate professionals for any and all assistance 
on research design, analysis, selected levels of significance, and 
interpretation of output file(s).

The author is entitled to exclusive distribution of this tutorial. 
Readers have permission to print this tutorial for individual use, 
provided that the copyright statement appears and that there is no 
redistribution of this tutorial without permission.

Prepared 980316
Revised  980914
end-of-file 'sign_tes.ssi'

Please send comments or suggestions to Dr. Thomas W. MacFarland

There have been visitors to this page since February 1, 1999.