twk: tests on the web quickly "twik"

twk

twk is a program designed to aid the production and grading of numerical on-line quizzes. A URL of:

http://www.physics.csbsju.edu/cgi-bin/twk/path/file.html

launches program which reads the (mostly) HTML file:

/var/www/httpd/twk/path/file.html

file.html would typically begin:

*set show=1
*echo
<title>My Title</title>
<h2>

etc...

Lines that start with "*" are not displayed on the browser; instead these are commands for the program to take some actions. Here are a list of these commands:

Commands

ECHO

Enables echo-to-browser of non-command lines

NOECHO

Disables echo-to-browser of non-command lines

SET vname1=value1 vname2=value2 etc.

Assigns values to variables. Variables that did not previously exist are created.

LET vname=expression

Evaluates the expression and puts result in vname. Variables that did not previously exist are created.

USE variable-list

The list of variables in variable-list are created with value zero. The "variable" F(X) is a special case: USE F(X) initializes F(X) with an expression previously supplied by the common gateway interface (CGI). Similarly USE FUNCTn compiles the string FUNCTn as the expression for F(X).

WRITE expression format

Evaluates the expression and prints-to-browser the result using the fortran format format. The program uses the format the guess if expression is intended to be real, integer or byte (ascii) data. Note the extension of fortran formats to include "web.sig" which is similar to fortran g format except the result is web friendly: i.e., 6.02 × 10²³ rather than 0.602E24.

LOG expression format

Evaluates the expression and prints-to-access_log the result using the fortran format format. The program uses the format to guess if expression is intended to be real, integer or byte (ascii) data. (web-format is not available.) Note that a variety of POSTed form data is automatically written to the access_log

SHOW

Prints-to-browser a variety of information about the state of the program. This is mostly for debugging.

HELP

Prints-to-browser a summary of commands. This is mostly for debugging.

IF (condition) THEN

The logical expression: condition is evaluated. If the expression is .true. the block of lines to the corresponding ENDIF, ELSEIF, or ELSE are read; if the expression is .false. the block of lines are not read. The condition must consists of atoms which are simple comparisons using exactly one of .eq. .ne. gt. .ge. lt. .le.. For example

b**2.gt.4*a*c .

These atoms may be connected with boolean binary operations: .AND. and .OR. and the boolean function: .NOT.(). For example:

((x.gt.4).and.(x.lt.10)).or..not.(answer(1).eq.116.or.answer(1).eq.84)

IFELSE (condition) THEN

ELSE

ENDIF

See above.

HIDE vname format

Writes-to-browser

<input type=hidden name=vname value=XXX>

This is used to send values to following pages via CGI.

STOP

Discontinue output-to-browser and process no additional lines of the file.

The first line of our example:

*set show=1

requires an explanation. show is a variable that controls the display of calculated results. The default (show=0) disables all calculated output (e.g., the results of WRITE).

Example A

Click here to see a command-listing for example A; click here to see the results. I you hit Netscape's "Reload" key, the numbers used in the problem will change. Note that the list of "random" numbers used is determined by the seed, which, in this example, is set by the time-of-use and process id. One could just as easily make the seed determined by user data like a social security number.

Now this problem needs to be graded. If you looked closely at the command-listing or the document source, you should have noticed the FORM:

<form METHOD="POST" action="/cgi-bin/twk/examples/A.jr">

This statement says that the form inputs will be POSTed to the program twk which will use the file:

/var/www/httpd/twk/examples/A.jr

to do the grading.

Click here to see a command-listing for A.jr; click the "Grade Me" button on the above form to see how the quiz is graded. The raw data is automatically logged to access_log, I've appended the total graded score and the correct answer to that log. Click here to see the tail of the access_log. For all the below examples you can return here to see the log produced.

Notice that user-entered data makes grading difficult. Judgment is required, e.g., is the entered number close enough to count? What happens if nothing is entered or simple character data (e.g., "five gs") is entered. Try it and see! With character data the variable ends up with a value equal to the length of the string. Thus "five gs" would produce g=7 (however you are disallowed from entering that many characters).

Example B

Click here to see some options for restricting user data entry. click here to see the results. You may want to hit the "Grade Me" button to see the results, look at B.jr file that grades the problem, and view the log

Example C

Because of the problems of user entered numbers, twk was designed to handle multiple guess questions, including whole tests of multiple guess questions.

Here is an example. Note that this is a straight HTML file; twk is doing no rewriting, so you can View the Document Source in Netscpae. Here is a summary statement as to how the SHOW variable works:

SHOW - amount of help displayed; 0:nothing 1:writes 2:wrong 3:wrong+help1
       4:wrong+help 5:wrong+notsure 6:wrong+notsure+help1  
       7:wrong+notsure+help 8:all 9:all+help1 10:all+help

Questions with character answers (i.e., multiple guess questions) should be named as numbers in the range 1-1024. The characters selected in the form will be compared to answers in the .jr file, the results tabulated in variables like NRIGHT, and finally logged in the access_log. Take a look at the access_log. See how the results are summarized in a line where a lower case letter indicates a correct answer and an uppercase letter indicates a wrong answer. The "not sures" are indicated below the right/WRONG line. Note the use of the underscore variables (e.g., 66_ ) as "not sure" flags. If you were taking a survey rather than grading answers you would record a non-occurring correct answer (say X) in the .jr file so everything would be wrong and use SHOW to turn off displayed grading (e.g., SHOW=1). Non-responses would be tabulated in the variable NBLANK.

The structure of the .jr file is similar to the previous examples except that non-command lines not subject to echo are to have the form:

question_number correct_answer
line 1 of "help"
additional lines of help

question_number correct_answer
line 1 of "help"
additional lines of help

etc...

the line between questions must be totally blank: no white space like tab or space. While the function ANSWER() is filled in as soon as you hit the Submit button, the functions NQUEST, NRIGHT, NBLANK, and RIGHT are updated as each line of the answer from the .jr file is read. Thus you can insert code to evaluate question:

1 A
Stupid: A is obviously the right answer!

*if (right.eq.1) then
*elseif (nblank.gt.0) then
*echo
why did you leave this equation blank?
*noecho
*let nblank=0
*else
*echo
why did you answer
*write answer(1) 'a1,1H?'
clearly A is the right answer!
*noecho
*endif
2 B
another easy question

Example D

I've described how you can make the numbers in a question vary from hit to hit. You may also want to vary the entire question. Alternative questions have names like: 1_, 2_, ..., 1024_. twk can choose question 1 or 1_ at random. (Note that in Example C, I used those alternative questions for the "not sure" button.)

Here is the .html file; Here is what it looks like; Here is the .jr file; you may want to view the log.

Note that instead of using the intrinsic grading options encoded by SHOW you can provide you own grading text. The .jr program can know each "random" question selection if it is passed the initial value of SEED.

Example E

Random selection of one question between more than two alternatives is also quite possible: just use a different question number for each alternative. Since you probably want to show the user sequential question numbers, there will be a disconnect between the question number as known by the user and the question number used by jr. Again you can either use the intrinsic grading options in SHOW (e.g., the total NRIGHT) or provide your own grading dialog.

Here is the .html file; Here is what it looks like; Here is the .jr file which uses its own grading dialog rather than SHOW; you may want to view the log.

Example F

Our multiple guess questions (e.g., in Example B) are constructed with varying input (e.g., planet mass) and varying options which are adjusted according to the inputs. In this way we hope to test if students know how to develop and apply a formula for the answer in terms of the inputs. In physics we try to stress this ability to work formally with "letters" rather than concretely with "numbers" (e.g., dealing generally with a mass of "m" rather than a specific mass of 5 kg). Since science is exactly seeing the general case in specific cases, I believe that learning to think in "letters" is a key outcome for science classes. So why not directly test if students know the formula? The response that students prefer to deal with numbers and if they can solve many particular cases they really know the general case is, of course, incorrect and not to the point. (This argument can be dressed up to say that problems with "letters" are developmentally inappropriate.) We, of course, want to avoid the false generality of formula memorization such as that that produces the famous "Three Laws of Electricity" (V=IR, I=V/R, R=V/I) situation. Whatever its pedagogical merits, there are many problems implementing computer formula checking. The "Three Laws of Electricity" are in fact one law, but it takes a fairly sophisticated symbol manipulating program like Mathematica to detect this equivalence. (This is one small example demonstrating the need for human grading.) In addition formula entry involves special skills for encoding formulas in computer-readable form (e.g., what is the exponentiation symbol? Is juxtaposition enough to denote multiplication or is a special symbol required? Is A/B*C, A/(B*C) or (A/B)*C? etc...) The formula-entry options in twk are not up to solving these general problems. The formula-checking option available in twk is basically the reverse of the multiple choice situation: put random values in the entered formula and see if the results match the correct formula. The required twk-readable formula format is basically that of fortran; naive users will certainly fail without help. With these provisos, here is an example: .html file and what it looks like and the grading .jr file.

Example G

This .html file and what it looks like and the grading .jr file are more like a traditional quiz stressing equations rather than numbers.

Security

Instructors commonly accept unvalidated user-supplied data (e.g., student name) on classwork. On special exams at large universities I have seen picture ids used to validate user-supplied data (e.g., is this guy claiming to be "John Smith" a ringer?). The low cost of multiple submission (not even a nickel for the Xerox machine!) can entice students to do what they would not usually do on hardcopy classwork. This is particularly true if grading is instantaneous: students have been taught from computer games that "Game Over" comes at no cost, one just hits the "Try Again" button. If grading is done on-line you will want to publish a policy on the value (if any) of the second draft of submitted quizzes. Clearly with on-line graded multiple-guess quizzes, anyone can score 100 percent simply by repeated submission. The policy may even entice students to submit initial drafts under pseudonyms. It is surprisingly easy to detect this behavior by looking at the access_log, however prevention is quite easy. The solution is to require username/password access to your documents. This can be done simply by putting the twk program in a directory that has an ".htaccess" file like:

AuthUserFile /var/www/httpd/passwords/file
AuthGroupFile /dev/null
AuthName twkUsers
AuthType Basic
<Limit GET POST>
require valid-user
</Limit>

where the file contains the usernames and encrypted passwords. Validated usernames are logged to the access_log. Although hardly fool-proof, that last four digits of the students social security number provides a password that the instructor typically knows prior to the start of class. (In fact agreeing on a username can be more difficult. Here at CSB/SJU students use first initial, middle initial, lastname (e.g., jfkennedy), but students may lack a middle initial and the definition of "lastname", etc. may be culturally dependent.) Since usernames and passwords can be confused and forgotten it is wise to post a help page. Here is an example.