CIT 591 Assignment 9: Short Token Counter
Fall 2004, David Matuszek

General idea of the assignment:

This is a programming contest.

The details of the assignment are exactly the same as for the previous assignment: Count tokens from either a String or a Java source file, and display counts in a GUI. However, in this assignment you are to make your program as short as possible, using any Java tricks you can come up with. The best (shortest) programs will win bonus points.


You must use the exact classes and methods specified in the previous assignment (Token Counter). As in the previous assignment, you may not use the classes StringTokenizer, StreamTokenizer, or Pattern. I still recommend the use of a state machine, but if you find a better way, feel free to use it.

Your goal is to minimize the number of non-comment tokens used in your program. That is, (names + keywords + character literals + integer literals + real literals + string literals + operators + punctuation). Since this will probably make your program even harder to read, you may want to add comments to explain any unusual code.

Your "program" will consist of all the classes used to create your program, not counting your JUnit test class.

Purposes of this assignment:

First, let me say that this is not an exercise in good programming practice. Good programs are those that are easily read and understood. By making your program as short as possible, you may do some things that reduce the clarity of the program. That's OK. (Good indentation, spacing, and comments are still required, and do not contribute to the token counts; your GUI must still be easy to figure out).

To do well on this assignment, however, you will have to eliminate redundancies from your code. You will have to examine each part of the program to see whether there is a shorter way of doing the same thing. You may need to introduce methods. It has been my experience that people can learn a great deal from doing this sort of careful examination and revision of code.

As an additional benefit, this assignment should convince you of the value of doing frequent JUnit testing. Since you are making changes to a (presumably) already working program, any change you make might break the program. Without good automated tests, testing is slow and cumbersome, and you will be tempted to make multiple revisions between tests. With JUnit, testing is only a button click away.

I highly recommend the use of Eclipse. In addition to all its other benefits, Eclipse provides better JUnit test reporting than BlueJ, and it provides (via right click -> Local History -> Replace with...) an easy way to revert to any previous working version.


I will once again assign new partners. Each of you should be coming into this assignment with a complete, working JavaTokenCounter program. You and your partner should look over each other's programs and decide which one is the better one to start with--maybe the shorter program, or maybe the simpler program (although these are likely to be the same). Or you can take parts of one program and parts of the other. Or you can start all over again. Whatever.

You can turn in the same program jointly, or (if you have a hard time getting together) you can turn in individual programs.

Note: If you have not yet turned in your previous assignment (due midnight, December 6), you may not use any part of your new partner's code as part of that assignment.


First, your submitted program must be correct. Incorrect programs will receive a zero grade. I will supply a JUnit test class, and your program must pass all the tests therein. My JUnit test class will be as complete and correct as I can make it; the first person to point out deficiencies or errors will get a few extra credit points for doing so. (Note that my JUnit test will test sequences of symbols, not complete Java programs; but all comments, string literals, and character literals will be properly closed.)

The best (shortest) working program will receive 150 points, plus a small "trophy" (of no particular monetary value). The second place program will receive 140 points, the third place program 130, and the fourth place program 120. The next four programs will receive 100 points. After that, each successively larger program will receive 5 fewer points: 95, 90, 85, ..., 50. No correct program will receive fewer than 50 points because of its size.

As usual, points may be deducted from any program, regardless of its size, for stylistic problems that do not affect the token count (including poor spacing, indentation, comments, or difficulty of use). You may violate any other style rules in order to reduce the token count, provided that you explain that in comments; begin such comments with the word STYLE (to make it easy for us to find them).

Due date:

Well, that's a bit strange, too. Please turn in (via Blackboard) a preliminary version of your program by midnight, Thursday December 9. Your Blackboard comments should include, in addition to your partner's name, a count of how many tokens your program contains. This is so that I can post preliminary results on the class web site (hopefully on Friday morning).

Since Friday, December 10, is a class day, we will have an optional lab that day. This is an opportunity to meet with your partner to decide whether to do additional work on your program, and if so, how much. Please please please discuss with your partner beforehand whether you will attend; don't leave them stranded.

The final version of your program should be turned in (again, with a token count) before midnight, December 14. If you don't turn in a second version by then, your preliminary version will be graded as final. Late programs will be accepted as usual (5 points/day, up to a week), but will not compete and will not earn more than 100 points.