CS 224 Project 1 - Writing JackTokenizer

Build a tokenizer for Jack programs

Take a look at the project in Chapter 10 of N2T and complete Stage 1, implementing the JackTokenizer module as described on pp219-220 in the text.  The text also explains how you can test the correctness of your tokenizer by comparing your output with the output provided in two test programs. 

Create a Java Project in Eclipse.  Create a class JackTokenizer as described on pp214-215 in the text.  You can use an Enum type or constants of  the type "final static int" to represent the various tokenTypes and keyWords. 

Also create a class CompilationEngine.  This class should only have a constructor and a compileClass method at this point.  The compileClass will contain a single while loop which loops while the tokenizer has more tokens and writes the results from the tokenizer to the output file.

Since the argument to your code can be either a .jack file or a directory containing multiple .jack files, the following main program will deal with those two cases. 

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileOutputStream;
import java.io.OutputStreamWriter;

public class Jack {

@param args
    public static void main(String[] args) {
        String path = args[0];
        String[] files = new String[1];
        File directory = new File(path);

        CompilationEngine compiler = null;

            String fileName = directory.getName();
            files = directory.list();
            files[0] = directory.getName();
            path = directory.getParent();

        for (String inputFile: files){
            int period = inputFile.indexOf('.');
            String extension = inputFile.substring(period+1);
            String file = inputFile.substring(0, period);

            if (extension.equals("jack")){
                compiler = new CompilationEngine(path+"/"+inputFile, path+"/"+file+".xml");



When you execute your main method, you will need to add an argument to the Eclipse run configuration.  Select Run-Run Configurations and then select the Argument tab.  Type a string in the box for the path to the argument which will be compiled.  So for example, if you have a directory which contains both your Jack Project directory and the Square directory with the test files, your argument would be "../Square" which takes you up one directory from the project directory and then selects the Square directory. 


Academic integrity is very important as you embark upon this project.  Wrestling and finally mastering the concepts needed to complete this project will greatly improve your understanding of how programming languages work. Since your learning is the highest priority to me, I want to clearly emphasize the following:

  1. The process of figuring out what you need to do to complete the project is where the learning is happening.  By cheating and looking at an online or another student solution, you are robbing yourself of this learning.  Remember that learning is why you are here, right?!
  2. You will sometimes find yourself frustrated and stuck as you embark upon this learning process.  When this happens, instead of cheating yourself by looking at a solution, talk out your problem with me or a fellow student. 
  3. Helping a fellow student does not involve giving them code but you can explain to them how to proceed.  Doing so benefits both the student getting assistance and the student helping out.

Submission and Grading:
Zip and submit your entire project directory in GoucherLearn by the due date even if it is not working perfectly.  After I have graded on the correctness of your code and made comments, if you have errors I will ask you to resubmit after making the changes I suggest.  In doing so, you will have the opportunity to earn back up to 50% of the points you lost in the first submission.  You will need a tokenizer for the rest of the project so it is important that you get this working.