CS340 Lab 7 - Learning with Decision Trees

1. We will take a look at decision trees by trying out the restaurant example from the text. Copy the files learn.lsp, dtl.lsp, and restaurant.lsp from ~jillz/cs340. Run xlisp and load those files. Take a look at the restaurant.lsp file and see that this defines a learning problem with the training set from the text. We can construct the decision tree for that problem with the command

(define dt (decision-tree-learning restaurant-problem))

Take a look at the decision tree, dt. The tree is represented as a list of lists with the attribute being at the root followed by lists of the form (value . <tree>). Draw out the tree that dt represents.

We can create a random test case for the restaurant problem with the following command

(define example (car (random-examples 1 (restaurant-problem ‘get-variable ‘attributes))))

Take a look at the example you constructed. We now want the decision tree to predict the outcome of that test set with the following command

(dt-predict dt example)

Trace through the example by hand and verify that you got the correct value from the decision tree. Briefly explain how the result was derived.

2. The text says that any boolean function can be represented by a decision tree by writing the function as a truth table. We have seen that some boolean functions like the restaurant function can be represented very efficiently with a decision tree. There are some functions however that can not be represented efficiently with a decision tree. Consider the majority function which has n boolean inputs and returns 1 iff the majority of the inputs are 1.

a) Write the truth table for this function with 3 inputs i1, i2, and i3.

b) Use the decision-tree-learning function to build a decision tree for this problem. The inputs will be the attributes of the problem and the output will be the goal of the problem.

c) Why is the majority function not represented efficiently with decision trees?

d) Suppose we have a majority function for a very large n. If we took a sample of examples as our training set and built and decision tree, how do you think it would perform on a test set? Try this out for a fairly small n of 6. Use any training set you want of size 6. Try several (at least 6) test set examples. How well did it perform and why?

Remember what training set and test set you used here because you will need them again.