Noise exists when the tree is induced with large amount of fringe data attached. This is data that will never result in a positive outcome when searching the tree for a decision, and only impedes the speed of the search. Attempting to eliminate these issues is the reason for the existence of pruning algorithms.
Trees can either be pre-pruned or post-pruned. Pre-pruning involves controlling the growth of a tree as it is induced, and blocking new subtrees that predictors believe won't yield fruitful results. Pre-pruning algorithms, though more efficient, do yield some problems. The most common objection to pre-pruning is that the tree might be pruned at a node when the node's descendants would not have been pruned by the same criteria. That is why most pruning methods are of the bottom up post-pruning variety. Post-pruning refers to the method of stepping through an already existing tree and reducing subtrees to leaf nodes or deleting them altogether. Most pruning algorithms are actually post-pruning algorithms.
Classical Methods.
Minimal Cost-Complexity Pruning (MCCP).
Developed by Brieman (1984), MCCP prunes based on two factors: cost complexity and a constant V1. Cost complexity C for a tree T is defined as the sum of the resubstitution cost and the product of the number of T's leaf nodes, factored by the user defined parameter V to determine the limit of the tradeoff between cost and complexity1 .This algorithm takes place in two parts: First, several smaller trees, beginning with the induced tree and finishing with the root, are generated by continually pruning the subtree with minimum C. Using bottom up sequencing, the trees in the sequence result in being nested from the root node. In contrast, however, with many other bottom up algorithms, MCCP considers all internal nodes contiguously, rather than only those at the lowest point in the tree. The second step consists of choosing the best tree created in the first sequence based on its size and accuracy in regards to the validation set.
DISTANCE METRICS A metric, , is a global function on a domain which indicates a human notion of distance. In our case, the domain is the set of all possible data instances. A metric function must conform to the following five axioms 1. non-negativity, 2. symmetry, 3. identity, 4...
The language model provides the structure that allows decisions to be made (often based upon probabilities, again using HMM's) as to how phonemes may be combined into words and sentences. ... {Deshmukh, 1999] takes us through these aspects and, while pointing out that most systems use a tri-gram back-off language model (for reasons of practicality which will be explored later in this essay), explains that various other approaches have been explored including higher order N-grams, long-range dependencies, cache, link and trigger models, class grammars and decision tree clustered language ...
