Visualizing decision trees in scikit-learn

For single decision trees:

http://scikit-learn.org/dev/modules/tree.html

http://stackoverflow.com/questions/10570042/visualizing-a-decision-tree-example-from-scikit-learn

Hints on how to do it for a Random Forest or Extra Tree classifier:

http://stackoverflow.com/questions/17057139/how-to-find-key-trees-features-from-a-trained-random-forest

http://stackoverflow.com/questions/17362576/random-forest-implementation-in-python

11 open source tools for making the most of machine learning

A summary of open source machine learning libraries.

http://www.networkworld.com/article/2855100/opensource-subnet/11-open-source-tools-machine-learning.html

Random sample consensus (RANSAC)

Random sample consensus (RANSAC) is an iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers. It is a non-deterministic algorithm in the sense that it produces a reasonable result only with a certain probability, with this probability increasing as more iterations are allowed. The algorithm was first published by Fischler and Bolles at SRI International in 1981.

A basic assumption is that the data consists of "inliers", i.e., data whose distribution can be explained by some set of model parameters, though may be subject to noise, and "outliers" which are data that do not fit the model. The outliers can come, e.g., from extreme values of the noise or from erroneous measurements or incorrect hypotheses about the interpretation of data. RANSAC also assumes that, given a (usually small) set of inliers, there exists a procedure which can estimate the parameters of a model that optimally explains or fits this data.

http://en.wikipedia.org/wiki/RANSAC

Tutorial:

http://vision.ece.ucsb.edu/~zuliani/Research/RANSAC/docs/RANSAC4Dummies.pdf



Matlab code:

http://www.mathworks.com/discovery/ransac.html

Python code:

http://wiki.scipy.org/Cookbook/RANSAC

Just for fun RANSAC song:

https://www.youtube.com/watch?v=1YNjMxxXO-E

Corner Detector

Detecting corners is often a good first step in computer vision.  If you can match corners from two images you are well on your way to figuring out how they fit together for example.

Corner detection is an approach used within computer vision systems to extract certain kinds of features and infer the contents of an image. Corner detection is frequently used in motion detection, image registration, video tracking, image mosaicing, panorama stitching, 3D modelling and object recognition. Corner detection overlaps with the topic of interest point detection.

http://en.wikipedia.org/wiki/Corner_detection


Lecture slide decks:


http://www.cse.psu.edu/~rcollins/CSE486/lecture06.pdf

http://courses.cs.washington.edu/courses/cse577/05sp/notes/harris.pdf

Tutorials:

Python/OpenCV

http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_feature2d/py_features_harris/py_features_harris.html

YouTube video:

https://www.youtube.com/watch?v=vkWdzWeRfC4

Matlab code:

http://www.mathworks.com/matlabcentral/fileexchange/9272-harris-corner-detector

Deep Learning

Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence.

Website of organization dedicated to all things deep learning:

http://deeplearning.net/

Multiple tutorials and a wealth of other info:

http://deeplearning.net/tutorial/intro.html


Peer review paper:

Theoretical results suggest that in order to learn the kind of complicated functions that can represent high- level abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers or in complicated propositional formulae re-using many sub-formulae. Searching the parameter space of deep architectures is a difficult task, but learning algorithms such as those for Deep Belief Networks have recently been proposed to tackle this problem with notable success, beating the state-of-the-art in certain areas. This paper discusses the motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer models such as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks.

http://www.iro.umontreal.ca/~bengioy/papers/ftml.pdf


Deep learning implementations in many languages:

http://deeplearning.net/software_links/

Restricted Boltzmann machine

Learning to use RBM's is on my todo list...I'll update when I get around to it.  RBM's are just one technique for deep learning.

The Restricted Boltzmann Machine (RBM) has become increasingly popular of late after its success in the Netflix prize competition and other competitions. Most of the inventive work behind RBMs was done by Geoffrey Hinton. In particular the training of RBMs using an algorithm called "Contrastive Divergence" (CD). CD is very similar to gradient descent. A good consequence of the CD is its ability to "dream". Of the various machine learning methods out there, the RBM is the only one which has this capacity baked in implicitly.

http://bayesianthink.blogspot.com/2013/05/the-restricted-boltzmann-machine-rbm.html#.VCnWzikijjI

This is some Matlab code a guy made of a class he was taking.  It is probably not great but if you are working in Matlab it is probably better than starting from scratch:

https://code.google.com/p/matrbm/

RBM tutorial:

http://deeplearning.net/tutorial/rbm.html#rbm


RBM in scikit-learn:

http://scikit-learn.org/stable/modules/neural_networks.html



A Practical Guide to Training Restricted Boltzmann Machines:

http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf

Multi-scale Oriented Patches (MOPS)

MOPS is a useful tool for many computer vision and related applications. They key is that it gives you a way to generate features that are (mostly) invariant to rotation and scale to use as input for machine learning/artificial intelligence algorithms.

http://www.cs.bath.ac.uk/brown/mops/mops.html


Technical report from Microsoft Research:

http://research.microsoft.com/pubs/70120/tr-2004-133.pdf

Nice slide deck on MOPS:

http://www.csie.ntu.edu.tw/~cyy/courses/vfx/08spring/lectures/handouts/lec06_feature2_4up.pdf


Example app from Microsoft Research:

http://research.microsoft.com/en-us/um/redmond/groups/ivm/PhotoTours/