Joomla! site syndication http://thaines.com Sun, 20 May 2012 13:32:40 +0100 FeedCreator 1.7.2 http://thaines.com/images/M_images/joomla_rss.png http://thaines.com Joomla! site syndication http://thaines.com/content/view/112/15/ Given that decision forests (The name with which many people are referring to random forests these days.) seem to be somewhat in fashion right now I decided I wanted an all-singing, all-dancing implementation of one. Ok, maybe the need for one for the bit of research I am planning to do next was the real motivation, but it seems fun to pretend I have some vague comprehension of fashion. Obviously I already have my swood implementation, but that is very limited, and doesn't have the features I need or the modularity required to extend it so that it does, hence a from scratch rewrite. Also, since writing swood, I attended the ICCV 2011 tutorial (http://research.microsoft.com/en-us/groups/vision/decisionforests.aspx) on decision forests, which has strongly influenced this new code. You can find it is the module 'df' in my usual code store, as accessible via the menu. This new implementation is currently somewhat limited - better than swood, but nowhere near where I intend it to end up. Classification support is reasonably complete however, and it does both continuous and discrete features, not to mention it has support for incremental learning. Important features I will be adding in the future include regression support, multiprocessing and the ability to express what it is doing using C++, for speed. Programming - Machine Learning Tue, 14 Feb 2012 10:45:22 +0100 http://thaines.com/content/view/111/16/ The 13th International Conference on Computer Vision (http://www.iccv2011.org/) (2011) is just days away, and my paper promises code. So, my paper is: Delta-Dual Hierarchical Dirichlet Processes: A pragmatic abnormal behaviour detector (http://thaines.com/content/research/2011_iccv/ddhdp.pdf) by me, Tom. S. F. Haines, and Tao. Xiang. The above links through to the pdf, as you would expect, and you can also get the poster from here (http://thaines.com/content/research/2011_iccv/ddhdp_poster.pdf). The code is available from my Google code project, which is accessible from the menu. The approach itself is an extremely complex topic model (Non-parametric and Bayesian.) designed with video data in mind. Its stated purpose, which we demonstrate, is to be able to do semi-supervised learning of behaviour that is normal, except that it is happening in an unusual context. An example of this is we might want to differentiate between people crossing the road and people crossing the road whilst traffic continues to drive across the crossing. The results are somewhat weak however, due to the difficulty of finding real world examples of such behaviours with enough examples for training/testing. I am also not convinced that it fully converges, or that taking a single sample from such a complex Gibbs sampler is sensible (Which is all we can do given the run time.). The complexity is its main failing however - there are so many random variables being sampled, often using extremely complex techniques, that no sane person would ever reimplement this algorithm. Given how much work was involved I'm still somewhat surprised I got it implemented and working in the first place! Research - Computer Vision Thu, 03 Nov 2011 13:58:34 +0100 http://thaines.com/content/view/110/15/ A week today ICCV will have started, so I am currently preparing to upload my code for it. As that code is an extension of the Dual Hierarchical Dirichlet Processes [DHDP] algorithm of X. Wang, K. T. Ma, G-W. Ng and W. E. L. Grimson I thought I should upload my implementation of that first. Which is to say you can now find an implementation in my Google code project (http://code.google.com/p/haines/). Its a bit rough, and very C++ heavy on account of the complex Gibbs sampling that needs to be done. As a bonus its also possible to switch off the 'Dual' part of it to get a straight HDP implementation, or at least something that is for all practical purposes equivalent. Expect another post within the next week concerning my actual ICCV paper, and its associated code. I'll also upload a variational HDP implementation at some point - I worked out the maths a while ago as a learning exercise, but have not yet had the time to convert it into code (Only way to be sure I got the maths right! Plus it might prove useful someday:-) ). Programming - Machine Learning Sun, 30 Oct 2011 20:30:13 +0100 http://thaines.com/content/view/109/16/ Next week is the British Machine Vision Conference 2011 (http://www.computing.dundee.ac.uk/bmvc2011/), at which I have a paper - Active Learning using Dirichlet Processes for Rare Class Discovery and Classification by T. S. F. Haines and T. Xiang. The paper can be obtained from http://thaines.com/content/research/2011_bmvc/dp_al.pdf (http://thaines.com/content/research/2011_bmvc/dp_al.pdf) whilst the source code can be obtained from my code store under the directory dp_al - click on the source code menu item to head there. Edit: You can download my slides, with an additional bonus slide, from http://thaines.com/content/research/2011_bmvc/dp_al_pres.pdf (http://thaines.com/content/research/2011_bmvc/dp_al_pres.pdf) and a video of me giving the talk will be available at some point, once the bmvc people set it up - at that time I'll stick a link up here. Edit 2: You can now watch the video over at the bmva website: http://www.bmva.org/bmvc/2011/proceedings/paper9/index.html (http://www.bmva.org/bmvc/2011/proceedings/paper9/index.html) Whilst potentially conceptually tricky it is a very easy to implement active learning method. It assumes that the categories have been drawn from a Dirichlet process, which is to say that there is a Dirichlet process mixture model over the samples in the pool. As assumptions go it is a pretty weak one, and yet the moment you make it you gain the probability that an entity in the pool belongs to a previously unseen class. An old if mostly unexplored concept (The entropy method of active learning tends to do a little better.) is to select from the pool the entity which the classifier is most likely to classify incorrectly. On its own this idea only targets entities to refine the boundaries of existing categories, but if we include the Dirichlet process assumption, such that an entity could be misclassified either because it is the wrong side of a boundary or because it is an example of something new, its character changes. We now have an algorithm (Refereed to as P(wrong) in the paper.) that selects entities to discover new classes and refine the boundaries of the existing classes, that is particularly suited to situations where there are rare classes. It balances these two tasks without any real parameter tuning. This makes it a perfect candidate for many real world problems, such as automated surveillance, where entirely new behaviours regularly appear but boundary refinement of the existing classes remains important. Research - Computer Vision Wed, 24 Aug 2011 16:26:22 +0100 http://thaines.com/content/view/108/15/ A couple years ago, in a city a fair distance away I made an entry for pyweek (http://www.pyweek.org) 8. It was based on the subject of kettling (Trapping protesters in an area, and often not allowing them to leave.) - it was entered and nobody liked it, so I basically forgot about it. Anyway, it recently came up twice: 1) An article in edge made a comment that someone should make a game on this subject. Well, I already did;-) 2) A friend wanted to make a game about the recent riots in London, so I mentioned this to him. As the above friend then wanted to have a go, and couldn't get my original entry (http://www.pyweek.org/e/Wibble/) to work I uploaded it to the internet so it can be played in a web browser. Programming - Panda 3D Wed, 17 Aug 2011 09:56:37 +0100