Moving to Django
My web server has been in desperate need of replacement for some time - its version of Ubuntu server was reaching end of life, and it kept running our of RAM because it was using the smallest of the old type of Amazon web services virtual machines. This meant that every time someone tried to brute force the 3Dami websites password (disturbingly common - the problem with using Wordpress) or we had a major burst in traffic the database would use up too much memory, and be killed, to save the server as a whole. Rather irritating to say the least - the old setup actually had a cron job run every 10 minutes to start the db server again if it had gone down!

Have been waiting for the release of Ubuntu server 16.04, which is now installed on a 'one size bigger' server, with SSD storage instead of magnetic - this server should be a hell of lot faster, and much more robust. Doesn't matter for this, my personal website, but for the 3Dami website (same server!) it will really help, particularly as we are about to start the advertising push for this years event(s).

Unfortunately, as I discovered with the move, my personal website was not compatible with more recent versions of PHP, on account of running on a really old CMS. In fact, I have been using that CMS since I was an undergrad at university, which may give an indication of just how old and unsupported it is. As I am hardly downgrading PHP it seems my bank holiday weekend is now an exercise in reimplementing my website using Django. Am aiming for feature parity and maintaining the urls from the previous website (as redirects to the new and improved urls). At the time of writing this post I have only got the bare minimum up - its going to be at least another day before it's back to normal.

Sunday evening edit: Website now has feature parity with the old one, though is looking really ugly and I have copied over only the most recent articles from the old system.

Monday morning edit: All of the old content is now up. Now I need to go make sure all the old (external) links work and make it pretty.

Monday lunchtime edit: All done!
Last weekend I pushed two more modules to my Helit Github. Both are part of the 'My Text in Your Handwriting' project. Similarly to the mean shift module, they are reasonably generic, and can be used to solve many tasks:

Fast Random Forest (frf):

This is the result of my frustration with using the scikit-learn random forest implementation. Whilst the actually random forest is perfectly acceptable, the file I/O is a joke (its dependent on Python\'s pickle, which is not appropriate for loading/saving large numpy arrays). It had reached the point where I was spending more time loading the random forest from disk than actually using it, and this was in a GUI, so I could not ignore it. Hence I wrote my third (!) random forest implementation, as the other two did not have the necessary features.

To be clear, the use of 'fast' in the name is a reference to the file I/O more than the actual training/testing time. Whilst training/testing is fast, the code is also very generic, supporting many different scenarios, so there are certainly faster implementations out there. Its actually about the same speed as scikit-learn for training, though considerably more flexible. Its designed so that, with the exception of an index built after loading, the file layout and in-memory layout are identical - loading is therefore seriously fast, as its two reads for the header then two reads per tree. Two reads are because the first gets the size of the object, then the second gets the remaining data. Rebuilding the index takes almost no time as well. The file I/O is also exposed such that sending models over the network/between processes is trivial, so that you can distribute both training and testing if you want.

Feature set is currently fairly standard, but it supports multiple output features as well as the usual multiple input features, and any feature can be discrete or continuous. For output that is the difference between a classification tree or a regression tree, though you can also have mixed mode with multiple outputs. Code is very modular and all in C - being my third go at this its disturbingly neat;-)

To figure out how to use it I would focus on the many tests provided, even if some of them are quite silly. The minion type classifier is my favourite, but none of them can be accused of sanity! Its about 8000 additions according to git - I feel sorry for my fingers.

Gaussian Belief Propagation:

I originally used GBP for my paper 'Integrating Stereo with Shape-from-Shading derived Orientation Information', many moons ago, but reimplemented it with a Python interface for the 'My Text in Your Handwriting' paper. Unlike the previous version is allows for an arbitrary graph - you can use a chain to get Kalman smoothing, or a 2D grid to uncurl a normal map (which is an included demo), for instance. It can also be used to solve sparse linear equations - a demo is included. About 4000 additions according to git - this brings the total for the handwriting project up to 20K, and I am yet to publish anything handwriting-specific!

Whilst coding it I added support for TRW-S, in addition to the usual BP-with-momentum. This was an experiment to see what kind of difference it made. GBP is expected to converge to the global solution (if it exists), so you don't expect improved results, but it certainly converges faster. I also tried using it to solve linear equations - it can solve matrices that don't converge with normal BP, which surprised me. This result was only ever confirmed experimentally however, and I haven't figured out what the mathematical basis for it may be.

Yesterday I pushed a rather large update to my Github code repository, for the mean shift module. Over 8K lines were changed when I applied the patch! (development was in a different repository, so a patch was the easiest way to move it over)

Technically, I had already shared all of the mean shift modules code as relevant to my paper "My Text in Your Handwriting" some time ago, but the module had continued to be developed for another project, that then moved on and stopped using it. This is simply a good opportunity to share code that would otherwise remain unused.

Originally, the mean shift module just did mean shift, albeit for arbitrary data (I had needed to do the traditional 2D segmentation scenario, but also 3D mode finding in the RGB colour cube). But then I needed kernel density estimates, and of course mean shift is just a technique for finding the modes of a kernel density estimate, so it makes sense to support that, as the exact same interface works. It also gained the subspace constrained variant of mean shift (Gaussian kernel only) as an experiment that went nowhere - it was too computationally intensive to be justified. Plus I had gone to town with a proper set of spatial indexing data structures and a fully featured interface. This is the state the code was in before yesterdays update.

The improvements are fairly extensive. Firstly, a lot of bugs have been squished, and there are now loads of test files, 34 in fact. Some generate pretty pictures:-) Secondly, it has much better support for directional distributions and composite kernel support - you can, for instance, now have a feature vector with the first three entries position and the second three rotation (angle axis representation) and build a proper distribution over the location and orientation of an object, by setting the first three to be Gaussian, and the second three to be a mirrored Fisher distribution over the angle axis converted to a Quaternion (which will happen internally, so you can continue to use angle axis externally). This conversion system has to be configured, but is very flexible, within the constraints of the built in converters.

The biggest improvement however is support for multiplying the distributions represented by MeanShift objects, using the technique found in the paper "Nonparametric Belief Propagation", by Sudderth et al. This works for all kernels, though its only really optimised for Gaussian and Fisher (inc. composites of them and the mirror Fisher. For everything else it drops back to Metropolis-Hastings for drawing, and Monte-Carlo integration for estimating the probabilities fed to the Gibbs sampler. Slow!). This makes it trivial to implement the nonparametric belief propagation technique, as you can use MeanShift objects as messages/beliefs.

This is a little unusual for me. Normally, I only upload code when its attached to a paper, so anyone who uses the code will (I hope!) cite the associated paper. Sure, this module is used, heavily, by "My Text in Your Handwriting", but the entirety of this update is not. My only other module like this is my Dirichlet process Gaussian mixture model (which is also my most used module - people occasionally cite my background subtraction algorithm when using it, but there is no shared code, just a shared idea. Just my luck that my most popular code only rarely leads to citations!). But this code has been sitting around for a while, and it would be a waste to not publish it. So here it is - if anyone wants to collaborate on a research paper that uses it, then please email me!

It seems like forever since I last posted about my research. This paper actually took three years and four days, from the first day I worked on it through to the day it was accepted! Obviously I worked on other stuff during that period, but its still a hell of a long time for one paper, even if the results are awesome (Well, I think so!). Its also long: 15 pages (double column!), followed by 2 pages of references then 11 pages of appendices (a total of 28 pages), not to mention the almost 9 minutes of supplementary video. I almost feel sorry for the reviewers, but then they kept asking for it to be longer...

Put simply, we present a method that allows us to replicate the handwriting of anyone for whom we have a sample. Any scan will do - nothing special is required at capture time. So we can take a sample of Arthur Conan Doyle's handwriting and write anything we want in it:

Sherlock Holmes does not say that in any of the books Arthur Conan Doyle wrote - it was actually introduced by a 1929 film (Or possibly a stage play in 1899. Or possibly in a totally different series of books by P. G. Wodehouse in 1915. Its fuzzy.). The above is a historical impossibility.

Here is the main video:

My favourite bit is the replacement of text in a film. I had a lot of fun setting up a green screen in my bedroom, with proper lighting, and then building a virtual set (camera mapping of a HDR photo) to composite it into. There was a wide shot with a corpse in the original, but that got cut for some reason... The mini-music video is also particularly fun - just don't ask how many takes it took to get it in time with the music! There is also an interface video.

You can download the paper here: My Text in Your Handwriting (Warning: That includes all of the appendices, so you may want to print just the first 18 pages.), and it also has a project web page over at UCL. Edit: Its now up on the ACM website

Its not a paper for which I can give a one line technical summary - its a systems paper rather than an algorithm paper (not that we set out planning to make a systems paper!). Technologies used include dynamic programming, Kalman smoothing, graph cuts, random forests, mean shift, transfer learning and a whole bunch of image processing operations. One of my favourite tricks is where I smooth text in a signed distance function representation to ensure that thin lines don't get destroyed, but there is so, so much in there. You will have to read the paper! The key point is it really does work. We tested it on real humans (well, humans found at a university, which are a reasonable approximation) and they were unable to tell the difference between real and fake, even when we gave them examples of the authors real handwriting (I get it right most of time, but not always!).

So why did we do it? (beyond the usual 'because we can', of course) There were some jokes about using it to rob banks - we are poor academics after all. But to be serious some disabilities take away a persons ability to write, a stroke for instance, and in such cases the ability to replicate your own handwriting (training the model using pre-disability examples), may mean a lot. There is also all the trivial/potentially profitable stuff. The most prominent example is that gift/flower/card companies always include this horrible printed message. Our system would allow much greater personalisation, either with your own handwriting or with the handwriting of a celebrity (who would presumably get a royalty! The world just keeps getting stranger...). It also allows us to rewrite history, as demonstrated above, so I'm sure that the Ministry of Truth will be very interested;-)

Code will be published, but is not online yet (Well, the mean shift module is.). This is the largest, most complex system I have ever created, and its going to take months to get it all cleaned up and online. I will be presenting this paper at SIGGRAPH 2016, so am aiming to get it all done by then. We will also be releasing our dataset - that will happen much quicker, but there is little value until we have also released code to load the included files.

All Posts