Since I started my PhD, I’ve been wrestling with one major problem – what I want to do as a project is well understood mathematically but not very widely understood practically. Since the major thrust of my project is both inventing a new bit of mathematical theory but also practically implementing it, that has caused a problem; before we can get any work done on new things, we have to first reproduce the OLD things. That’s not so easy when there are no “instruction manuals” on how to do what we need to.
So for the past three months the main thrust of my work has been finding out how to implement this kind of algorithm called the Maximum Entropy Method (we call it MaxEnt) for cleaning up messy images. That’s a whole bunch easier than it sounds, because I’ve had to spend a lot of time understanding the nitty-gritty bits of how to make such an algorithm work in not three, not four, but somewhere around one million dimensions! Each pixel in the image you want to process represents one “dimension”, so if you have a megapixel image you suddenly have a very thorny problem! Getting MaxEnt to work at all then becomes a big challenge. So that’s more or less what I’ve spent my time doing – trying to get it to work.
Simple(ish) idea; horribly complicated thing to achieve. I’ve written thousands of lines of code and spent hours, days, debugging patiently. Finally, today, I got the MaxEnt method to work on a real set of data in about 1200 dimensions – it solved in something like a minute. That’s an absolutely massive milestone for me, since it means I can start doing the real exciting things that I has been hoping to start on day 1.
So for me, today was a major win. It works. Pain (almost) over. Now I can move onto more interesting – and more challenging – projects!