Intelligence, Robots and Big Data

What do Wal-Mart, Google and Lady Gaga all have in common? They are all fans of Big Data.

I didn’t know that until today, when Prof. Ping Jiang, who has just joined the department, gave his inaugural lecture. Prof. Jiang was talking about properly large amounts of content. Google create around 25 Petabytes of data every day (that’s a  1 with fifteen zeroes after it). Wal-Mart are registering over 1 million customer transactions an hour. And Lady Gaga (or more probably her manager Troy Carter) are getting input from 31 million Twitter followers and 51 million Facebook fans when they consider what to do next.

Big Data holds useful nuggets of information and lets you do do lots of powerful things. But the problem with big data is that it is, well, er, big. And we are not talking about sheer size here, we also need to consider the rate at which we are adding to the data, and the speed that we want to get useful things from the raw numbers.

It seems that the best way to decide when you are dealing with big data is when conventional techniques break down. If it would take your network of servers several hundred years to deliver the result of one query on your data set, then you are dealing with big data. And the only way to really deal with this is to divide and conquer by spreading the processing around as much as you can, and doing the maximum amount of work you can when you first get the data in.

Prof. Jaing took as an example the problem of machine vision, in the context of robots that can navigate around autonomously. This is a complex problem, with huge amounts of data coming in from the robot’s visual sensors alone. An intelligent robot would need to be very intelligent indeed just to be able to find its way from one office to another.

But if you spread the vision sensors around the building, getting them to perform all the motion and object tacking, you can reduce the intelligence that you need in the robot itself and lose a lot of complexity. Your robot can move a lot more confidently, as the systems controlling it can “see” much further ahead and react to changes in the environment. You are dealing with the big data coming into your system by processing the raw information as it arrives and converting it into a useful form that could be shared by all the devices navigating in an area.

It’s early days, but it did look to me like this did hold the prospect of actually having useful robots working with us.

Fascinating stuff.

Robotics Talk on Monday 16th

Don’t forget that on Monday 16th at 6:00 Prof. Ping Jiang is giving his inaugural lecture in the Middleton Hall at the University. It is on the subject of robotics and intelligence. Looks like fun, there might even be some robots in attendance…

Shelf Portrait

I’m not sure that I’ve actually got the hang of this whole “shelfie” thing.

Morning Light

You get some lovely colours during the sunrise at this time of year..

I took these with the Lumia 1020 phone. I’m needing to get my camera out less and less these days.

Fiddling with Hardware at C4DI

Jon Moss gets things started.

We had a great evening at the C4DI tonight. It was another of the “You Really Should Be” events and just to make it even more interesting I was giving one of the presentations. But first we had Jon Moss, making the point that “You Really Should Be speaking at events”. He made a great case for talking in public, not least because it is fun once you get going. He also made the very good point that the aim of a presentation is to get your audience to do something and what this is should be set out and reinforced at the beginning and the end of your presentation, when the audience is most engaged.

My Audience. Thanks for the applause.

Then it was my turn. My topic was “You Really Should be Fiddling with Hardware”, and I’d brought along some hardware that I’d fiddled with, including coloured lights that can be controlled over Bluetooth from my Windows Phone. Everything worked, eventually, and folks clapped at the end, which is always nice. You can find the slides from the presentation here.

Next up was John Polling who reckoned “You really should be working on a side project”. By side project John means something which takes you into new areas and helps you develop your skills. I like the sound of this. I’ve always found that the technical things that I’m not really supposed to be doing are the ones that I find the most interesting at any given time. By formalising this and giving an outlet for folks who want to make something different you can get happier developers and some surprisingly useful outcomes.

Finally we had Steve Fewster. who rounded things off with “You really should be - developing apps for the ‘Global Cloud Accounting’ community”. I’m not that familiar with accounting, but I do know about the cloud and how lots of business process is now moving into it. Steve, who is in the business of providing tools and services that add value for cloud users, took the line that “There’s gold in them thar hills”. In other words, the market is just getting going for people who want to bolt their ideas onto cloud based accounting systems. The system makers are keen to encourage an app infrastructure and so they make it easy to create apps and there is lots of scope for neat ideas to become the next big thing.

All in all, a very thought provoking and enjoyable evening. Thanks to C4DI for putting it together.