When we think of enhancing vision, we tend to think in terms we can really get a grasp on:
- Zoom in (enhance!)
- night vision
- microscopy (really zoom in)
- long exposure (astronomy)
- timelaspse (watch that glacier hussle)
- slow motion (mythbusters finales)
My favourite development in computer vision during SIGGRAPH 2012 was called Eulerian Motion Enhancement of Video.
Essentially, it amplifies motion in video. You have to see it to really get a grasp on what it means, so here is the video. Check out the throbbing arm artery!
The secret here is to look at changes in the video from moment to moment, just like when you flash between two photos you took at a party and can see the differences easily. The algorithm tracks the differences between many frames over time noting the differences only.
But then the clever bit – what if you only pay attention to certain rates of change and ignore the rest? For example what if you had video of two pendulums swinging next to each other – one short and fast, the other long and slow – using Eulerian methods, you could ignore the fast motions or ignore the slow motions, much like a graphic equalizer in audio can isolate bass and treble frequencies in your music. You could effectively filter out either the slow or the fast pendulum depending on your ‘equalizer’ setting.
Once you’ve isolated the motions you want to enhance, you add the resulting difference image frame back into the video. If you really want to enhance it you do this several times in a row using a feedback loop. The more times you feed back the difference, the more that specific motion in the video gets exaggerated compared to other motions in the video.
This works best for periodic motions, like a pulse or guitar string or heartbeat, but enhancing specific motions is pretty awesome. In fact, you do it all the time without really being aware.
How? Well, try waving your hand at the very edge of your vision holding while up 2 fingers. Can you notice the motion – YEP. But can you count the fingers? NOPE. The edges of your vision are tuned to detect motion changes, not detail, while the centre portion of your vision is tuned for detail. You’re not going to read a book out of the corner of your eye, but you’ll definitely notice the sabre toothed tiger coming at you.
Of course, you have to watch the video, still images don’t really catch the effect!
Jufa 