Energy Characterization and Optimization of Image Sensing Toward Continuous Mobile Vision

ACM International Conference in Mobile Systems, Applications, and Services (MobiSys) |

A major hurdle to frequently performing mobile computer vision tasks is the high power consumption of image sensing. In this work, we report the first publicly known experimental and analytical characterization of CMOS image sensors. We find that modern image sensors are not energy-proportional: energy per pixel is in fact inversely proportional to frame rate and resolution of image capture, and thus image sensor systems fail to provide an important principle of energy-aware system design: trading quality for energy efficiency.

We reveal two energy-proportional mechanisms, supported by current image sensors but unused by mobile systems: (i) using an optimal clock frequency reduces the power up to 50% or 30% for low-quality single frame (photo) and sequential frame (video) capturing, respectively; (ii) by entering low-power standby mode between frames, an image sensor achieves almost constant energy per pixel for video capture at low frame rates, resulting in an additional 40% power reduction. We also propose architectural modifications to the image sensor that would further improve operational efficiency. Finally, we use computer vision benchmarks to show the performance and efficiency tradeoffs that can be achieved with existing image sensors. For image registration, a key primitive for image mosaicking and depth estimation, we can achieve a 96% success rate at 3 FPS and 0.1 MP resolution. At these quality metrics, an optimal clock frequency reduces image sensor power consumption by 36% and aggressive standby mode reduces power consumption

Glimpse

We see a new wave of innovations in mobile computing that are based on capturing and processing a user’s visual experience in real-time, on the go. While this is cool, several major hurdles need to be overcome first. For example, today the energy and computing cost of continuously capturing, processing and analyzing our visual experience is challenging. In Project Glimpse we are investigating and pursuing experimental and analytical characterization of image sensors, high bandwidth and low-jitter networking and novel cloud computing techniques that enable online reactive analysis. This video shows one of the scenarios we are building to showcase our research results.