Chieh-Jan Mike Liang, Nicholas D. Lane, Niels Brouwers, Li Zhang, Börje F. Karlsson, Hao Liu, Yan Liu, Jun Tang, Xiang Shan, Ranveer Chandra, and Feng Zhao
Scalable and comprehensive testing of mobile apps is extremely challenging. Every test input needs to be run with a variety of contexts, such as: device heterogeneity, wireless network speeds, locations, and unpredictable sensor inputs. The range of values for each context, e.g. location, can be very large. In this paper we present Caiipa, a cloud service for testing apps over an expanded mobile context space in a scalable way. It incorporates key techniques to make app testing more tractable, including a context test space prioritizer to quickly discover failure scenarios for each app. We have implemented Caiipa on a cluster of VMs and real devices that can each emulate various combinations of contexts for tablet and phone apps. We evaluate Caiipa by testing 265 commercially available mobile apps based on a comprehensive library of real-world conditions. Our results show that Caiipa leads to improvements of 11.1x and 8.4x in the number of crashes and performance bugs discovered compared to conventional UI-based automation (i.e., monkey-testing).
|Published in||MobiCom (International Conference on Mobile Computing and Networking)|
|Publisher||ACM – Association for Computing Machinery|
© ACM. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version can be found at http://dl.acm.org.
Chieh-Jan Mike Liang, Nic Lane, Niels Brouwers, Li Zhang, Börje F. Karlsson, Hao Liu, Yan Liu, Jun Tang, Xiang Shan, Ranveer Chandra, and Feng Zhao. Context Virtualizer: A Cloud Service for Automated Large-scale Mobile App Testing under Real-World Conditions, Microsoft Research, September 2013.
Chieh-Jan Mike Liang, Nicholas D. Lane, Niels Brouwers, Li Zhang, Börje F. Karlsson, Ranveer Chandra, and Feng Zhao. Contextual Fuzzing: Automated Mobile App Testing Under Dynamic Device and Environment Conditions, Microsoft Research, March 2013.
Ranveer Chandra, Börje F. Karlsson, Nicholas D. Lane, Chieh-Jan Mike Liang, Suman Nath, Jitu Padhye, Lenin Ravindranath Sivalingam, and Feng Zhao. Towards Scalable Automated Mobile App Testing, Microsoft Research, January 2014.