Users are increasingly relying on smartphones for computational tasks, hence concerns such as app correctness, performance, and security become increasingly pressing. Dynamic analysis is an attractive approach for tackling such concerns via profiling and monitoring. We first present two main infrastructural components developed in our group that permit dynamic analysis of apps, including dozens of top-100 apps, running directly on smartphones and without requiring access to the app's source code: the A3E systematic explorer and the RERAN record-and-replay tool. A3E combines static analysis with dynamic exploration to allow substantial apps to be explored systematically. RERAN allows app executions to be recorded and replayed very effectively by intercepting and replaying the stream of sensor data coming into the app. Next, we present several applications of our infrastructure: a multi-level (sensors, OS, network) app profiling study which has revealed interesting findings about app network traffic, as well as differences between paid apps and their free counterparts; and strategies for app testing and debugging that have helped us find and reproduce bugs in popular apps.