This page shows the up-to-date working progress of my research at the intersection of machine learning, robotics, and visualization.

Reinforcement learning (RL) is an effective class of decision making algorithms for a broad variety of applications. Modern vision-based RL techniques use convolutional neural networks (CNN) as universal function approximators to choose which action to take for a given visual input. Until recently, CNNs have been treated like "blackbox" functions, but this mindset is especially dangerous when using them for control in safety-critical settings.


Reinforcement Learning on Autonomous Vehicles

Machine Learning | 2017
Work in progress
In collaboration with Sam Green


This video demonstrates the use of deep reinforcement learning to train a drone to reach a designated spot based on purely visual inputs. The freedom of control is two-degree, namely, forward and backward.

It also shows the drone’s view (visual inputs) on the upper right, and a data visualization of the outputs on the lower right. The blue area indicates forward probabilities, yellow area indicates backward probabilities, and the moving bar shows the actual action at each step. As the visualization shows, the blue area slowly takes over the place through the five episodes and the moving bar becomes more stable.