Alvin Shek (ashek), Ellis Brown (elbrown), David Noursi (dnoursi), Nilay Pande (nmpande)
https://github.com/ellisbrown/representation-exploration
https://zippy-oatmeal-4b5.notion.site/Self-Supervised-Representation-Learning-via-Curiosity-Driven-Exploration-0cff22673f244cfeb6a6c3c8a0d0af37
“The performance of machine learning methods is heavily dependent on the choice of data representation” — Bengio et al, 2012
As machine learning continues to be applied to more complex and important tasks, this dependence on the data representation will only increase. While current machine learning methods are bottlenecked by representation quality, current methods for learning representations are bottlenecked on the dataset size. But this process of creating large static datasets, as is the mainstream practice [12], is expensive, time consuming, and heavily prone to human bias.
Machine learning practitioners have increasingly been focusing on paradigms such as unsupervised and self-supervised learning to help alleviate the expense of supervision in working with bigger datasets; however, these methods still suffer from the issues of static datasets. One promising approach to learn good representations without a fixed datasets is by directly interacting with the environment. The visual state space of real environments/simulators can be quite huge and intractable to explore fully. Hence, in this project, we investigate intelligent curiosity driven exploration strategies to learn good representations from a simulator using self supervised learning objectives. We discuss the effectiveness of different strategies, issues and future directions of research in this field.
A good representation is able to capture the distribution of the underlying causal parameters for any input. A good representation enables a supervised machine learning algorithm to (1) converge quickly on a downstream task, (2) transfer to new tasks/environments/modalities easily, (3) cluster data in a meaningful way and (4) capture the rich features of the data compactly and efficiently. This is summarized by the expression “the success of machine learning algorithms depends largely on the choice of data representation” [7].
Representation learning has achieved most success where a labeled dataset of images is available for machine learning. However, achieving a good representation of the data with self-supervised objectives requires an even larger set of unlabelled images. As tasks get more challenging, we need larger real world datasets to learn good underlying representations. Such large, curated datasets are a) expensive and time consuming to build and b) are not representative of the real world. Biology teaches us that real world biological entities learn directly from interactions with nature or the real world. Moreover, we have seen in practice that actions in the real world/simulator provide a rich source of self-supervision [1]. Hence, to make progress in AI, we need to investigate self supervised representation learning by directly interacting with world/simulators.
Given the large visual state space of the real world, random exploration will fail to cover the full distribution of images. Samples will be essentially drawn from a normal distribution centered around the initial state. This project is an attempt to intelligently explore the world in order to learn rich visual representations.
We explore relevant prior work on curiosity driven exploration and self-supervised representation learning.