Reducing the Data Demands of Smart Machines

Machine learning (ML) systems today learn by example, ingesting tons of data that has been individually labeled by human analysts to generate a desired output. As these systems have progressed, deep neural networks (DNN) have emerged as the state of the art in ML models. DNN are capable of powering tasks like machine translation and speech or object recognition with a much higher degree of accuracy. However, training DNN requires massive amounts of labeled data–typically 109 or 1010 training examples. The process of amassing and labeling this mountain of information is costly and time consuming.

Beyond the challenges of amassing labeled data, most ML models are brittle and prone to breaking when there are small changes in their operating environment. If changes occur in a room’s acoustics or a microphone’s sensors, for example, a speech recognition or speaker identification system may need to be retrained on an entirely new data set. Adapting or modifying a model can take almost as much time and energy as creating one from scratch.

Read more