Smartphones Will Get Even Smarter With On-Device Machine Learning

Engineers are on the cusp of on-device machine learning, as evidenced by the first NIPS workshop on the subject in late 2017, and the advent of new neural processors, such as Kirin 970 from Huawei and Snapdragon 845 from Qualcomm.

Thus far, progress in artificial intelligence has been fueled primarily by the availability of data and more computing power. Classical machine learning has mostly been built on a single central node (usually in a data center) with full access to a global dataset and a massive amount of storage and computing power. Currently, many deep learning algorithms reside in the cloud, enabled by popular toolkits such as Caffe and TensorFlow, as well as specialized hardware such as tensor processing units.

But this centralized approach won’t work for things and applications that require low latency, such as flying a drone, controlling a self-driving car, or sending instructions to a robotic surgeon. To perform these delicate tasks, and other activities experts can’t yet anticipate, future wireless systems will need to make even more decisions at the network edge (closer to devices), more quickly and more reliably, even when they lose connectivity.

This realization has sparked a huge interest in distributed machine learning, a new paradigm in which training data that describes a problem is stored across a very large number of nodes, which work together to find a solution.

On-device machine learning (or on-device AI) is similar—it’s essentially about training a high-quality centralized model in a decentralized manner. Training data is unevenly distributed and every device has access to a tiny fraction of it.

There are clear advantages to doing it this way: Unlike cloud-based artificial intelligence, on-device AI should preserve privacy because training data is not logged in the cloud, but kept locally on each device. Training is also done locally and updates are aggregated and shared with peers over wireless links or via a cloud server. That way, all devices have access to the same global model.

Still, there are several challenges that engineers and researchers must solve to bring the capabilities of on-device machine learning to the masses. To guarantee that privacy is preserved no matter what, researchers need to incorporate differential privacy, whose aim is not to reveal whether a certain data point was used during training.

They must also incorporate techniques such as federated learning and transfer learning when training data is sparse. Here, instead of learning from scratch, the algorithms learn a model in a rich data source domain and transfer that knowledge to a target domain, as an efficient way to tame the cold-start problem.

Moreover, since devices have limited resources, on-device machine learning must optimize the model running on the device (tweaking the number of layers, the number of neurons per layer, and other parameters) and power usage, while also considering prediction accuracy and privacy constraints.  [READ MORE]