Developers
June 18, 2020

Neuropod, Uber's Deep Learning Engine

Forget about integrations and migrations. One interface for all DL frameworks

Today we will talk about a service that works within ATG, the Uber Advanced Technologies Group. This is the arm of the company focused on high-end technology development and implementation.  

Neuropod, the service we are going to talk about today, is a deep learning algorithm that works to provide self-driving technology. The use of deep learning aims to build and train models to process sensor inputs, to identify objects, and to predict the route of the indicated objects.  

Self-driving technology is always improving and is not only being developed by Uber. It has the eyes of Google and of Tesla on it. For a couple of years, this was one of the big projects of the majority of big companies. They know that this might be the future of transportation.

As the technology evolves, Uber looks for new ways to improve their models. For this, they try and test different frameworks that work with Deep Learning. There are currently many frameworks developing their inclusion with DL and some already work with it. TensorFlow and PyTorch are the most used ones.

Uber discarded adding support to the development of an entirely new DL framework. It's not worth it in an opportunity-cost paradigm. Uber introduces Neuropod, an open-source Deep Learning service. Neuropod provides a uniform and simple interface. This way, developers can simplify their production of their models.

Uber doesn't limit itself to the use of one and only one framework. They have used multiple Deep Learning frameworks in the past years. In 2016 they used Caffe2, and in 2017 they integrated into TensorFlow. Afterward, Uber migrates to Pytorch. ´

The integrations and migrations from one platform to the other were not easy. There was trouble with memory issues. After all those years of migrations and interrupted development, Uber finds its place and in 2018 Uber uses its own built framework, Ludwig.

Neuropod is the ultimate solution. It is an open-source library the helps Deep Learning frameworks look and feel the same at the time of running any model. Neuropod supports the most used DL frameworks. TensorFlow, PyTorch, Keras, and TorchScript.

The service was released in early 2019, and since then they have deployed hundreds of models across the entire ATG. The models include demand forecasting, Estimated time of arrival, prediction for rides, menu transcription for Uber Eats, Object Detection for self-driving vehicles, among others. As we can see, Neuropod not only works for internal purposes, but it works to develop better in terms of the business of Uber.

As we can see, the development of the Neuropod was not out of the blue. It had a purpose and its purpose was to solve a problem. What was the problem? Well, the migration from multiple frameworks, and the limitation of the errors and time taking issues of multi-framework development, was solved with a connector. This service allowed the Uber team to develop differently.

How does Neuropod Work?

Neuropod wraps all existing models and turns them into one package. Containing the initial and original model, and adding metadata and other data to it. 

Some of the models are heavyweight, meaning that they have large inputs and the model is even larger. For this problem, Uber has a solution. Instead of running the models on a CPU, they use remote GPUs.

If neuropod didn’t exist, any platform builds to make models would have to run Keras Remotely, TensorFlow remotely PyTorch remotely and TorchScript remotely. Besides this, all applications would need to run models of the frameworks in a local directory.

It may sound doable and simple, but here's the point: Local applications and remote applications have different implementations. Neuropod solves the entire problem. You can run the service remotely, and the service can run models on multiple frameworks simultaneously.

Uber looks forward to expanding Neuropod after its recent open-source release. They look forward to implementing new features and enhance current ones. They open-sourced their library to make it accessible to the community, therefore allowing contributions and improvements to it. Uber states: "Neuropod has been very useful across a variety of deep learning teams at Uber and we hope that it'll be beneficial to others, too."

In conclusion, Uber has developed over the past years, a service that allows the user to run models on multiple frameworks. Neuropod is focused on Deep Learning and helps Uber do their DL models. It helped the platform to develop some of its features such as Estimated time of arrival, UberEats menu transcription, Self-Driving technology, among others. Uber used many frameworks before getting to the conclusion of developing a service that would solve their migration and integration problems. As uber hopes the platform serves you, and you can serve the platform by becoming part of their community.

TagsUberDeep Learning
Lucas Bonder
Technical Writer
Lucas is an Entrepreneur, Web Developer, and Article Writer about Technology.

Related Articles

Back
DevelopersJune 18, 2020
Neuropod, Uber's Deep Learning Engine
Forget about integrations and migrations. One interface for all DL frameworks

Today we will talk about a service that works within ATG, the Uber Advanced Technologies Group. This is the arm of the company focused on high-end technology development and implementation.  

Neuropod, the service we are going to talk about today, is a deep learning algorithm that works to provide self-driving technology. The use of deep learning aims to build and train models to process sensor inputs, to identify objects, and to predict the route of the indicated objects.  

Self-driving technology is always improving and is not only being developed by Uber. It has the eyes of Google and of Tesla on it. For a couple of years, this was one of the big projects of the majority of big companies. They know that this might be the future of transportation.

As the technology evolves, Uber looks for new ways to improve their models. For this, they try and test different frameworks that work with Deep Learning. There are currently many frameworks developing their inclusion with DL and some already work with it. TensorFlow and PyTorch are the most used ones.

Uber discarded adding support to the development of an entirely new DL framework. It's not worth it in an opportunity-cost paradigm. Uber introduces Neuropod, an open-source Deep Learning service. Neuropod provides a uniform and simple interface. This way, developers can simplify their production of their models.

Uber doesn't limit itself to the use of one and only one framework. They have used multiple Deep Learning frameworks in the past years. In 2016 they used Caffe2, and in 2017 they integrated into TensorFlow. Afterward, Uber migrates to Pytorch. ´

The integrations and migrations from one platform to the other were not easy. There was trouble with memory issues. After all those years of migrations and interrupted development, Uber finds its place and in 2018 Uber uses its own built framework, Ludwig.

Neuropod is the ultimate solution. It is an open-source library the helps Deep Learning frameworks look and feel the same at the time of running any model. Neuropod supports the most used DL frameworks. TensorFlow, PyTorch, Keras, and TorchScript.

The service was released in early 2019, and since then they have deployed hundreds of models across the entire ATG. The models include demand forecasting, Estimated time of arrival, prediction for rides, menu transcription for Uber Eats, Object Detection for self-driving vehicles, among others. As we can see, Neuropod not only works for internal purposes, but it works to develop better in terms of the business of Uber.

As we can see, the development of the Neuropod was not out of the blue. It had a purpose and its purpose was to solve a problem. What was the problem? Well, the migration from multiple frameworks, and the limitation of the errors and time taking issues of multi-framework development, was solved with a connector. This service allowed the Uber team to develop differently.

How does Neuropod Work?

Neuropod wraps all existing models and turns them into one package. Containing the initial and original model, and adding metadata and other data to it. 

Some of the models are heavyweight, meaning that they have large inputs and the model is even larger. For this problem, Uber has a solution. Instead of running the models on a CPU, they use remote GPUs.

If neuropod didn’t exist, any platform builds to make models would have to run Keras Remotely, TensorFlow remotely PyTorch remotely and TorchScript remotely. Besides this, all applications would need to run models of the frameworks in a local directory.

It may sound doable and simple, but here's the point: Local applications and remote applications have different implementations. Neuropod solves the entire problem. You can run the service remotely, and the service can run models on multiple frameworks simultaneously.

Uber looks forward to expanding Neuropod after its recent open-source release. They look forward to implementing new features and enhance current ones. They open-sourced their library to make it accessible to the community, therefore allowing contributions and improvements to it. Uber states: "Neuropod has been very useful across a variety of deep learning teams at Uber and we hope that it'll be beneficial to others, too."

In conclusion, Uber has developed over the past years, a service that allows the user to run models on multiple frameworks. Neuropod is focused on Deep Learning and helps Uber do their DL models. It helped the platform to develop some of its features such as Estimated time of arrival, UberEats menu transcription, Self-Driving technology, among others. Uber used many frameworks before getting to the conclusion of developing a service that would solve their migration and integration problems. As uber hopes the platform serves you, and you can serve the platform by becoming part of their community.

Uber
Deep Learning
About the author
Lucas Bonder -Technical Writer
Lucas is an Entrepreneur, Web Developer, and Article Writer about Technology.

Related Articles