After training, what do we do with our models?
Models alone don’t have much value — it’s all in how you use them. Whether that’s to drive decisions within your business, or to provide new features for your customers, the role of a serving framework is to bring your models to life.
With a model serving framework, you can
- Interact with a model via an API. Because of this, anything that talks to your model can do so without knowing any internal details such as which tools were used to train it or what language it’s written in.
- Deploy the model in the cloud alongside other components of your applications.
- Scale the model easily to meet user demand.
For a concrete example, suppose you run an online store, and you want each of your customers to see personalised product recommendations. There are lots of ways to train a model for this task, but assuming you’ve already done that part, the next challenge is in getting the website talking to it.
Even though the model might be complex, a model serving framework will hide that complexity, leaving us with a simple API so that, whenever we want a customer to see recommendations, all we need to do is query that API.