What are foundation models, and how can they help implement AI at scale?

Semi-supervised learning models are trained on a dataset that contains a mixture of labeled and unlabeled data. The goal is to use the labeled data to improve the model’s performance on the unlabeled data. AI experts turn to semi-supervised learning when training data is difficult to obtain or would cost your company an arm and a leg. This, for instance, may happen in medical settings where various healthcare IT regulations are enacted. Some common examples of semi-supervised models include pre-trained text document and web content classification algorithms.

Unsupervised learning models are fully trained on unlabeled datasets. They discover patterns in training data or structurize it on their own. Such models, among other things, could segment information into clusters based on the parameters they’ve uncovered in a training dataset. ML engineers turn to auto-encoders, K-Means, hierarchical clustering, and other techniques to create unsupervised machine learning solutions and improve their accuracy.

Reinforcement learning models interact with their environment without specific training. When achieving a desired outcome — i.e., making a prediction the developers have hoped for — the models get rewarded. On the contrary, reinforcement learning models are penalized when making wrongful assumptions. The approach allows AI algorithms to make more complex decisions than their supervised and semi-supervised counterparts. An example of reinforcement learning in action would be autonomous vehicles or game-playing artificial intelligence like AlphaGo.

Generative AI models produce new data similar to the data they’ve been trained on. This data may include text, images, audio clips, and videos. The ChatGPT solution mentioned in the previous section belongs to this category of foundation AI models. Other examples of generative AI solutions include the DALL-E 2 tool, which creates images based on descriptions written in natural language, and the Synthesia.io video platform, which uses text-based inputs to produce video content.

Transfer learning models can solve tasks other than they’ve been trained on. For instance, computer vision engineers may leverage pre-trained image classification algorithms for object detection. Or harness existing NLP solutions for more knowledge-intensive tasks, such as customer sentiment analysis. Some popular pre-trained machine learning solutions include OpenCV, a computer vision library containing robust models for object classification and image detection, and Hugging Face’s Transformers library offerings, such as generative pre-trained transformer (GPT) — i.e., a rich language model whose third generation (GPT-3) powers the ChatGPT service.

Meta-learning models, unlike their task-orientated equivalents, literally learn to learn (no pun intended). Instead of devouring data to solve a specific problem, such models develop general strategies for problem-solving. This way, meta-learning solutions can easily adapt to new challenges while using their resources, such as memory and computing power, more efficiently. ML experts tap into meta-learning when training data is scarce, or a company lacks definitive plans regarding AI implementation in business. TensorFlow, PyTorch, and other open-source machine learning libraries and frameworks offer tools that allow developers to explore meta-learning techniques. And cloud computing providers like Google help ML experts and newbies train custom machine learning models using AutoML.

Foundation models will help you implement AI faster, cheaper, and with fewer resources involved. Creating and deploying an AI solution requires considerable time and resources. For every new application, you need a separate well-labeled data set. And if you don’t have it, you’ll need a team of data experts to find, cleanse, and label that information. According to Dakshi Agrawal, CTO of IBM AI, foundation models help cut down on data labeling requirements by 10-200 times depending on a given use case, which translates into significant cost savings. On the business side, you should also consider the rising cloud computing expenses. Google, for instance, spent as much as $35 million to teach DeepMind to play Go. And while your AI project may not be half as ambitious, you could easily spend $300,000 in cloud server costs alone to get your AI app up and running. Another reason to use foundation models, such as generative AI solutions, is the opportunity to quickly prototype and test different concepts without investing heavily in R&D.

You can reuse foundation AI models to create different applications. As their name implies, AI foundation models can serve as a basis for multiple AI applications. Think about driving a car. Once you’ve got a driver’s license, you don’t need to pass the exam every time you buy another vehicle. Similarly, you can use a smaller amount of labeled data to train a general-purpose foundation model that summarizes texts to process domain-specific content. And foundation models possess “emergence” capabilities, too, which means that a model, once trained, may learn to solve problems it was not supposed to address or glean unexpected insights from training data.

Foundation AI models help achieve your company’s sustainability goals. Training one large machine learning model can have the same environmental footprint as running five cars over their lifetime. Such a heavy carbon footprint stands in sharp contrast with the fact that 66% and 49% of businesses are increasing the efficiency of energy use and developing new climate-friendly services and products, respectively. With foundation AI models, you can train intelligent algorithms faster and utilize computing resources wisely — not the least thanks to the models’ architecture that takes advantage of hardware parallelism, executing several tasks simultaneously.