Top computer vision applications in different industries right now

Growth in visual data driven by spreading mobile technology that allows us to create billions of images a day

Increasingly affordable computing power to process this massive amount of visual data

Innovative hardware

Advancements in deep learning algorithms

Object classification that assigns objects in an image to a predefined class. With binary classification, the system answers a simple question: whether this object belongs to, say, a class of apples, tourist attractions, or cats.

Object localization that locates an object in an image by enclosing it into a bounding box.

Object detection that does both of the above for many objects in an image, assigning labels and localizing the objects by drawing bounding boxes around them.

Semantic segmentation that understands every pixel of an image and associates it with a class label (a car, a person, etc.) by creating object masks, with objects of the same class presented as a single entity.

Instance segmentation that does semantic segmentation and additionally identifies different instances of the same class so that you don’t get just a one-color mask for three parked cars in a street view photo but label the vehicles with three different colors, identifying their boundaries.

Minimizing human interactions in stores to boost safety, deliver a digital shopping experience, and lower labor costs

Personalizing customer experience for increased engagement and more effective upselling and cross-selling strategies

Leveraging next-gen in-store analytics to prevent stockouts, enhance store layout designs, and optimize staff scheduling

Automated checkouts and cashierless stores combine computer vision technologies with shelf sensors and deep learning to recognize shoppers, detect items they place into their carts, and automatically charge them on their accounts for items bought when they leave the store. Apart from the famous Amazon Go example, Chinese giants Lenovo, JD, and Alibaba have also rolled out automated stores using sophisticated computer vision technology. In the startup space, AI provider Tiliter’s self-scanning scale can automatically identify fresh produce, allowing customers to check out without the need for barcodes or packaging.

In-store navigation systems, based on computer vision software, can find the most efficient route to products on the customer’s in-app shopping list and recalculate the route if the person decides to look at other items. For example, several years ago, home improvement chain Lowe’s launched an indoor mapping app developed with Google. Combining computer vision technologies and augmented reality software, the app reportedly locates users in an aisle without active Wi-Fi or GPS required.

Product information display apps feature Scandit’s mobile barcode scanning software based on computer vision that recognizes objects for shoppers to get a personalized offer or information on the product in the store.

Visual search solutions that leverage deep learning algorithms make it more convenient and easier for online shoppers to discover products, returning visually similar results. eBay was among the first to offer its customers this engaging shopping experience.

Personal recommendations are enabled by an array of computer vision systems. Among them are virtual mirrors that also use augmented reality, allowing shoppers to try on various clothes virtually (like Amazon) or experiment with makeup products (like Sephora’s app or the Bourjois Magic Mirror). In-store virtual dressing room technology, embedded in Findmine’s Complete the Look solution, offers shoppers a touchscreen display with a CV-powered camera that recognizes items they are wearing to create an outfit based on images from the retailer’s catalog. Ecommerce sites are eyeing computer vision-enabled body scanning technology like the Bodygram service to provide size or clothing recommendations. Neutrogena’s Skin360 app invites customers to measure their skin health by assessing facial attributes and skin pixels and recommends a skin care routine. The Lolli & Pops candy store uses facial recognition to identify regular shoppers and offer them personalized product recommendations or loyalty discounts.

Inspection systems relying on computer vision technologies help retailers improve inventory management. Examples range from the Shelfie technology that uses shelf-mounted computer vision cameras to alert staff about out-of-stock or incorrectly displayed items to the mobile robot Tally that not only notifies shop assistants of gaps on shelves but also can detect damaged packaging and inaccurate pricing.

Connected stores use computer vision-powered cameras to analyze customer journeys and get product-related insights. Among them are Samsung’s famous pop-up store infused with capabilities to analyze dwell time, demographics, and other customer data, and Serbian fashion retailer Legend World Wide’s store built in collaboration with Deloitte to gauge customer movement heatmaps.

Understanding students’ learning behaviors to drive personalization and improve learning experiences

Automating classroom monitoring to deter cheating in tests

Assessing students’ papers to reduce the burden on educators

Student engagement detection and personalization. With computer vision systems, educators can measure students’ mood and behavior to capture signs of engagement or distraction both online and in a classroom. Solutions in this space include Emotuit, which uses facial recognition to analyze students’ emotional responses to online content, and the Little Dragon learning app that also reads facial expressions to detect frustration or boredom and adapt learning content. In addition, computer vision-driven insights help teachers regroup students into a more comfortable environment to improve learning.

Attendance monitoring and automated online proctoring have been made easy with computer vision-powered webcams that are used to identify students and flag cheating behaviors by tracking their postures or eye movements. Examples include UAuto with multi-factor authentication launched by the ProctorU proctoring service and automated proctoring solutions like Respondus Monitor and Mettl.

Handwritten character recognition is an area where computer vision is also expected to shine, as advanced algorithms are able to not only recognize responses written by students but also assist with their automatic evaluation.

Improving patient identification to prevent wrong person procedures

Delivering a more accurate diagnosis through medical imaging analysis

Providing assistance in surgery training and real-world surgeries for better outcomes

Delivering rehabilitation assistance to patients

Patient identification systems use computer vision-based cameras that help improve facial authentication of patients from check-in to discharge to prevent wrong person procedures.

Medical image analysis assisted by computer vision technology is transforming radiology, helping practitioners interpret X-rays, CT scans, MRIs, and even microscopic images of cellular structures more accurately when diagnosing breast, brain, lung, or skin cancer. Computer vision applications in medical imaging also feature solutions to estimate a human pose in analyzing symptoms of neurological and neuropsychiatric disorders, monitor blood loss to optimize blood transfusions, diagnose eye conditions, and even detect COVID-19 (a deep convolutional neural network called COVID-Net has shown 90% accuracy in diagnosing COVID-19 based on chest X-ray images).

Surgical simulationhttps://itrexgroup.com/blog/top-generative-ai-in-healthcare-use-cases/ and assistance leverages computer vision technology to increase surgical precision. Apart from assistant surgical robot systems, there are solutions like Proprio Vision that combine computer vision with ML and VR to create 3D visualizations for surgeons in the operating room. In the surgical training space, Touch Surgery is a famous mobile simulator that provides a detailed guide to surgical procedures.

Rehabilitation applications feature computer vision systems that are being developed to supervise exercise routines as part of at-home rehabilitation for rheumatoid arthritis, sports injury, brain injury, or stroke.

Capturing performance data to aid coaches in training sessions and athletes in self-training

Introducing advanced player or ball tracking methods to improve the viewing experience or help referees in decision-making

Collecting performance statistics for scouts, sports betting sites, and other industry professionals

Tracking systems powered by computer vision-enabled cameras detect and track moving players or balls in an array of games such as soccer, tennis, baseball, or golf. Top examples include SentioScope, designed by Sentio for soccer player tracking and analysis, and the SportVU 2.0 optical tracking technology that gives football coaches a holistic view of matches. Computer vision systems also improve shooting accuracy in basketball training (Noah System), help swimmers refine their techniques by collecting data from stroke rates to real-time velocity and turn times (FINIS LaneVision), and can even take over in part the job of umpire in professional tennis matches (Hawk-Eye ball tracking solution).

Self-training solutions that are based on computer vision techniques like human pose estimation or motion tracking help users stay active at home and bring training to a more professional level. For instance, MotionMind AI developed by ITRex is capable of capturing the minutest of movements in striking 3D, not only tracking a user’s progress but also adjusting training intensity and planning ahead to maximize results.

In-depth data analytics platforms analyze actions of players on the ice or field, producing meaningful insights either for building better game strategies or making smart decisions on players in the scouting market or engaging viewers. Among famous examples are Sportlogiq that collects raw data from video feeds to produce game models and AutoSTATS that can analyze any recorded basketball game for performance improvement and scouting insights.

Identifying pests and weeds with greater accuracy to optimize the application of chemicals

Monitoring crop development and the environment to maximize yields and produce better quality according to rising customer expectations

Automating livestock management to prevent flock and herd losses and reduce the need for manual inspection

Pest and weed detection systems feature Blue River Technology’s See & Spray solution equipped with intelligent cameras that can distinguish between crops and weeds to apply herbicides to the right plants. Computer vision methods are also used in precision agriculture for automatic identification and counting of flying insects and even for identifying apple diseases early.

Observation, harvesting, and prediction systems are developed to detect the ripeness of fruit, including through color ratings of cherries in an outdoor environment, or pick vegetables in a greenhouse using robots. Computer vision-aided monitoring solutions include SolarXOne, a 100% autonomous solar-powered drone system from XSun that provides farmers with HD images capturing crop and soil conditions. AI solutions from Brazil’s Cromai can gauge information about the color, shape, and texture of crops. High-definition cameras from SWIR Vision Systems are equipped with sensors to help monitor soil moisture for yield prediction.

Livestock management systems using drone technology can perform automatic counts, detect sick or injured animals, find stray animals, spot grazed areas, and even move cattle. A fair example is the autonomous drone technology developed by Israeli firm BeeFree Agro to herd cattle.

Implementing automated quality control to increase manufacturing accuracy, improve productivity, and produce better quality

Deploying monitoring solutions to cut inspection time, minimize safety risks, improve operator productivity, and increase cost efficiency

Reduce human involvement to protect workers from hazardous environments

Next-level quality control is today enabled by intelligent computer vision-powered cameras directed at a manufacturing line. Examples include Pharma Packaging Systems’ machinery for the pharmaceutical industry to automatically count tablets or capsules on production lines and WebSPECTOR, a surface inspection system that identifies defects in items, stores images, and collects image-related metadata to classify errors by type and grade. Сomputer vision technology is also used to guide assembly operations, like assembly verification solutions from Acquire Automation that measure product components versus manufacturing specifications, check caps and fill levels, and verify packaging components.

Robot palletizing systems guided by machine vision automatically load or unload boxes and items to and from pallets.

Predictive maintenance systems assisted by computer vision technology and sensors have made it much easier to track the condition of critical infrastructure and determine when maintenance is needed. For example, FANUC’s Zero Down Time takes photos and collects metadata to uncover any potential problems in the machine. Oil and gas giants such as Shell, ExxonMobil, and BP are using computer vision-powered predictive maintenance to anticipate failures in their equipment.

Intelligent monitoring solutions, including drone-assisted systems, allow companies to conduct remote inspections of their sites and assets. This application of computer vision is especially important in mining, an unsafe industry for workers, where operators need to collect visual data in difficult areas. Visual inspections of well sites using the Osprey Reach system, for instance, have enabled operators to reduce routine site visits by half.

Warehouse management systems using computer vision technology reduce inventory times from hours to minutes, delivering huge savings on operational costs. Among such systems is the Gather AI platform that uses drones and connects to IoT devices for scanning and counting inventory. Another example is Amazon, which is rolling out the Pegasus robot technology at its sorting centers, claiming that the robot can improve sorting accuracy by 50%.

Mobile computer vision enables a contactless delivery process for retail, logistics, post, and parcel businesses, transforming smartphones and other smart devices into computer vision-enabled barcode scanners like solutions from Scandit. The frictionless way to pick items from warehouses, distribution centers, or retail rooms is growing in popularity. In retail, barcode scanning software is used for order fulfillment while allowing shoppers to safely collect items with one contactless mobile scan.

Safety monitoring solutions using computer vision help keep public spaces safe during the pandemic by detecting ill employees or students as well as monitoring social distancing or exposure times. At construction sites and on the manufacturing floor, computer vision systems like IRIS monitor behavior-based safety and can protect people working around hazardous zones by alerting machine operators of dangerous events or incidents.