How an AI organisation can make a difference during the pandemic

Making premises safe during COVID-19 with IRIS Computer Vision

Akash James
7 min readJun 4, 2021

An induction to or summation of the pandemic and its effect all over the world would be superfluous. Most of us have gone through a tidal shift — we have started working from our homes, children have shifted to online education and we’re more reliant on online services. Technology in various verticals has helped curb the magnitude of problems people around the world faced. Business meetings are now video calls, your biweekly visit to the grocery store is now conveniently delivered to your doorstep and let’s not forget your increased OTT consumption. The biggest change arguably has to be adopting new norms like social distancing, adhering to wearing masks and frequent sanitisation. Enforcing these to be compliant was imminent to re-establish services at facilities where human interaction is inevitable. The recent turn of events was an indication to us at Integration Wizards to deliver, helping humanity get back on its feet.

People photo created by freepik

IRIS, our flagship product is a Computer Vision platform that delivers state-of-the-art insights and analytics by using CCTV cameras. The need of the hour was to build modules into IRIS that can help make facilities and establishments alike COVID-19 compliant. This lead to the development of, what we call — Mask validation and Social Distancing validation. IRIS helped several malls, production and manufacturing facilities, and retail stores restore daily services.

The modules

Building computer vision modules that would help in reporting and analysing human activities was our primary focus. Checking for masks, social distancing, footfall and occupational density would help cater to our goal. Detecting and tracking people were common in both modules. We utilise neural networks for object detection to get the highest possible accuracy levels.

Mask Validation

Face Mask Classification | Illustration by Integration Wizards

The end goal here was to classify the people who were wearing masks from the ones who weren’t. The two challenges we encountered were the detection of small and occluded faces and accounting for makeshift homemade masks. RetinaFace fine-tuned on masked faces to increase detection accuracy was a good choice — it boasts high accuracies on face datasets and has a low power consumption of 2 gigaflops. Accounting for homemade, surgical and commercially available masks in a short time was a gargantuan task. Thanks to advances in computer vision, creating synthetic data for image classification accelerated our development time. Twenty iterations of ResNet50, half a million images of real and synthetic images and some engineering magic helped build a real-world mask classifier that had an accuracy of 95%.

Social Distancing

Social Distancing | Illustration by Integration Wizards

Cameras are mono-vision, i.e they do not generate depth information. Building an accurate social distancing validation algorithm needs to take depth into account to avoid false positives of people that may be further on the z-axis but relatively close on the x and y axes. Installing stereo-vision cameras for this is not economical. The CV team developed a proprietary algorithm to map a 3-D plane to a 2-D image. This algorithm accounts for the z-axis based on the size and position of person detections. The algorithm also charts out distances between all people visible and identifies groups with distance infringements.

Footfall and Density Analysis

Footfall Analysis | Illustration by Integration Wizards

Footfall analysis has been one of the first offerings bundled in IRIS. Our version of footfall works differently and is not limited to a person crossing a pixel-line. With IRIS deployed on all cameras pointed at entry/exits of a premise, we can determine the occupational density. This helps security personnel know that safety standards are being followed — without exceeding the maximum capacity of floor space.

IRIS in action

Here’s a sneak peek at IRIS’ inference on some publicly available videos.

IRIS AI in action | GIF by Author

Scaling on multiple cameras

A game of scales | Image by Author

Each of our customers had a vastly varying number of cameras at their premises. While most had a nominal number like 25, some had a much larger coverage involving over 200 cameras. IRIS is architected to be scalable by leveraging several acceleration libraries. Focused to be scalable both horizontally and vertically, IRIS can run on embedded devices and GPUs alike.

Economical feasibility comes with cost-effective hardware processing more information. IRIS runs multiple camera streams in parallel, performing data decode on GPUs with high throughput. By running the entire pipeline on a GPU, we avoid data transfer between the CPU and GPU scope, thereby eliminating the CPU bottleneck.

Accelerating neural networks for low-latency performance using NVIDIA TensorRT | Image by Author

At Integration Wizards, we specialise in making neural networks accurate with a lower number of parameters and a relatively lower flops rating. Combine this with NVIDIA’s TensorRT and quantisation, we get a potent mix of highly accelerated neural networks giving us performance well over 1000 fps on a single GPU. That’s not where it ends. IRIS capitalises on GPU horsepower by running the entire inference pipeline on the GPU. That includes not just inference but decoding the input frames, accelerating logic blocks and image manipulation. We pull this off with a variety of tools like NVIDIA’s GStreamer plugins and CUDA kernels to accelerate every last bit of the application.

The IRIS Experience

IRIS is a comprehensive solution, providing its users with data on the go. Data generated by edge devices or cloud engines are processed and shown meaningfully. Several billion data points are generated per day across our customers. This data is encapsulated into meaningful Key Point Index metrics for our customers to derive insight and make business optimisations.

Phoenix Marketcity mockup IRIS dashboard | Illustration by Integration Wizards

Everyone loves using smartphone applications! It’s only logical to bring the power of our AI platform in the palm of your hands — monitor premises, acknowledge alarms, view live streams and view summaries of your chain of operations.

IRIS native app support | Image by Integration Wizards

Customers Satisfied

The pandemic caught us off guard, yet we managed to design, develop and deploy a solution in a matter of weeks. Several customers benefit from IRIS, making them COVID ready in a matter of days.

One of the largest steel manufacturers of the world had to resume services and deploying IRIS helped them enforce the new norms, and within weeks they saw a drop in the violation trend.

With IRIS actively monitoring and logging infringements, employees were more compliant to follow protocols and these graphs show you how violation reduced drastically over a period of time.

Reduction in Mask Violations | Image by Integration Wizards
Reduction in Social Distancing Violations | Image by Integration

They see over five thousand check-ins into the facility during a single shift.

Flipkart and Myntra, being one of our premium customers, utilise IRIS in their sorting facilities to ensure workers are being compliant. Being vital to the age of the internet, these organisations had to keep their employees safe to cater to millions of souls with their door-to-door deliveries.

Malls — Several businesses were beginning to resume once the lockdown rules were relaxed. Phoenix chain of market cities, Inorbit mall, Viviana mall and Infiniti mall use IRIS to monitor footfall count and floor density to ensure optimal numbers are maintained, to reduce the risk of spread of the virus.

SpotOn uses IRIS to make its premise safe by utilizing Health and Safety and COVID modules.

One of the largest liquor manufacturers on this planet trusts IRIS to maintain compliance.

Reduction in violations by over 50% | Image by Integration Wizards

Britannia makes sure their employees follow compliance, helping them resume food and beverage manufacturing services.

The takeaway

The engineers at Integration Wizards believe IRIS can make a disruption in low-cost and affordable AI solutions. With an ever-increasing library of supported neural networks and use-cases, IRIS aims to cater for a wide variety of customers spanning across different verticals. Interested in what we’re going to do next? IRIS 2.0 will bring a no-code platform, enabling customers to deploy complex Computer Vision solutions, all with a couple of clicks. Get in touch to find out more!

IRIS 2.0 | Illustration by Integration Wizards

--

--

Akash James
Akash James

Written by Akash James

Artificial Intelligence Architect at SparkCognition | NVIDIA Jetson AI Ambassador & Specialist | Speaker | Hackathon Enthusiast

No responses yet