CLOBOTICS – An Interesting Company In AI+IIoT Space Won The Startup Showcase Judges Award At O’Reilly’s AI Conference This Week.
Clobotics focuses on capturing and analyze images for energy and retail. Its solutions combine computer vision, artificial intelligence/machine learning, and data analytics software with different hardware form factors, including autonomous drones, mobile applications and other IoT devices to help companies automate time-intensive operational processes, increase efficiencies and boost the bottom line.
The 80-person company headquartered in Shanghai & Bellevue (WA) was founded in 2016 by four AI researchers (ex-Microsoft) and Carnegie Mellon. In the mere 21 months since the company’s establishment, it has already filed 30 patents.
CLOBOTICS Caught Our Attention With Its Automated Drone-based Wind Turbine Inspection Technology. A Solution Space With Vast Potential To Impact O&M Costs For Large Electricity Providers And Utilities.
Global energy companies such as Enel, E.On, and AES are starting to invest in drones for tower and blade inspections. When inspecting wind turbines, the drone can be equipped with a digital camera, a thermographic camera or a combination, depending on the scope of the inspection task. A digital camera provides proof of the visual failures and damages of the tower, nacelle, rotor blades and bolt jointings.
Beginning of 2015, there were nearly 270,000 individual wind turbines operating globally, with more than 800,000 blades spinning on these turbines which are continually battered by the elements over time causing gradual wear out. Deterioration can cause reduced energy production in early stages and catastrophic and costly blade collapse if left unnoticed. With an increasing proportion of grid power coming from renewable power-sources (solar, wind, hydro, others) going forward, the type of solutions being developed by Clobotics are poised to play an important role in the space for a long time to come.
Windfarms have been deployed manually operated drones for inspection of their wind turbines for taking pictures for a while now, as a way to supplement their remote ground based inspection via telescopes. Under the control of human operators standing close to wind turbines, these individual drones will survey wind turbine blades, snapping photos that look for weather damage that can weaken the blades. Drones capture defects better than human eyes, however humans don’t have the capacity to process all of the data that wind-turbine-inspecting drones can capture.
Clobotics takes this to the next level with a solution that supports inspection automation for both both small/individual wind-turbine as well as large windfarm operators. Fleets of drones operating near autonomously use LiDar to create a 3-D pointcloud “map” of turbines that in turn creates a software driven inspection path for the drone around each individual turbine blade and machinery housing in a manner that fully utilizes this unique 3-D spatial data captured around the turbines. The software instructs the drone when and where to take a photo.
The drone then follows this uniquely calculated inspection path to take images of all four faces of each blade using multiple passes for each blade and the housing. The machines send the images back in real-time and Clobotics analyzes the pictures using machine learning to detect problems much faster than a human inspector could. The images captured by the drone are used to create an Orthomosaic – a detailed, accurate photo representation of an area, created out of many photos that have been stitched together and geometrically corrected (“orthorectified”) so that the scale is uniform, ie. the photo has the same lack of distortion as a map. Doing so alows an accurate representation of the surface, having been adjusted for topographic relief, lens distortion, and camera tilt.
The Clobotics’ solution covers data collection, damage identification, classification, and recommendations. By integrating customized UAV drone hardware with computer vision software, it automatically takes pictures of wind turbine blade surfaces looking for defects and damage from weather and wear, and then uses computer vision to inspect the images on the cloud. The system notifies maintenance personnel of damages, deterioration, and other early warning signs, which are critical for reducing the cost of maintenance. (In future we expect software could also re-direct the drone cameras to points of special interest and views of different angles of the tower, nacelle, rotor blades and bolt jointings. )
Automation offers the flexibility to conduct inspections on-demand rather and at set intervals. Every image is tagged with the data that customers need to locate, annotate, and make decisions about timing of blade repairs. Precise, accurate data location provides a clear benchmark and a digital timeline of damage progression.
Several Good Reasons To Automate Inspection Processes For Wind-Turbine Operators.
Pressure to reduce O&M costs for wind turbines. According to Alexander Kueppers, Investment Manager at Statkraft Ventures – “We consider automation and standardization of O&M-related tasks the next big challenge in cost competition after hardware costs for renewable energy assets have dropped significantly over the last years,”
Manual Inspections are time & resource intensive. With inspection time per turbine of 15-30 minutes, compared to an entire day’s shutdown for manual inspections, drones reduce man-hours and turbine downtime for maintenance checks by over 75 percent. Ground-based inspections typically miss 15-20 percent of damages found by drones, risking higher failure rates and energy losses. Lightning strike damages at the blade tip, can lead to 6-8% efficiency loss and 500 percent increase in failure rate, while lesser damage, such as a trailing edge split, can result in 3-6 percent efficiency loss and 200 percent increase in failure rate.
Sending wind techs up-tower to inspect wind turbine blades for damage is dangerous and time-consuming. Once up there, some inspectors rely only on a cell phone camera to snap pictures of any problems. The inspection process of an entire field can take months. Risks of drone collisions with assets in bad weather and windy conditions creating physical risks to plant operators and to property. According to Clobotics CTO Yan Ke – “We chose these verticals because the business processes which we help to improve are labor intensive, tedious, and sometimes dangerous for people to do. It takes a crew of inspectors half a day to inspect a turbine which we’re able to finish with one person in less than half an hour.” A Navigant Research survey estimates the market for drones for wind-turbine inspections expected to hit $6 billon by 2024.
Operator-fatigue. Wind turbines present a difficult inspection environment. Turbines are located in windy locations, and as a pilot navigates around the turbine blades, the wind speed and direction can change rapidly and unexpectedly because of how the turbine structure interacts with the wind. This makes manual flights exceedingly challenging and potentially dangerous. Large large visual obstructions presented by turbine blades and housings also create problems for visual line-of-sight type drone inspections. Drone pilots must also adhere to numerous safety regulations associated with flight operations near a structure. For personnel on the ground, this typically involves standing a planned distance away from path of the drone, during takeoff, landing, and flight.
Compliance requirements for FAA safety regulations. FAA’s Part 107 commercial drone certification program has resulted in a flood of ‘hobbyist’ drone pilots looking to earn extra cash. They will often solicit wind companies with the promise of safe, industrial UAVs, but show up inexperienced and often with a cheap, unreliable drones. More than 66,000 remote pilots were certified by the FAA in 2017 in the the U.S. alone. (There are currently more than 110,000 drones registered with the FAA—representing 2.5X growth from 2016. More than 600,000 drones are expected to be registered by 2022.) Hobbyists typically lack wind experience and extensive drone flying skills, he says, so they pose a greater risk to wind-turbine site operations. This may result in asset damage, lost revenue, or serious accidents.
The Data Management & Workflow Challenges
For the amount of data that is processed with a typical inspection (e.g. 600 images or 6GB per wind turbine), reading and consuming the information requires appropriate data and analytics infrastructure. Humans don’t have the capacity to process all of the data that wind-turbine-inspecting drones can capture. Automated drone inspections let customers identify problem areas faster, and optimize repair schedules and costs earlier and more accurately. A good way to help prioritize which defects must be attended to right way vs those than could wait until a larger scheduled maintenance shutdown. Measurable and repeatable data improves a wind operator’s ability to predict problems in the field, and deploy repair crews before an issue escalates, leading to higher operating efficiency/availability and lower overall operations and maintenance costs over the life of the turbine.
There are three main challenges enterprise asset owners face when integrating drones into their O&M work flows:
Incorporating data from multiple disparate sources in a way that meets IT security, scaling, and access requirements
Reducing data review time and minimizing human error
Integrating the reporting and analytics within existing business systems. There is also a lot of interest in connecting drone data to everyday tools like CAD, BIM, GIS, and other software.
Looking At The Future
As industry begins to harness ML/AI for solving more industry domain problems, the importance of having the largest possible data-sets gathered from a wide range of customers and drone inspection flights will become a lot clearer. Additionally, the ability to tag and annotate this large corpus of images accurately using specialized technical manpower becomes a powerful basis for differentiating one drone services provider from another.
Phil Christensen of Bentley Systems envisions a day when offshore wind farms have a hanger for a drone and at set intervals (perhaps daily, monthly or quarterly) that drone will fly around the turbines and capture data, which it will then transmit back to the base for analysis — no human intervention at all.
About Clobotics Fundraising
Clobotics recently raised $12M in its Series A funding bringing its total funding to $21M. The new investors include Nantian Infotech Venture Capital, a subsidiary of the China and Silicon Valley-based IPV Capital, and China’s Wangsu Science and Technology Co. Previous funding came from South Korea’s KTB Network, California’s GGV Capital and the Capital Development Investment Fund Management in Beijing. Many of Clobotics’ customers are international companies and the business is eager to expand operations into South East Asia, Europe and South America.
Dr. Yan Ke, co-founded Clobotics, a computer vision startup, with three other ex-Microsoft executives in 2016. His Ph.D. thesis topic was on using computer vision to automatically recognize human actions in videos. Prior to Clobotics, Yan spent eight years at Microsoft leading the Bing Entity Understanding Group, where he architected and developed the core algorithms for Bing’s Knowledge Pane, Question Answering System, Satori Knowledge Graph, and Web Index Selection. His work in part helped Bing’s world-wide market share grow from 8% to 21%. He is a recipient of the Intel Research Scholar Award, NSF IGERT Fellowship, Microsoft Technical Leadership Award, multiple Microsoft Gold Star Awards, published over 18 top tier conference and journal papers, and holds 7 U.S. patents.