Introduction
Follow along as we build a proof-of-concept automated pothole detector prototype with a map-based dashboard. We’ll do it all with 5 lines of code and $100 of electronics. Once built, your system will detect potholes, send pothole incident data to a cloud system, and map all potholes on a cloud dashboard. Oh, and it’s so easy, a marketing guy can do it.
A truck willing to hit as many potholes as possible
If you decide to follow along at home, just get our Feather Starter Kit for ESP32. They are in stock now and have everything you need except the USB cable (and who doesn’t have 647 USB cables in that one drawer you never talk about?).
Use Case: Passive monitoring of potholes and road conditions.
Target: Municipalities and Departments of Transportation
In December, my colleague Rob Lauer set up a fun internal hackathon for us to get hands-on with our products. A majority of our staff participated and “competed” to win midwestern themed prizes. Another colleague, Tina Kocak, recently attended the Raleigh Connected Communities Conference and shared some of the interesting things happening in the smart cities arena. Her experiences inspired me to see just how far I could go with one of our products, the Notecard. As the head of marketing, I don’t have an electronic engineering background so it was important to find something meaningful to build while avoiding deep technical skills. As the saying goes, “friends don’t let marketing guys write firmware”.
Vision: Using Smart City Data to Manage Road Conditions and Prioritize Roadwork
Surveying roads is a continuous job requiring manual oversight and reporting of all road assets under management. Potholes annoy drivers, increase vehicular wear and tear, and decrease roadway safety. Once a pothole starts to form, the increased vehicular bounce caused by surface irregularity increases the rate of surface material failure. Potholes are not self-healing, rather a road crew must:
Remove the surface material
Remove the subsurface material to correct the foundational deficiency
Replace the subsurface material and sufficiently compress
Replace the surface material and smooth out the grade
Repairs are costly often requiring specialized equipment and materials. Further pothole repair impacts traffic flow adding to driver frustration, thus it is important to properly plan and prioritize roadway repairs. Given all of the above, pothole work is thankless. When was the last time you remember giving your DOT a 5-star review and some fire emojis for having smooth roads?
Improve Reliability & Connectivity of Transportation System
Promote Economic Growth Through Better Use of Our Infrastructure
Make NCDOT a Great Place to Work
By collecting surface irregularity data and filtering by condition severity and vehicles impacted, the NCDOT can better serve North Carolinians and improve their goal performance by improving collection expense and redirecting resources where they are most needed. Remember, it’s our tax dollars at work so let’s have them work efficiently.
Technical Solution
Daily, NCDOT vehicles drive roadways to perform work. What if we could automate pothole detection, grade the severity, and provide data on each incident to a centralized location? To be successful, a solution would need to be:
Automatic – not adding additional driver responsibilities
Use existing DOT IT systems to avoid licensing, data privacy, training, and governance issues
Be cheap enough for wide deployment into environments where devices will inevitably be broken or lost due to the rugged nature of the job
The Blues Wireless Notecard has many of the necessary components and integrations to power the build. Coupled with the Notecarrier AF expansion board and an Adafruit Huzzah microprocessor, a working demo device can be built for $100. Once the system is set up, there is no need for field configuration or adjustment. The device is monitorable and administrable via remote connection through the Notehub application. Notehub may be configured to send data directly to state IT systems, including commercial cloud systems or custom applications. Because device configuration and software changes are pushed over-the-air to each device, maintenance procedures don’t require highly technical staff nor physical possession of the device, further reducing operational expense.
This is a fun project to play with because it’s easy enough to make and the overall system is impressive, given the little amount of work required. Here’s the steps:
Then drive around, swerving into potholes, and looking at the results.
[Editors note, insert legal disclaimer about reading trace files while driving]
Upgrades for a Field Version
For a field deployment, it would be necessary to both provide electric power to the unit and mount the device on the host vehicle without alterations to the host vehicle. One power option for certain workloads would be to integrate a solar panel. Naturally, you’d want to minimize battery usage because vehicle positions change often, and it is unreasonable to burden drivers with always parking the vehicle for optimal solar energy collection.
Another idea, be it a bit far-fetched is to charge via kinetic capture. After all, the host vehicle will be bouncing around from hitting potholes and we should be able to capture some of this kinetic energy. Sadly, in reality, it would be very expensive to implement a kinetic capture system given the power we’d need. Perhaps Elon Musk will read this and send his best folks to work on the problem.
More realistically, we could wire in the system to the host vehicle. Many DOT vehicles have a trailer light connector socket, and we could piggyback on that system.
Lastly, we could place the system in a sealed enclosure with an inductive charging system and expect the DOT staff to charge devices as needed. Such a system would let us build a rugged system up to the challenges of DOT work and workers without introducing brittle charging elements or holes in the enclosure. On the flip side, it would require DOT staff to manage the devices, a new responsibility on their working day.
My Smart City Pothole Reporting Dashboard
In the dashboard screenshot below, you’ll notice a trip consisting of 1 inbound and 1 outbound segment. The Pothole-o-matic is designed to measure the types of vehicular shock incurred when hitting potholes, snapshot the location via an in-built GPS system, then push the data into the mapping dashboard. Each circle on the route indicates potentially broken pavement. If pushed, I’d confess to swerving INTO several potholes for the sake of science.
The GPS system needs a reasonable view of the southern sky and won’t easily penetrate a brick home. An extra long USB-A to USB-Micro A helps here. So does a conference badge (see below figure).
Below, see my expert field testing setup. Later versions secured the device a bit better to the dashboard. I made sure not to cushion the device so it record maximal hits. Life is tough, get a helmet.
I tested the Pothole-o-matic by driving on surface streets. I’m happy and sad to say I had a hard time finding potholes. The Notecard can be configured to trigger events based on shock values, so you can easily tune the device to pick up on the vibrations or shock to fit your purpose. I left my accelerometer setting at the lowest setting.
Interestingly, the Notecard will also track “free-falls”. A free fall is when the accelerometer detects less than 1G of force across all its sensors. Free falls were a better detector of good potholes than just using the shock value. The free-fall parameter is not passed into Notehub at the moment; thus I didn’t use it in the reporting.
I recommend turning “trace” on to see all the interesting information generated by the Notecard, but make sure you safely stop your vehicle before doing so. [Editors note: Insert more legalese here.]
To make our generated pothole data useful for the NCDOT planners, we include a map-based dashboard plotting each pothole occurrence. Future versions may include the ability to indicate pothole severity or calculate detected incidents by proximity so planners can efficiently design maintenance truck rolls.
Our dashboard is provided by Datacake. By simply configuring Notehub and Datacake, and copy/pasting a small code snippet, we get an unbroken line of data streaming from our device in the field, out to the publicly viewable map dashboard. Here’s an easy to follow tutorial written by a co-worker.
I did make a few mistakes configuring Datacake, and my co-worker Paige helped me through my self-inflicted issues.
What Did I Learn?
Blues Wireless products are so easy to use, marketing people can build stuff. My solution works well because data is automatically shipped over the cellular network without me having to get too deep in the technical parts. Further, the accelerometer and GPS devices I’m using are baked into the Notecard, saving me the trouble of finding in-stock components, understanding how they work, and integrating them successfully. It was straightforward for me to unbox our products, paste in JSON commands, use web forms to configure what I wanted, then watch the magic happen.
Additionally, there are some opportunities for Blues Wireless to fill out the accelerometer capability by exposing the free fall metric and the raw accelerometer readings. For my purpose, it’s fine to just detect reasonable bumps, but it would be better to have more data on which to make improvements.
Additional Uses for the Solution
My solution contains vibration/shock detection and mapping to produce reporting for potholes. There are other use cases for my solution.
Safety and compliance – During construction, certain machinery makes shocks that impact surrounding buildings, businesses, or people. We’ve all seen pilings being driven into the earth through mechanical means during the construction process for bridges, overpasses, or large commercial buildings. By placing several Pothole-o-matic devices around the perimeter of a construction site, it’s possible to measure shock magnitude and location then initiate mitigation protocols.
Warranty Management – Certain equipment is rated for duty-loads. When a customer exceeds the duty load rating and the equipment breaks, it can be quite difficult to apportion responsibility for the breakage. By installing a Pothole-o-matic on a piece of equipment, the manufacturer can be apprised of any severe shocks to the equipment and be better prepared to administer the right amount of warranty benefits given the situation.
What’s Next
I’m keen to hear your experiences when you try this project for yourself. When you do, post a picture on Twitter and mention us @blueswireless. We like seeing what people do with our products and to say thanks, we’ll rummage around the swag room and send something your way. Happy building!
