ScootSafe: Intelligent Scooter Fall Detection

 
41472006_173783426832260_3693957871283004919_n.gif
 

It all started when…

I was hurling, gracefully, down the esplanade. I was late for work, however the view was too beautiful to ignore— as I stared at the beautiful city flashing past my eyes my eyes at nearly 20mph I began to feel it happen.

Speed wobble.

And like that I was on my way to the urgent care.

For scooter enthusiasts, reading the scooter’s movements is critical; however, the rider is often distracted which is an issue both for the rider and the operating company. We’ve designed a solution for that issue - Scootsafe is a simple phone-based diagnostic that help scooter commuters by alerting them to crashes before they happen, and getting them help when they do.

How it Works

Scootsafe requires nothing more then the users phone. By placing the phone in the pocket, we leverage the powerful gyroscopes in the phone to read the movements of the scooter rider and determine their fall potential.

 
Group 3test.png
 

Scootsafe assumes that the rider of a scooter is essentially an inverse pendulum with a relatively strong righting moment that consists both of the righting moment of the scooter itself, and the riders self corrective actions.

The speed wobble itself, is an interesting affect itself to understand, and gets into control theory.

Systems that are stable are systems where disruptions are reduced, and you return to a equilibrium at some point. Unstable systems are where, when disturbed, the system goes out of control (our speed wobble).

giphy (3).gif

Think of a marble on balancing on the back of a bowl, versus one in a bowl. You can maybe balance a marble on the top of it, but if you move it at all, it will start rolling off the edge - and roll away faster and faster (falling out of control). Instead, a marble in a bowl, if you bump it, will jiggle back and forth, but self correct.

A scooter is a relatively complex system, and within certain parameters, it is a stable system— however, there exist specific parameters where the system will be unstable.

The first component is gravity, and it is deeply unstable— when you lean left or right, you move your center of mass away from right-above the scooter. The further you are off center, the more significant your weight will torque you into the ground.

The next component is the correcting, stabilizing component. When you are moving, and you lean to the left, you start turning left. This means you are now pointed a few degrees to the left - but you still have forward momentum. This forward momentum tries to flip you over the opposite direction, pushing you back up-right and scooting along.

Why is this a problem at speed?

Well, the correcting-force will be stronger. So when you just lean slightly to the left, you'll receive a very large force kicking you to your right. That force might be so large, that it pushes you over to your right. Now you begin turning to the right. The problem is, that that force is still pushing you over to the right because you're still pointed to the left relative to your initial forward momentum.

 
4.png
 

After a short delay, you'll turn enough to the right that you're now pointing right, relative to that forward momentum, and now the corrective force is trying to rotate you back left. This delay is the catch. If the delay is short, you'll go back and forth like this. This is the wobble.

tenor.gif

If the delay is longer then you'll over-correct too quickly, before you turn enough to correct the over-correction. The faster you're going, the longer this delay. During this delay, both gravity and your correcting force are conspiring to tip you in the same direction. Because you have to be at high speeds for this to happen in the first place, the correcting-force is pretty strong, and very difficult for you to counteract by shifting your weight.

At this point, any small mistake you make will result in those forces winning, and you having a nasty crash.

This overcorrection is what Scootsafe attempts to notify the user of before the crash.

Our Big Data Strategy

In order to determine when a user is going to fall, we need to be able to correctly diagnose each part of a ride.

test.jpg

In order to do this, we had many riders do a difficult journey on a scooter and collect the gyro data. The journey included normal riding, a section to generate speed wobbles, and then a fall. The total of these rides were collected in aggregate and split into unique ‘qualia’ used to diagnose each ride section.

fft.jpg

A simple FFT of these signals shows how unique each qualia is, and the frequencies that are characteristic of them. As each of these are roughly periodic, but lack a single characteristic frequency to identify, it is easy enough to identify the bands that characterize each of these behaviors. What is most important to notice here is that turning has a quantifiably different profile than speed wobbles, which means that we should be able to differentiate both.

asads.png

Scootsafe compares two signals by collecting a continuous sampling of frequencies in the x and y and transforming them into a 3x2 matrix, continuously with a time buffer of 10 clicks, and comparing that spatially with a known ‘meme’ or apocryphal signal collected from an average of previous known actions. The higher the cosine similarity, the closer the signal is to being that class.

The benefit of continuously running these convolutions, is much the same as as why we use n-grams to characterize words on NLP. A fixed sequential buffer will often mischaracterize the beginning and ends of samples by splicing them in the middle. A continuous convolution solves this problem.

phone.jpg

With this done, we can characterize each convolution for its probability (just the cosine similarity) of being each time of feature. In order to save computational time, we combine off-pocket and on-pocket falls as a single class with two different meme signal projections, and take the similarity that is closet. Additionally, removing the “getting on scooter” class and simply dropping the first five seconds of data saves enough computational time that we are now within 80% of real-time capable (reading from file, not live streamed). Switching from Matlab, to a more lightweight language would allow us to get to a fully real time Scootsafe.

Scootsafe’s fall detection algorithm seems to be working really well. When tested on a top speed ride down a hill selected to induce speed wobble, Scootsafe correctly diagnoses later convolutions as closer to the speed wobble meme-signal. Correctly diagnosing a fall seconds before the rider hit the cement.

The Real Utility

kingston.gif

While Scootsafe is certainly a useful tool for individual riders, its utility really shines when integrated with a large scale scooter operator. Companies like Bird or Lime operate thousands of scooters on the streets of various cities daily, and currently rely on user reports to diagnose scooter mechanical failure due to crashes. Integrating Scootsafe’s algorithm into the hardware already on the scooter would allow automatic crash detection, both to ensure maintenance of the fleet and help users when they fall.

Another interesting extension to this work is fall prevention itself. As speed wobbles can be solved by simple decelerating, a system that decelerates the scooter by some rolling variable based on the ground speed of the scooter could prevent accidents altogether.

I reached out to a friend at Bird, a dock-less scooter operator, and he confirmed they are working on similar tools internally which was incredibly exciting.