BehaWear: A wearable for detecting and classifying automobile driver behaviour

read 7 mts

Disciplined driving is a fundamental requirement in many countries, especially in India.  Honking, frequent swerving and sharp turns are major traffic congestion and safety hazards. While the advent of IoT is resulting in the evolution of smarter and connected cars, no current solution is catering towards Indian roads specifically for honking and swerving (over-steering or frequent lane changing).

This article covers a simple solution to address these two basic driver behaviour nuisances on roads.

Image Reference: Nissan’s Nismo Watch connects  

Background

This increased power of hardware and software with reduced cost has significantly helped in the development of a field which is today known as the Internet of Things or IoT.  IoT makes one physical object (home electronics, lighting systems, wearable devices, etc. ) connecting with each other and communicating data from one thing to other “thing” over the internet.

Sensors and associated electronic hardware are not only getting smaller but also extremely affordable. Over that, with the advent of single board computers (SBC) like Raspberry Pi and prototyping boards such as Arduino and NodeMCU, makes it easy for professionals and hobbyist to try out a new idea which can be the technology of tomorrow.  The final layer of sophistication comes from making this system truly intelligent. This is possible due to equally or perhaps even more fast-growing field of AI especially Deep Learning and Neural Networks.

Dr Parag Mantri along with Shravan Pagolu, Data Scientist at INSOFE aimed at designing and developing a wearable for human activity monitoring and later using the generated data from this wearable to predict behavior using Machine Learning models.

What is Wearable?

Wearable technology, wearables, fashionable technology, wearable devices, tech togs, or fashion electronics are smart electronic devices (electronic device with micro-controllers) that can be incorporated into clothing or worn on the body as implants or accessories.

Wearable devices such as activity trackers are an example of the IoT since “things” such as electronics, software, sensors, and connectivity are effectors that enable objects to exchange data (including data quality) through the internet with a manufacturer, operator, and/or other connected devices, without requiring human intervention[1].

Figure 1: Different types of wearable devices [2]  

 So, one such measurement for our discussion at hand is the measurement of movement. To measure the movement of an object/human, one can think of an electronic component called “Accelerometer”.

“An accelerometer is an electromechanical device used to measure acceleration forces. Such forces may be static, like the continuous force of gravity or, as is the case with many mobile devices, dynamic to sense movement or vibrations. Acceleration is the measurement of the change in velocity or speed divided by time”[3].

Today, we find accelerometers mostly in smartphones/watches and tablets with which even application developers are exploiting the intrinsic device functionality. One such scenario is the Compass App developed and marketed by Apple Inc in the late 2000s with the introduction of its flagship iPhone 3GS model.

The iPhone 3GS features iOS, Apple’s mobile operating system. The user interface of iOS is based on the concept of direct manipulation, using multi-touch gestures. Interface control elements consist of sliders, switches, and buttons. Interaction with the OS includes gestures such as swipe, tap, pinch, and reverse pinch, all of which have specific definitions within the context of the iOS operating system and its multi-touch interface. Internal accelerometers are used by some applications to respond to shaking the device (one common result is the undo command) or rotating it vertically (one common result is switching from portrait to landscape mode)[4].

Figure 2: A typical Accelerometer used in various devices [5]

Model Architecture

 Figure 3: Block diagram of the various components of the working model

 The primary objective of the hardware is to measure the acceleration and rotation of the user during various activities (not limited to) like walking, jogging, relaxing etc.

In the present context, the same principle of activity identification is proposed for automobile driver behaviour.

A typical Inertial Measurement Unit (IMU) can measure up to 9 axis (x, y, z values of Accelerometer + x, y, z values of Gyroscope + x, y, z values of Compass).

Acceleration for three axis is AX, AY and AZ.

In order to measure Rotation, a Gyroscope is used.

Figure 4: Degrees of Freedom [6]

 PitchY (Psi) and RollX (Rho) are the rotation measurements along X and Y axis and YawZ is a measurement along the Z axis.

Figure 5: Pitch and Roll calculation from Acceleration parameters [7]

The prototype being a dual-core architecture, we have developed the protoboard to have a dedicated display processing unit having a dedicated silicon core.

As it can be seen from the above block diagram, one core is dedicated to compute the movement of the subject (human) and measure the Acceleration and Rotation parameters and send the same using a wireless communication medium (Bluetooth in our case) to a user mobile device.

The mobile device acts like a gateway to the Cloud for further processing using robust AI / ML techniques.

Figure 6: Timing diagram of interchip communication between Core 1 and IMU

Figure 7: Sample data showing the Acceleration parameters depicting the author behaviour

Figure 8: Actual pictures of the working model

INSOFE joins the bandwagon with a different approach by building a wearable from scratch for detecting and classifying automobile driver behavior.

Watch this video:

Future scope

To make the system robust and accurate, the designer must take multiple sensor data into account. However, it is easier said than done as integrating and adding multiple sensors to the system increases complexity from both hardware and firmware viewpoints.

The primary presumption is the efficient power management and on-board memory optimization to fit the multiple sensor library functions on the SBC.

The secondary would be to make use of digital signal processing algorithms (DSP) like Extended Kalman Filter (EKF) to optimize the accuracy and authenticity of the multiple sensor output. In statistics and control theory, this is also known as Linear Quadratic Estimation (LQE) which results in to an robust discipline called Sensor Fusion.

The tertiary being the miniaturization of the proto board PCB itself to give the definition of a “Wearable” to the working model. Currently, the working model at hand (can be seen in the above actual picture) is in the size of a typical business card.

Below are some of the use cases where our working model is best suited:

  • Identifying Potholes[8]
  • Statistical Machine Learning of Sleep and Physical Activity[9]
  • Activity Recognition and Intensity Estimation in Youth[10]

It can be inferred from the above use cases that multiclass datasets can be generated for using a variety of Machine Learning algorithms to verify the accuracy and correlation of explainable variables in the data.

So, to conclude, as sensors are getting smaller and cheaper day-day, and with the vibrant open source community, Machine Learning for signal data has become a norm.

Also, with widely available sensors, rich Analytics frameworks can be built by “teaching” a machine to “identify patterns” in “real-time” just by integrating an inexpensive CPU  for processing data.

References

  1. https://en.wikipedia.org/wiki/Wearable_technology
  2. Enabling Technologies for the Internet of Health Things, Joel Rodrigues, Dante Segundo et al (Instituto Nacional de Telecomunicações), Murilo H. Sabino, Jalal Al-Muhtadi, Victor Hugo Costa De Albuquerque (Universidade de Fortaleza), https://www.researchgate.net/publication/322261039_Enabling_Technologies_for_the_Internet_of_Health_Things
  3. Accelerometers: What They Are & How They Work, Ryan Goodrich (LiveScience contributor), https://www.livescience.com/40102-accelerometers.html
  4. https://en.wikipedia.org/wiki/IPhone_3GS
  5. https://www.makerlab-electronics.com/product/triple-axis-accelerometer-gyro-breakout-mpu6050/
  6. http://www.dutchops.com/Portfolio_Marcel/Articles/Instruments/Gyroscopic_Instruments/Theory_Gyroscopes.htm
  7. Explore how we design automatic balancing camara mount, Yuzhuo Sun, Hanxiang Hao, Department of Electrical and Computer Engineering, Cornell University, https://courses.ece.cornell.edu/ece5990/ECE5990_Fall15_FinalProjects/Yuzhuo_Sun/ECE5990_project_demo/design.html
  8. Intelligent Pothole Detection and Road Condition Assessment, Umang Bhatt, Edgar Xi et al, Carnegie Mellon University, Pittsburg, https://dssg.uchicago.edu/wp-content/uploads/2017/09/bhatt.pdf
  9. Statistical machine learning of sleep and physical activity phenotypes from sensor data in 96,220 UK Biobank participants, Matthew Willetts,Sven Hollowell et al, https://www.nature.com/articles/s41598-018-26174-1
  10. Activity recognition and intensity estimation in youth from accelerometer data aided by machine learning, Xiang Ren, Wei Ding et al, https://www.semanticscholar.org/paper/Activity-recognition-and-intensity-estimation-in-by-Ren-Ding/4155779c1df5ca50faf1dd3f256a510f35443cb7
0

Leave a Reply

Your email address will not be published. Required fields are marked *