Smelling Fresh, Feeling Fresh!

Bymaturobot

Use an Arduino Nicla Sense ME to see if you need to freshen up after a workout

Project description

This project is part of the $ K-Way + Arduino partnership $ to develop a Smarter coat. These are two iconic Italian brands! Our project pitch was to build a smart sensor that would let you know if you got a little stinky after a workout and should take a shower. All of this would be powered using sensors and an ML model, developed in the $ Edge impulse studio $ . The project is a joint effort between Luke (the dad) and Elena, (the daughter).

A little while after we were notified that our project was selected, a package arrived in the mail from Italy containing a turquoise K-Ways coat and a Arduino Sense ME. The Arduino was packaged up with a nice case, with an internal battery.

Once we starting working on this project we faced a interesting question; how long would we have to go without showering in order to collect the data need to train the machine learning model? But then we realized that stink is all around us, from dirty laundry bins, to a trash can. For this project we learned how to use the Bosch BME688 4-in-1 gas sensor on the Nicla Sense and what its limits were. At first, we thought it couldn’t be done because we were not getting consistent reading. Before, I give a inspirational message I’d like to say that, yes, we did give up, and that we started to build a different project. But when that one couldn’t be done, we jumped back onto the Smelly Coat train. So here it is, the step by step process that we took to build a coat that will tell you if you need to shower. Hopefully this guide can help you avoid the mistakes that we made.

Day One

This was the first day that we decided would officially chose this project, February 6th, 2023. We have just figured out how we will complete the project and what sensors we will be using aboard the Nicla Sense ME. We started with using the humidity and gas sensor, since they will help determine if you are sweaty and smelly. On order to sample something smelly, we started to collect data from armpits. Of course, we collected some clean objects like clean laundry and a recently showered tester. From there we recorded all our data in Edge Impulse and labelled it as either smelly or clean, so that the machine could began its learning.

Day Two

Now that we have enough samples, today we uploaded the software to the device. But this is when things started to go down hill. For starters, the humidity data wasn’t working in the model. The readings from the gas and humidity sensors were too far apart and it may have been confusing the ML training. The values for humidity ranged from 0 – 100 and the gas sensor values were from 3,000 -25,000. To get things working, we ended up cutting the humidity sensor and just using the gas sensor.

But we weren’t out of the jungle yet. When we had uploaded the code to the board it started acting crazy and was confusing that clean was dirty, and dirty was clean. But then it dawned on us that we may need more samples. So when you build this yourself, make sure to have a decent amount of samples, this way the ML Model will be more accurate. Aside from all the things we needed to improve, we did actually make progress. Before we uploaded the new model to the Arduino, we tested how accurate it was through Edge Impulse, and things looked good. There were some times where the machine guessed incorrectly, but other than that the machine did a good of separating clean samples from the dirty ones.

Day Three

Now that we have our ML Model successfully detecting smelliness, we needed to find a way to alert the wearer that they should shower! The best option would be having an App that would buzz your phone when the sensor detects smelliness. Bluetooth Low Energy is a wireless protocol that makes it easy to connect a sensor to your phone. And the Nicla Sense ME has BLE built-in… Score! 🎯 Unfortunately, we ran into some trouble when we tried to load the Nicla with both the ML Model and the BLE stack… it ran out of memory and nothing would work. Looking in different forums, it turns out you can get both of them to load if you use Platform.io instead of the Arduino IDE. We tested it out and using Platform.io works, but it made the whole project a lot more complicated.

Instead of a fancy App, we decided to go back to basics and just use the built-in RGB LED on the Nicla. When it detects clean air, it makes the LED Green, and for bad smells, it makes it Red. When the model is uncertain on how things smell it will make it Yellow.

With all of this working, Elena took the sensor out for a run. Unfortunately, or perhaps fortunately, she did not run hard enough to trigger the smelly detector. We are going to keep running and keep working on this! Our Project in Edge Impulse is $ available here $ if you want to give it a try.

CODE

/* Edge Impulse ingestion SDK
 * Copyright (c) 2022 EdgeImpulse Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 *
 */

/* Includes ---------------------------------------------------------------- */
#include <smelly-coat_inferencing.h>
  #include "Nicla_System.h"
#include "Arduino_BHY2.h" //Click here to get the library: http://librarymanager/All#Arduino_BHY2

/** Struct to link sensor axis name to sensor value function */
typedef struct{
    const char *name;
    float (*get_value)(void);

}eiSensors;


/** Number sensor axes used */
#define NICLA_N_SENSORS     1


/* Private variables ------------------------------------------------------- */
static const bool debug_nn = false; // Set this to true to see e.g. features generated from the raw signal


Sensor gas(SENSOR_ID_GAS);

static bool ei_connect_fusion_list(const char *input_list);
static float get_gas(void){return gas.value();}

static int8_t fusion_sensors[NICLA_N_SENSORS];
static int fusion_ix = 0;

/** Used sensors value function connected to label name */
eiSensors nicla_sensors[] =
{
    "gas", &get_gas,
};

/**
* @brief      Arduino setup function
*/
void setup()
{
    /* Init serial */
    Serial.begin(115200);
    // comment out the below line to cancel the wait for USB connection (needed for native USB)
    //while (!Serial);
    Serial.println("Edge Impulse Sensor Fusion Inference\r\n");

    /* Connect used sensors */
    if(ei_connect_fusion_list(EI_CLASSIFIER_FUSION_AXES_STRING) == false) {
        ei_printf("ERR: Errors in sensor list detected\r\n");
        return;
    }

    /* Init & start sensors */
    BHY2.begin(NICLA_I2C);
    gas.begin();

    nicla::begin();
    nicla::enableCharge(100); 
    nicla::leds.begin();
    nicla::leds.setColor(yellow);
}

/**
* @brief      Get data and run inferencing
*/
void loop()
{
    ei_printf("\nStarting inferencing in 2 seconds...\r\n");

    delay(2000);

    if (EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME != fusion_ix) {
        ei_printf("ERR: Nicla sensors don't match the sensors required in the model\r\n"
        "Following sensors are required: %s\r\n", EI_CLASSIFIER_FUSION_AXES_STRING);
        return;
    }

    ei_printf("Sampling...\r\n");

    // Allocate a buffer here for the values we'll read from the IMU
    float buffer[EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE] = { 0 };

    for (size_t ix = 0; ix < EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE; ix += EI_CLASSIFIER_RAW_SAMPLES_PER_FRAME) {
        // Determine the next tick (and then sleep later)
        int64_t next_tick = (int64_t)micros() + ((int64_t)EI_CLASSIFIER_INTERVAL_MS * 1000);

        // Update function should be continuously polled
        BHY2.update();

        for(int i = 0; i < fusion_ix; i++) {
            buffer[ix + i] = nicla_sensors[fusion_sensors[i]].get_value();
            ei_printf("Gas Sensor: %.2f\n",buffer[ix + i]);
        }

        int64_t wait_time = next_tick - (int64_t)micros();

        if(wait_time > 0) {
            delayMicroseconds(wait_time);
        }
    }

    // Turn the raw buffer in a signal which we can the classify
    signal_t signal;
    int err = numpy::signal_from_buffer(buffer, EI_CLASSIFIER_DSP_INPUT_FRAME_SIZE, &signal);
    if (err != 0) {
        ei_printf("ERR:(%d)\r\n", err);
        return;
    }

    // Run the classifier
    ei_impulse_result_t result = { 0 };

    err = run_classifier(&signal, &result, debug_nn);
    if (err != EI_IMPULSE_OK) {
        ei_printf("ERR:(%d)\r\n", err);
        return;
    }

    // print the predictions
    BHY2.update();
    float current = gas.value();
    ei_printf("Gas Sensor: %.2f\n",current);
    ei_printf("Predictions (DSP: %d ms., Classification: %d ms., Anomaly: %d ms.):\r\n",
        result.timing.dsp, result.timing.classification, result.timing.anomaly);
    for (size_t ix = 0; ix < EI_CLASSIFIER_LABEL_COUNT; ix++) {
        ei_printf("%s: %.5f\r\n", result.classification[ix].label, result.classification[ix].value);
    }
    if (result.classification[0].value > 0.80) {
          nicla::leds.setColor(green);
    } else if (result.classification[1].value > 0.80) {
      nicla::leds.setColor(red);
    } else {
      nicla::leds.setColor(yellow);
    }
#if EI_CLASSIFIER_HAS_ANOMALY == 1
    ei_printf("    anomaly score: %.3f\r\n", result.anomaly);
#endif
}

#if !defined(EI_CLASSIFIER_SENSOR) || (EI_CLASSIFIER_SENSOR != EI_CLASSIFIER_SENSOR_FUSION && EI_CLASSIFIER_SENSOR != EI_CLASSIFIER_SENSOR_ACCELEROMETER)
#error "Invalid model for current sensor"
#endif


/**
 * @brief Go through nicla sensor list to find matching axis name
 *
 * @param axis_name
 * @return int8_t index in nicla sensor list, -1 if axis name is not found
 */
static int8_t ei_find_axis(char *axis_name)
{
    int ix;
    for(ix = 0; ix < NICLA_N_SENSORS; ix++) {
        if(strstr(axis_name, nicla_sensors[ix].name)) {
            return ix;
        }
    }
    return -1;
}

/**
 * @brief Check if requested input list is valid sensor fusion, create sensor buffer
 *
 * @param[in]  input_list      Axes list to sample (ie. "accX + gyrY + magZ")
 * @retval  false if invalid sensor_list
 */
static bool ei_connect_fusion_list(const char *input_list)
{
    char *buff;
    bool is_fusion = false;

    /* Copy const string in heap mem */
    char *input_string = (char *)ei_malloc(strlen(input_list) + 1);
    if (input_string == NULL) {
        return false;
    }
    memset(input_string, 0, strlen(input_list) + 1);
    strncpy(input_string, input_list, strlen(input_list));

    /* Clear fusion sensor list */
    memset(fusion_sensors, 0, NICLA_N_SENSORS);
    fusion_ix = 0;

    buff = strtok(input_string, "+");

    while (buff != NULL) { /* Run through buffer */
        int8_t found_axis = 0;

        is_fusion = false;
        found_axis = ei_find_axis(buff);

        if(found_axis >= 0) {
            if(fusion_ix < NICLA_N_SENSORS) {
                fusion_sensors[fusion_ix++] = found_axis;
            }
            is_fusion = true;
        }

        buff = strtok(NULL, "+ ");
    }

    ei_free(input_string);

    return is_fusion;
}

Similar Posts

Leave a Reply