Over recent years, Big Data and Artificial Intelligence has dominated the tech sector, resulting in natural concerns for individual privacy. Mozilla’s ethos of “Internet for people, not profit” was definitely reflected at Mozilla Festival (MozFest) 2018, with many sessions raising awareness of internet and data privacy.
I attended MozFest in 2017 and was intrigued by numerous facilitators and their sessions all promoting the festival’s theme of a healthy internet. The theme this year was “Your Data & You” and I took inspiration from last year and proposed a group gallery session - IoT & Me. The goal? To interactively show how third party developers might use Internet of Things (IoT devices) within their apps to gather data and build up a personal profile of a user. Our group wanted to show the significance of user profiles and what they might be used for - specifically targeted advertisements.
Before I presented the project at MozFest 2018, I initially thought our project was too ambitious as it required a lot of team collaboration and even simple machine learning knowledge. However discussing the idea with a few friends, we decided to tackle the problem together.
As a team, we worked out what IoT devices would be suitable for the project. We initially decided to use an Amazon Echo, a Sonos Sound System, a Philips Hue Light Bulb and a Smart Fridge. However due to space, portability and cost limitations, we were unable to use the Smart Fridge and Sonos Sound System. Instead, we decided to use an Arduino Uno and the Spotify App (coupled with a Bluetooth Speaker) to act as substitutes for these devices. The Arduino Smart Fridge required four Light Dependant Resistors (LDRs) and a cardboard chassis to house the wires. We placed a food item on each LDR so that the circuit didn’t complete and used C to program the Microcontroller. After the food item was lifted from the LDR (exposing the LDR to light), the circuit would complete and TeraTerm logged the data, allowing us to see what food item was taken.
With the chosen devices, we set out the user journey for the session. The user would:
Speak to the Amazon Echo, saying a phrase like “Tell me a joke”.
Choose a playlist from Spotify, such as ‘Dance Party’.
Select a particular colour for the Philips Hue Light Bulb, such as the colour ‘Blue’.
Pick a food item from our Arduino Smart Fridge, like an apple.
In order to get the data from the Echo, Philips Hue and Spotify, we used Python and the device’s associated Python library - for example I used Spotipy to obtain the data from Spotify. Afterwards, the user must click a button that passes the data gathered from each device to an algorithm that uses Machine Learning (developed with Python using the sklearn library).
Since we wanted to see how developers might build a personal profile of a user based on IoT device usage, we created the algorithm so that it would generate a personality type for the user. In order to deduce the personality types, we trained the algorithm with data gathered from our college and a few MozFest facilitators. We surveyed around 60 people asking them four different questions, one for each IoT device. The questions included “What’s your favourite colour?” and “What’s your favourite genre of music?”, each with four options. At the end, we asked them what personality describes them best: Logical, Loud & Talkative, Loves to Give Advice, Calm & Self Reflective, Energetic, Adventurous and lastly a Leader. This information provided sufficient (though not ideal) data to train the algorithm, allowing it to predict future personalities with different data and people.
In the final week proceeding MozFest, we took each component that we individually worked on for the project and pieced it together. With the help of Electron JS, we were able to create a visualisation for the user’s personality - slightly nicer than having it generate in a text file! Testing it in college worked fine, however certain devices decided not to work on the day of our session after carefully setting it up the day before; a typical tech demo issue! Thankfully, it all worked in the end and I am happy to say that the session was a success. The algorithm was able to predict the right personality at least 50% of the time, and having free sweets definitely enticed some people to visit us!
I am very proud of what I have personally achieved and what the group has achieved together, but I also hope that I have made an impact on other “MozFesters” demonstrating how easy it is for developers to build personal profiles of them with basic IoT device usage.