Hackathon project by Manasa Rao, Stephen To and Juan Benavides
Would you like to learn an easy way to control a Zigbee-based Light with Alexa voice commands?
Or how about controlling a Zigbee-based Light automatically with an Occupancy Sensor?
Furthermore, how would you like to learn how to easily create your own Bluetooth mobile application to control a Zigbee-based Light and access the Internet?
In this Hackathon project we demonstrate how to easily accomplish all of these cool features.
Silicon Labs released on November 2017 the new Dynamic Multiprotocol Software (DMP) for our Wireless Geckos. In summary, the DMP software enables the simultaneous operation of Zigbee and Bluetooh Low Energy (BLE) on a single radio chip.
The best example to showcase this technology is by controlling and monitoring a Zigbee-based Light directly over Bluetooth with a smartphone mobile application. This example is actually available with the DMP SDK and our Hackathon project is based on it.
The following image illustrates the scope of this blog:
The Demo featured in this blog is based on the official DMP Light/Switch Demo that consists of two devices:
Light: it runs Micrium OS to switch between Zigbee and Bluetooth on a single radio.
Switch: it supports Zigbee to provide wireless control of the Light.
Click here to get started with this demo.
In the original demo, users press a push button on the wireless Switch device, which in turn, sends a Zigbee Light Link (ZLL) command to toggle the Light On or Off.
The Light device runs Micrium OS to switch between Zigbee and Bluetooth while sharing the same radio. This allows a mobile application to connect to the Light via Bluetooth.
Users tap the mobile application to toggle the Light On/Off.
The Light device receives the Bluetooth notification and not only toggles the Light On/Off but also updates the corresponding ZLL Cluster Attribute so the Switch remains in sync.
This blog shows you how to add an Occupancy Sensor to the Switch device by connecting the Silicon Labs Optical Sensor Expansion Board (Si1133/Si1153) to the Switch's EXP Header (I2C).
The Optical Sensor Expansion Board has a series of optical sensors to detect the presence of a person in the room. The occupancy sensor is used by the embedded application to control the light automatically.
Click here to see Stephen's blog on how to add this occupancy sensor to the DMP Light/Switch Demo.
This blog is also going to show you how to control the Light with Alexa Voice Commands.
To enable Alexa Voice Commands, we are also going to show you an easy way to create your own Bluetooth Mobile Application. This mobile application will use the smartphone as a gateway to access the Internet.
Click here to see Juan's blog on how to create your own BLE mobile app to control the DMP Light/Switch Demo.
Users will get to control the Light with Voice Commands and this blog will describe how to setup the system in Amazon Web Services (AWS).
Click here to see Manasa's blog on how to add Alexa capabilities to the DMP Light/Switch Demo.
In the following, I will do an analysis of the Windows operating system (OS) from the point of view of communication with Bluetooth Low Energy devices – in our case with different types of SensorTags: Thunderboard React, Thunderboard Sense (both produced by Silicon Labs Company), CC2650STK and CC2541DK (both developed by Texas Instruments Company).
I what follows, I will analyze Windows 7, Windows 8.1 and the following Windows 10 versions:
The analysis will be done from the following points of view:
All the tests were done using 18.104.22.168 version of the blessTags application. The blessTags application was built having as support the Windows SDK – Bluetoothapis. Functions like BluetoothGATTGetCharacteristicValue, BluetoothGATTGetDescriptorValue, BluetoothGATTGetServices or BluetoothGATTSetCharacteristicValue were used.
This application, blessTags (BLE SensorTags) application, can be downloaded from the Windows Store Apps: https://www.microsoft.com/store/apps/9p054xsjjr1n. For more information, demo, practical applications, examples etc. please visit the following blog: http://www.blesstags.eu.
Windows 10 - Anniversary Update - Version 1607
This version of the Windows 10 operating system is the best one, from the point of view of Bluetooth Low Energy devices. It can pair without any problem with all SensorTags (regardless of the software version running on them), with which blessTags application knows how to work (CC2650STK, Thunderboard React, Thunderboard Sense and CC2541DK), and all the information from the Bluetooth’s Services Get Generic Access and Get Device Information is acquired without any problem.
Analyzing the data acquisition speed (for CC2650STK and CC2541DK devices) using notifying and reading mechanism of data transfer, we can observe the following:
If we will analyze Thunderboard React and Thunderboard Sense, we will get the equivalent results – they work without any problem in the Windows 10 Anniversary Update environment.
In fact, all the presentation movies of the blessTags application main functions and of the different specific features (like Gadgets) have been made with the support of the Windows 10 Anniversary Update.
A small demonstration and a proof of the above statements in the following movie:
Windows 10 - Creators Update - Version 1703
The Creators Update version of Windows 10 is the worst operating system (OS) from the point of view of Bluetooth Low Energy devices.
Almost nothing is working. Microsoft acknowledged that the Creators Update broke Bluetooth Low Energy (reference 1 and reference 2). The Microsoft company promised a hotfix as soon as possible. But since then they have released an updated version of Windows (Fall Creators Update) and nothing has happened – up to now within the Windows 10 Creators Update version, the Bluetooth Low Energy still does not work.
There are a large number of posts on forums in which different peoples complains regarding different types of Bluetooth devices that stop working after upgrading to Creators Update (see here, see here, see here, see here etc.).
The results, I'm going to show right away, were obtained after many tests: (1) on a desktop PC that had a CSR4.0 Bluetooth USB dongle (CSR8510 A10) and (2) on a Dell Inspiron P66F laptop with an integrated Bluetooth LE device. I know there are many solutions on the internet to fix several types of Bluetooth issues. I tried almost all, but nothing was working (update the Bluetooth driver, run Windows troubleshooter, disable and enable Bluetooth related services etc.).
So, let’s present the results:
The operating system has a strange behavior when the pairing process is initiated. In the list of discovered devices, the SensorTag appear and disappear (with a period of 1 … 1.5 s). Finally, when a mouse clicks succeed on the SensorTag, the pairing process accomplishes and the LEDs on the Thunderboard React (the blue and the green ones) have a period when they are flashing consecutively in an atypical mode.
The reading of the characteristics of the Generic Access Service (0x1800) can be done without any problem, but the reading from Device Information Service (0x180A) fails on all four existing characteristics.
Setting the sensors (embedded on SensorTag), the mode of acquiring data (on Thunderboard React you have only the following possibility: (1) to get data through the notification from 3 sensors and (2) to read data from the other four sensors) is impossible. Therefore, the impossibility of obtaining the actual data from sensors results directly from here.
The same pulsating process, observed for Thunderboard React, was found to be also existing for Thunderboard Sense – when we want to achieve the pairing process. But here, things are even worse: after pairing, the blessTag program cannot detect the SensorTag. So, no active device – no entity from where the blessTags application to acquire the data.
Also, I was able to get Generic Access data. But, at the Get Device Information service, from 9 characteristics only 6 responded and only from them it was possible to get information.
Instead, I cannot set up the device and I cannot retrieve data from sensors either through the read mechanism or through the notifications.
The behavior is identical with the behavior of CC2650STK (firmware version 1.40). At each connection attempt, you will get the following error message: "Try connecting your device again".
So, in conclusion, within this version of Windows 10 (Creators Update), it is impossible to communicate with any of the four types of SensorTags point out above. Consequently, I mention (once again) that here I have used the same software version that I also used in all test made on Windows 10 Anniversary Update.
Windows 10 – Fall Creators Update - Version 1709
This version of Windows 10 (1709 – OS Build 16299.19) is a huge step forward, compared with Windows 10 Creators Update (were on BLE almost nothing is working), but still has a long way to get to the level of Windows 10 Anniversary Update (1607) operating system.
But let's see why I made this statement:
In the same mode like in Windows 10 Creators Update, the SensorTag appear and disappear when we want to add a new Bluetooth device. The same behavior can be highlighted in the action center on Bluetooth’s quick action button were “Not connected" and "Thunderboard React" are displayed repeatedly (please see in the following movie this process starting from the time index 5.14 s). Immediately we can conclude that Thunderboard React is guilty, mainly due to a flawed implementation of the advertising mechanism by Silicon Labs engineers. But, searching on the internet, we will notice that other users reported the same problem to other types of BLE devices, after installing the Fall Creators Update – e.g. view this movie on YouTube.
After pairing the SensorTag, the blessTags application is not able to find the Thunderboard React device. So, at this point nothing is working: Generic Access and the Device Information services or data acquisition from the sensors embedded on Thunderboard React SensorTag.
The mode to behave is similar to the one of the Thunderboard React. This Bluetooth device is displayed and disappears repeatedly. When the pairing process succeeded, it is possible to take data from Generic Access Service. But from this point, nothing is working anymore.
I will treat these two devices here simultaneously because their behavior related to the Windows 10 (1709) operating system is similar.
The pairing operation and the reading, from the Generic Access and the Device Information services, are working perfectly without any kind of problems.
The problems only occur when we want to read information from the sensors. The data transfer mechanism through notifications does not work at all.
The only way to get data from the sensors, embedded in the SensorTag, is by means of the direct reading mechanism from the device. This approach has two issues: (1) lower data transfer speed (as we have shown above) and (2) if all the sensors accept one of the two data transfer methods (through reading and notification), the buttons on the SensorTag can be interrogated only through the notification mechanism. Thanks to this "feature" of the Windows 10 (1709) OS, the blessTags application implements, starting with version 22.214.171.124, the reading method for data acquisition also.
A problem appears with the CC2650STK SensorTag having the firmware version 1.20. If the process of pairing and data reading from Generic Access service works very well, the reading process from Device Information services is not possible. Moreover, the sensors reading (from this SensorTag with this firmware version) does not work through either one of the two possible mechanisms (reading or notification).
As a conclusion, up now on Windows 10 Fall Creators Update (1709, build 16229.19) only the SensorTags produced by TI (CC2650STK and CC2541DK) are working. More, they are working only in reading mode. But attention! Only CC2650STK firmware version 1.40 will work in this mode. Unfortunately, when you buy a CC2650STK you have a very high chance of taking a device with firmware revision 1.20. So, to be able to communicate with a such a type of SensorTag an upgrade it is necessary at least to the firmware version 1.40.
Below I present a movie that proves all these statements made above for Windows 10 Fall Creators Update.
Since the first release of Windows 10 Fall Creators Update (build 16229.19), on October 17th, 2017, there have been no improvements or errors corrections related with Bluetooth LE up to KB4054517 (released on December 12th, 2017). In KB4054517 (OS Build 16299.125) there is a key change on Bluetooth LE (see here): “Addresses issue with personalized Bluetooth devices that don't support bonding”. Since this message is very cryptic, I've decided to resume all my analysis donned so far and to see if there are any improvements compared to the first release of Windows 10 Fall Creators Update (build 16229.19). … and a little surprise, right now I am able to get: (1) data from Thunderboard Sense (from the sensors embedded on the SensorTag but only through the reading mechanism) and (2) all the information from Generic Access and Device Information services. There are no other improvements.
As a first Microsoft OS with BLE support, the implementation is satisfactory, but it is far to be an excellent one. The only devices that work with this operating system are CC2650STK and CC2541DK.
Setting the acquisition time to 150 [ms], for the CC2650STK, we can get the data (from all embedded sensors), complying the 150 [ms] sampling rate, through the notification mechanism without any problems. Unfortunately, using the CCC2650STK reading mechanism, we can get data (from all the sensors) with a period of 2 seconds.
The situation is getting worse when we are talking about CC2541DK. Through the notification mechanism, the data is obtained with a period of 0.4 ... 0.6 seconds. While using the reading mechanism we can retrieve the data with a fluctuating period of 2.8 ... 3 seconds. The conditions are the same: acquisition period 150 [ms] from all the sensors embedded on the CC2541DK SensorTag.
The Microsoft company has added support for the Bluetooth Low Energy (BLE) stack starting with the Windows 8 operating system. They have provided an API which enables applications to access BLE devices.
But the Microsoft has not ported the BLE API's to Windows 7. The Windows 7’s built-in stack supports only Bluetooth version 2.1/3.0, there is no support for BLE (4.0, 4.1 or 4.2). So, from the point a view of a developer it is impossible to communicate, in Windows 7, with a BLE device using Windows 7’s stack.
The TI company have a program called the BLE Device Monitor that is able: (1) to run on Windows 7 and (2) to communicate with a SensorTag. But you must use for these a special USB dongle (e.g. CC2540 Bluetooth Low Energy USB). If the source code for the USB dongle is free, the source code for the BLE Device Monitor is not available – it is only for the internal use of the TI company.
The Windows 10 Anniversary Update (Version 1607) is the best Windows version ever made by Microsoft from the point a view of Bluetooth Low Energy (BLE) devices – SensorTags in our case. Obviously, this is also due to the considerable number of improvements that took place at the Bluetooth LE level in the following OS builds (see for more info: https://support.microsoft.com/en-us/help/4000825): 14393.51, 14393.105, 14393.189, 14393.222, 14393.321, 14393.351, 14393.726 and 14393.1083.
The blessTags (BLE SensorTags) application can be downloaded from the Windows Store Apps: https://www.microsoft.com/store/apps/9p054xsjjr1n. For more information, demo, practical applications, examples etc. please visit the following blog: http://www.blesstags.eu.
Synthesizing all of the above results we will get the table below.
This project isn't really an end in itself, just a simple tool for working with the WGM110 Wi-Fi module, but it was suggested that I share it here, so I'm re-posting.
Sometimes it's useful to be able to pre-program modules before they're installed on a board, particularly if your board doesn't have space for a SWD header. The earliest versions of the WGM110 also had a defective DFU bootloader (stop bits were 1/2 bit time) that wouldn't work with some hosts, which made in-circuit programming difficult without SWD.
To address this I made a quick and dirty programming fixture for about $15. The components are a Mill-Max 854-22-010-40-001101 0.050" pitch 10-position pogo pin header to connect to the WGM110, a Samtec SFSD-10-28-H-05.00-SR cable with 20-pin 0.050" pitch plug for the JTAG side, some heat shrink tubing, and a 3D printed holder that I whipped up in Alibre.
This is not really the right JTAG connector - its polarity key is in the wrong place and it's latching - but it's the closest thing that Digi-Key happened to have in stock that had discrete wires. It fits on the dev board as is, and it'll fit on a P&E Cyclone's shrouded header if you snip off the key.
In the orientation shown in the top view photo, the pogo pins are wired to the following JTAG pins:
If I was going to be using this much I'd have machined mine from Delrin or anti-static ABS, but for only doing a few dozen units at most it didn't seem worth firing up a milling machine.
Without the holder piece, I can use the pogo pin plug to re-flash modules in circuit that have been bricked by DFU failures. This depends on having the PCB lands extend far enough beyond the periphery of the module to make contact. If you followed the recommended layout in the datasheet, it ought to work. I wouldn't want to have to do many boards this way without an alignment jig to hold it in place, but it works fine to just hold it in place if you're just fixing an occasional mistake and not doing production quantities.
Bottom view, with wiring to pogo pins:
The STL file and original model in Alibre format are attached. Keep in mind that I set the dimensions to make it work with my own 3D printer and the fit may require tweaking on yours.
It's no substitute for a proper in-circuit programming setup for production, but it's handy for prototyping and I figured I'd share it here in case anyone else needs such a gadget.
Power Deivery Protocol Introduction:
USB has evolved from a data interface capable of supplying limited power to a primary provider of power with a data interface. Today many devices charge or get their power from USB ports contained in laptops, cars, aircraft or even wall sockets. Users need USB to fulfill their requirements not only in terms of data but also to provide power to, or charge, their devices simply, often without the need to load a driver, in order to carry out “traditional” USB functions.
USB Power Delivery is designed to enable the maximum functionality of USB by providing more flexible power delivery along with data over a single cable. Also it enables alternative modes of operation by providing the mechanisms to discover, enter and exit Alternate Modes. The specification also enables discovery of cable capabilities such as supported speeds and current levels.
The USB Power Delivery specification defines how USB Devices may negotiate for more current and/or higher or lower voltages over the USB cable (using VBUS or CC wire as the communications channel). And defines mechanisms to discover, enter and exit Modes defined either by a standard or by a particular vendor. And also defines the mechanisms to discover the capabilities of cables which can communicate using Power Delivery.
Silabs has released the reference design for usb Type-c. Since more and more customers adopt our reference design, it's essential to investigate the Power Delivery specification, and implement a universal Type-C PD Analyzer base on our certified PD library.
The USB Power Delivery Analyzer (PD analyzer) base on EFM8BB3 can monitor the PD package data on the Control Channel lines CC1 and CC2, and don’t need intrusive the original system. It can aid to analyze the behavior of the whole USB Power Delivery System to promote the development of new USB PD product or troubleshoot the issue of the PD system.
The system architecture of PD analyzer connectors be illustrated in figure as below. The PD analyzer be connected to analysis computer through a USB-to-Uart Bridge for data transfer. The target device should connect the Analyzer’s type-c receptacle, and the plug of Analyzer should connect to another type-c device, typically, it is a source port.
The prototype of the PD Analyzer will be implemented with EFM8BB31F64G STK board.
The GUI tool be implemented with python + QT which be supposed compatible with major system. And packages the python programs into stand-alone executables with Pyinstaller.
The firmware be implemented base on Silabs' certified PD library
Main function of the PD protocol analyzer
Overview of the GUI
As below is the main perspective of the PD analyzer GUI, it comprises the following elements:
The message view window be consist of a list that displaying all USB protocol elements. The analysis software lists PD package on the left side of the display. Each PD package includes the SOP, Message ID, Number of Data Objects and Message Type information, also the payload and absolute time of each PD package be expressed in the view. And the PD package transfer direction is indicated with an arrow.
The Raw Data View will show more information about the selected power delivery package. In addition to the basic PD package information be provided by Message view, this display view allows the user to check the raw data (5B encoded) of each package, and all of the detailed information will be listed, also a brief description about each control message or data message be added to help the user understand the meaning of the PD package further.
Prototype of the PD Analyzer
The USB Power Delivery Analyzer (PD analyzer) device be implemented with EFM8BB3. EFM8BB3 has a Configurable Logic block which be consisted of multiple Configurable Logic Units (CLUs). CLUs are flexible logic functions which may be used for a variety of digital functions, such as replacing system glue logic, aiding in the generation of special waveforms, or synchronizing system event triggers. The USB PD analyzer adopt the CLU as a BMC decoder, the module can decode the BMC code independently of the CPU, has the benefit of reducing the workload of the CPU.
The PD analyzer device be consisted of three parts, EFM8BB3 STK board, CP210x USB-to-Uart bridge, a Type-C receptacle and plug connector. As below is the top view and bottom view of the PD Analyzer device.
The P0.1 of STK board should connect to GND for a reference GND. And P1.1 P1.2 is the input channel of CC pin, please connect them with the Type-c connector’s A5 and B5 pin. Also we should connect the P1.7 and P2.0 to the Rx and Tx pin of CP210x for data transaction.
Capture PD package data
Ensure that the PD Analyzer device is connected to the analysis computer and the power is on.
Launch the PD Analyzer application by double clicking “PD Analyzer.exe”
At the main window, click the Capture -> Connect menu to create the connection between Analyzer device and computer. The Connect menu will become disabled if connection be established, otherwise a warning message window will be popped to info the connection failed. And please reset the analyzer device and try to connect it again.
After ensuring the connection, please start to capture the PD package by clicking the Capture -> Run. And then connect the target Type-c device with the receptacle of the connector, and the plug of the connector should be connected to other Type-c source. For a quick demo, plug the USB Type-C power adapter into the receptacle, and plug the connector into the Macbook.
The overview of the PD Analyzer be illustrated as below.
Uploaded the PD Analyzer firmware project base on EFM8BB3 as attachement, please just program the the PD_Analyzer.hex to EFM8BB31F64G to make it a PD Analayzer device.
And then refer to sector "PIN Connection" for how to connect the EFM8BB3 STK and CP210x.
Attachement GUI_Tool.zip is the stand-alone executables GUI tool which can run in the Windows PC without installing.
This project is created by Silicon Labs’ summer intern Rikin Tanna.
A magnetic alarm system uses a Hall Effect magnetic sensor and a supporting magnet attached to the door frame to determine if the door is opened or closed. This project includes a notification service that sends a message to your mobile phone when the alarm is triggered, for added security. By removing any moving parts from the system, the magnetic alarm system proves to be very reliable.
EFM32 Giant Gecko GG11 Starter Kit (SLSTK3701A)
WGM110 Wi-Fi Expansion Kit (SLEXP4320A)
Hall Effect Magnetic Sensor (Si7210)
My goal with this project was to demonstrate an important use case for the Si7210 as a component of an alarm system. Given that Silicon Labs is an IOT company, I figured it would be beneficial to use IFTTT, an IOT workflow service that connects two or more internet-enabled devices, to demonstrate our GG11 STK being used with an established IOT enabler. With this service included, I could also showcase the WGM-110 Wi-Fi Expansion kit working with the GG11 STK. The GG11 STK was chosen due to its onboard Si7210 sensor, its compatibility with the WGM-110 Wi-Fi Expansion kit, and its recent launch (to expand its demo portfolio).
The demo is split into 3 phases:
The GG11 STK was programmed using Simplicity Studio. Simplicity provides an ample array of examples and demos to help any beginners get started with Silicon Labs MCUs (this was my first experience with SiLabs products).
Below is a representation of data flow for the project.
The WGM110 is a versatile part, as it can act as Wi-Fi client, a local access point, or a Wi-Fi Direct (Wi-Fi P2P) interface. In this system, the WGM110 acts as a client, and it is a slave to the host GG11 MCU. Communication is based on the BGAPI command-response protocol via SPI on a UART terminal. Debugging this proved to be difficult, as there are two individual MCUs involved, but the Salae Logic analyzer allowed me to view the communication between the devices to help fix any issues I encountered. Below is a capture of a typical boot correspondence.
When the alarm is triggered, the WGM110 establishes a TCP connection with the IFTTT server and sends an HTTP Get request to the extension specified by the IFTTT applet I have created. Unfortunately, IFTTT allows free users to only create private applets, but creating the applet was simple: step by step instructions for creating my applet can be found in the project ReadMe file.
The GG11 STK comes with an onboard Si7210 Hall Effect Magnetic sensor. It can detect changes in magnetic field to the hundredth of Millitesla (mT), which is more than enough sensitivity for this use case. The part has multiple OTP registers that store various part configurations, and the calibration process specified earlier writes to the register that determines the digital output threshold through I2C. The Si7210 also features a tamper threshold, in case someone tries to fool the alarm by using a second magnet to replace the original magnet as the door opens. This threshold is configured to be slightly greater than the original calibration threshold to detect even the slightest tamper. When either threshold is crossed, the part automatically toggles an OTP digital output pin, allowing any programmer to easily interface the sensor into their designs.
Using this Project:
This project provides a good starting point for anyone who wants to utilize the Si7210 Hall Effect sensor and/or the WGM110 Wi-Fi Expansion kit working in sync with the GG11 STK. The expansion kit can also be used with the PG1 or PG12 boards, but my code may require a few changes in initialization, depending on which specific peripherals are used.
Below is a slide that details all the various features that I utilized for each part. Feel free to download the project (link below) and use my code to get started on your own projects!
Other EFM32GG11 Projects:
This project is created by Silicon Labs’ summer intern David Schwarz.
A real-time embedded spectrum analyzer with a waterfall spectrogram display. The spectrum analyzer displays the most recently captured magnitude response, and the spectrogram provides a running history of the changes in frequency content over time.
The original intent of this project was to demonstrate real time digital signal processing (DSP) using the Giant Gecko 11 MCU and the CMSIS DSP library. Since many use cases for real time DSP on an embedded platform pertaining to signal characterization and analysis, I decided that a spectrum analyzer would be a good demonstration.
The spectrum analyzer works by capturing a buffer of data from a user selected input source: either the microphone on the Giant Gecko 11 Starter Kit (STK) or the channel X input of the primary analog-to-digital converter (ADC0) on the Giant Gecko 11 device. It then obtains and displays the frequency response of that data. The display also shows a spectrogram to give the user information about how a signal is changing over time. The format used here is a ‘waterfall’ spectrogram, where the X axis represents frequency, the Y axis represents time, and the color of the pixel at that coordinate corresponds to the magnitude.
Below is a video demonstration of the final project, the legend on the right shows how the spectrogram color scale relates to intensity.
There are two parts to the video. One is for the mic input using classical music. The other is sweeping the ADC input using a function generator.
The block diagram above shows the steps required to convert the incoming time domain data to visual content. Certain parts of the process demanded specific implementations in order to function in real time.
I found it necessary to implement dual buffering to allow for simultaneous data capture and processing, which allowed for lower overall latency without losing sections of incoming data.
The microphone data also required further processing to properly format the incoming bytes. This needed to be done post capture, as input data was obtained using direct memory access (DMA).
Finally, I chose to only normalize and display 0 to 8 kHz frequency data since most common audio sources, including recorded music, don’t contain much signal energy above 8 kHz. However, to avoid harmonic aliasing, I decided to oversample at a frequency of 34133 Hz. I used this specific sampling frequency in order to give me 512 samples (one of the few buffer sizes the ARM fft function supports) in 15 milliseconds. This 15 millisecond time constraint is very important for maintaining real-time functionality, as humans are very sensitive to latency when a video source lags audio.
Using This Project:
This project provides a good starting point for anyone wanting to implement real time DSP on the Giant Gecko microcontroller. It can be run on an out of the box Giant Gecko Series 1 STK, or it can be configured with an analog circuit or module that generates a 0 to 5V signal as the input source. The complete source code and Simplicity Studio project files are linked below, along with inline and additional documentation that should be useful in understanding how the application works.
The ADC input mode and DSP functionality of this project is also fully compatible with any Silicon Labs STK using an ARM Cortex-M4 core (eg. Wonder, Pearl, Flex, Blue, and Mighty Geckos). The microphone and color LCD, however, are not present on other STKs.
This project is created by Silicon Labs’ summer intern Cole Morgan.
Background and motivation:
This project is a program that implements voice recognition for the Giant Gecko 11 (GG11) using the starter kit’s temperature and humidity sensor and the Wizard Gecko Module. My motivation to work on this project was mainly that I wrote another project that implemented voice recognition for the GG11 using the starter kit’s LEDs, and I wanted a more advanced application for my voice recognition algorithm.
The program works by first learning your voice through a small training protocol where the user says each keyword a couple times when prompted. After it has learned the user’s voice, the user can set either a temperature or humidity threshold by saying “set” followed by either “temp” for temperature or “humid” for humidity. After this, the user can say a number from 0-99 one digit at a time to set the threshold value; for example, saying “one nine” would be interpreted as 19. For instance, saying “set humid four two” would set a humidity threshold at 42% humidity. Then, if the humidity measured by the onboard sensor crosses this threshold, the user will receive a text.
Using my previous voice recognition project as a base, I first added the support for multiple word commands using the first command word “set” as a kind of trigger so that the program won’t get stuck in the wrong stage of a command. One side effect of using a lot more keywords than the previous project was that I had to stop storing the reference MFCC values in Flash, as there wasn’t enough space for all of them.
The next stage in my development was to interface the Si7021 temperature/humidity sensor on the GG11 starter kit. This stage was quite simple because there was already a demo for the GG11 that interacted with the Si7021, so all I had to do was integrate the LCD.
Then, I interfaced the Wizard Gecko Module (WGM) to connect to IFTTT via Wi-Fi and send an HTTP GET request. This part was the most difficult of this project because I have never worked with communication over Wi-Fi or sending HTTP requests. I designed two different IFTTT triggers for temperature and humidity so that the SMS alert message could be tailored to the type of threshold trigger.
Potential Use Cases:
This project is created by Silicon Labs’ summer intern Cole Morgan.
Background and motivation:
This project is a program that implements voice recognition for the GG11 using the starter kit’s onboard LEDs. My motivation to work on this project was mainly that I have never done anything remotely close to voice recognition before, and I thought it would be a good challenge. But another motivation was also that I am very interested in the Amazon Echo and the other emerging home assistant technologies.
The program works by first learning your voice through a small training protocol where the user says each keyword a couple times when prompted. After the program has learned the user’s voice, the user can turn the LED on, red, blue, green, or off simply by saying “on”, “blue”, “red”, “green”, or “off”.
My first step was getting audio input from the microphone into the microcontroller and storing it. This proved a little more difficult than I expected because I hadn’t worked with SPI or I2S before. In addition to this, I also had to design a sound detection system that captures as much significant sound as possible. I did this by squaring and summing the elements of the state buffer of the bandpass FIR filter that I apply on each sample and then setting a threshold for the result of that operation. This system turned out to be extremely useful because, in addition to saving processor time, it also time-aligned the data to be processed.
After this step, I began to implement the actual voice recognition. At first, I thought I could just find a library online and implement easily, but this turned out to be far from true. Most voice recognition libraries are much too big for a microcontroller, even one with a very large Flash memory of 2MB like the GG11. There was one library I found that was written for Arduino, but it didn’t work very well. So, I began the process of writing my own voice recognition algorithm.
After a lot of research, I decided I would use Mel’s Frequency Cepstral Coefficients (MFCCs) as the basis for my algorithm. There are a number of other audio feature coefficients, but MFCCs seemed to be the most effective. The calculation of MFCCs is basically several signal processing techniques applied in a specific order, so I used the CMSIS ARM DSP library for those functions.
After beginning work on this, I created a voice training algorithm to allow the program to learn any voice and adapt to any user. The training program has the user say each word a configurable number of times, and then calculates the MFCCs of that person’s pronunciation of the keyword and stores them in flash memory.
Next, because the input data was time-aligned, I could simply put all the MFCCs for the 4 buffers in one array and use that as the basis for comparison. In addition to this, I also calculated and stored the first derivative (delta coefficients) of the MFCC data to increase accuracy.
Lessons Learned and Next Steps:
My next steps are to apply the voice recognition to a temperature / humidity controller application, which should be easier than this LED application as the keywords are very different from each other unlike “on” and “off”.
This project is created by Silicon Labs’ summer intern Kevin Black.
The goal of this project was to perform one-way, encrypted, real-time, wireless voice communication from an embedded system to an arbitrary client like a laptop or tablet. This was accomplished using the EFM32GG11 starter kit for audio input/processing and the Wizard Gecko Wi-Fi expansion kit for wireless transmission. Audio data is sampled from the starter kit’s onboard microphone and encrypted with AES using the GG11 32-bit MCU; it is then streamed to any clients connected to the Wizard Gecko’s Wi-Fi access point, where it can be decrypted and played back only with the correct password.
Background and Motivation:
My project primary purpose was to demonstrate useful features of both the EFM32GG11 starter kit and the Wizard Gecko Wi-Fi expansion kit, as well as the two working smoothly in conjunction through the EXP header.
The first main feature it demonstrates is the EFM32GG11’s CRYPTO module, which exists on all the EFM32 Series 1 devices and provides fast hardware-accelerated encryption. The project utilizes the mbed TLS library configured to use the CRYPTO module, which speeds it up significantly. It demonstrates the high throughput of the CRYPTO module (up to ~123 Mbps max*) by encrypting uncompressed audio in real time with plenty of overhead. The type of encryption is 256-bit AES in CBC mode, which is currently considered universally secure.
(*Assuming 256-bit AES on the GG11 driven by HFRCO at 72 MHz)
Another motivation behind the project was to demonstrate two features of the GG11 starter kit itself: the onboard microphone, and the ability of the Wi-Fi expansion kit to easily attach to and be controlled through the EXP header. No examples existed for the microphone, and very few firmware examples existed for the Wizard Gecko in externally hosted mode. My projects demonstrate the quality of the built-in microphone by allowing the user to listen to the audio, as well as shows how to use the BGLib C library to communicate with the Wizard Gecko from an external host. Additionally, it demonstrates the throughput of a transparent/streaming endpoint on the Wizard Gecko.
Block diagram of data flow through transmitter device
The GG11 starter kit provides an onboard audio codec that automatically converts the PDM (pulse density modulation) data from the onboard MEMS microphones into PCM (pulse code modulation) data and outputs it on a serial interface in I2S format. The codec’s serial interface is connected to the GG11 USART3 location 0 pins, so reading in the audio data is simply a matter of initializing USART3 to I2S with the correct settings, enabling autoTx, and asserting an additional microphone enable pin.
The audio data arrives in 32-bit words, so the sample rate is controlled by setting the I2S baud rate to 64 times the desired sample rate (2 channels, 32 bits each). Each word contains a single 20-bit sample of audio, but very few systems support 20-bit audio, so for my project, I ignore the least significant 4 bits of each sample and only read 16 bits from each word. I also ignore samples from the right microphone, meaning the final audio data I obtain for processing is in 16-bit mono PCM format. The sample rate is easily configurable, but in the end, I settled on 20 KHz as that seems to be the upper limit of what the Wizard Gecko can handle while being high enough to cover the range of human hearing and provide clear and understandable audio.
The audio input data is transferred into memory using LDMA in order to save CPU cycles. The right channel data is repeatedly written to a single byte in order to discard it, while the left channel data is alternately transferred into two 16-byte buffers; when one buffer is being filled, the other is being processed by the CPU.
Encryption & Transmission:
When a left channel transfer completes, it triggers an interrupt that switches the current process buffer and signals that the next packet is ready to be processed. The GG11 then encrypts the current 16-byte buffer (16 bytes is the AES block size) using the mbed TLS library configured to use the CRYPTO module. In CBC (cipher block chaining) mode, the library automatically XORs the plaintext with the previous ciphertext before encryption.
The 256-bit key used for encryption is derived from a password using SHA-256. Only clients with the same password can obtain the correct key by hashing the password.
In my project, I decided to fix the initialization vector as all zeros. Normally, initialization vector reuse is considered bad practice and weak security; however, it only has the potential to leak data from the first few blocks of data streams with identical prefixes, and that poses an insignificant threat to my project due to the enormous quantity of blocks and the amount of noise in a meaningful segment of audio.
Once a block is encrypted, it is put into a first-in-first-out queue where it is transmitted over UART through the EXP header to the Wizard Gecko. Flow control is implemented using an additional CTS (clear to send) pin connected to the Wizard Gecko; the module can drive CTS high when it cannot keep up with the transmission rate, in which case the transmission halts and the queue fills up. The transmission is driven by interrupts, which allows it to run “in the background” while the next buffer is being encrypted, and does not block the main thread when the Wizard Gecko raises CTS.
The baud rate for UART transmission is configurable as long as the GG11 and the Wizard Gecko are both configured to the same value. Interestingly, however, the Wizard Gecko seemed to perform better (raise CTS for less time) at higher baud rates— perhaps because that increases the gap between packets— so I settled on 3 MHz.
The Wizard Gecko Wi-Fi module, when connected to an external MCU in hosted mode, operates in a command-response format. The GG11 sends commands through the EXP header via SPI, formatted with a binary protocol called BGAPI. When the Wizard Gecko is ready to send a response (or an event) back to the MCU, it raises a notify pin (also connected to the EXP header) that tells the GG11 to read and handle the message. All of the BGAPI commands and responses are defined in a C library called BGLib.
Upon initialization, my project configures the Wizard Gecko to be a hidden wireless access point and a TCP server. When a client connected to the access point opens a connection to the IP address and port of the TCP server, it triggers an event that is forwarded back to the GG11. The GG11 then enables the microphone and begins encrypting and transmitting audio via UART to the Wizard Gecko’s second USART interface (the one not used for BGAPI commands). That interface is configured in transparent/streaming mode, which means it forwards all received data unmodified to a single endpoint. Before the encryption starts, the GG11 configures this endpoint to be that of the connected client.
Accomplishments, Flaws, and Next Steps:
Ultimately, the project was successful and met its end goal of building a one-way encrypted voice communication device. Speech is clear and comprehensible at up to several inches away from the onboard microphone, and the real-time encryption is secure.
The primary flaw in the final implementation is that the Wizard Gecko itself has trouble constantly streaming a large quantity of data without interruptions. The module will occasionally “choke” for 1-2 seconds, during which it will stop transmitting and refuse to accept data by raising CTS. Performance is inconsistent, and the device will go anywhere from 10 to more than 60 seconds in between “chokes”. This causes frustrating gaps in the audio, much like a cell phone connection that is “breaking up”; although on average, the project is still quite usable for talking to someone. I added a blue LED that turns on whenever CTS is raised, so the user can at least tell when the device is not transmitting by observing the LED light up solid blue.
In the future, this behavior could likely be eliminated by changing the protocol that the device uses to transmit. Bluetooth would have much more bandwidth, or if the Wizard Gecko is still used, Wi-Fi Direct or a TCP connection over a third-party local area network (rather than using the Wizard Gecko as the access point). The last two options would make the demo much more difficult to use, so Bluetooth would be the ideal solution; this explains why Bluetooth has become so popular for real-life products with similar functionality.
Using this Project:
Follow the instructions in the readme of the encrypted voice transmitter folder to configure the Wizard Gecko and GG11 to act as the transmitter portion of the project.
To use the receiver, download the executable Java applet below and run the .exe file inside (no JVM installation required). Unless the IP address and port were changed in the firmware, leave those fields blank. Enter the password defined in the firmware (default “gecko123”).
After booting up the transmitter, wait for the LCD output to reach “waiting for client”, and then connect to the hidden access point that the device has created (default SSID is “Encrypted Voice Demo”).
Once the LCD displays “client joined”, click “Connect” on the Java applet’s dialog. When the status message below the connect button displays “Connected” in green, audio from the microphone should begin playing back on the PC.
[zip file containing encrypted_voice_transmitter (firmware source code)]
[zip file containing executable Java applet]
[zip file containing encrypted_voice_receiver (Java source code)]