Is it posible to update the app via OTA with a Thunderboard Sense?
I have tried to implement this as described in various documents (AN1045, AN1086) using a python based program to do the actual uploading. The whole process seems to work (boot to DUF mode, discover characteristics, write to OTA_Data handle etc.) but at the end, the device reboots into the same firmware as before.
Is the bootloader expecting some sort of security or something? I read through the bootload info (UG266) and it is very confusing. Could not figure out how to make and flash a new bootloader (or rather I can build one, but flashing... not sure what to do with the "single, combined" etc.), plus the slots do not look right.
Anyone got any suggestions on how this works?
Yes you can update the Thunderboard firmware via OTA.
There are two kind of OTA bootloaders: the legacy bootloader, which is added to the application by default at build time, and the Gecko bootloader, which has to be added afterwards. The former needs .ebl files and the latter needs .gbl files to be uploaded. I suggest you using the Gecko Bootloader, which has much more features (e.g. security), but you can go with the legacy one as well.
If you choose the legacy bootloader, you should simple follow the instructions of AN1045.
If you choose the Gecko bootloader, I suggest you reading these articles:
The flow is the same as in case of the legacy bootloader (AN1045), the only difference is using .gbl files instead of .ebl files.
Anyway, before implementing your uploader, I strongly suggest testing OTA upload with smartphone or with our OTA DFU host example. Have you tried these?
When using your uploader, are you sure you never got any error response while uploading the .ebl file via the OTA_data characteristic?
im sort of assuming the OTA bootloader is there, as I can see it boot into OTA mode, and I can read the stack and OTA version.
I do have to manually add all the OTA stuff to the source code, it’s not there by default.
The tb Sense 1 returns OTA 1, and the sense2 returns OTA 2.
There is some slight confusion over the OTA sequence, as one document says that you send 0 to the OTA control point at the start, send the data (to the data), then send 3 to the control point to indicate end of OTA. The other document makes no mention of this (just send the data to the data characteristic). I’m thinking this is one of the differences between OTA 1 (legacy) and OTA 2 (current).
I can’t be absolutely sure that I don’t get a write error writing the data (and it does send very fast), as I’m using write without response, but if I send more data than expected, I do get a write error (ie I continue to send data, even after the end of the file - I was thinking maybe the data needs padding out to 20 or 55 bytes).
As to your suggestion to use the smartphone or host examples, I would if I could, but I don’t have two WSTK kits, and after all, how hard can it be? LOL.
I feel like I’m missing something fundamental, as the mechanics seem to work. Ie does the external flash have to be enabled? Where is the data being written on tb Sense 1 as there does not seem to be enough space on the internal flash, without overwriting the current program (which it doesn’t do).
does anyone have a working example for tb Sense 1 or 2 that I can look over?
The sort of thing I’m struggling with, is that AN1045 says (in a kind of casual way) that the file att.h must be included in all OTA projects. Needless to say, this file does not exist.
att.h and att_def.h
AAT (Application Address Table) is a support structure (meta data) for the firmware update file format EBL, defining the size and loca- tion of the application within the upgrade image. It is the first item in the application image and has pointers to the application vector table. The stack supervisor will initialize the application vector table and call the stack location from the vector table, before calling the application.
While ATT does not define the size and location of the application within GBL upgrade images, it is still required. Therefore, AAT is a mandatory part of all projects, and applications must include the aat.h header file.
This confusing statement I believe means that the file aat.h (not att.h) has to be included for OTA to work. This file I have included, and does exist.
It is also unclear what you have to send to start OTA upgrades. Several sources say to send 0x0 to the OTA control point (but as far as I can see, you can send anything, as long as it causes the boot to OTA DFU mode). One document then says that you have to send 0x0 to the control point to start OTA transfer, where other documents do not mention this. So now it’s not clear if you have to send 0x0 twice, or just once (once to boot to OTA DFU, and once again when in DFU mode to indicate start of upload).
AN1045 says that after the data is sent, you send 0x3 to the control point, which will return 0x0000 if the CRC is correct, or 0x0480 if incorrect, and will remain in DFU mode (if incorrect). In fact my device does neither of these, and simply reboots into the existing app. Now I’m using write without response to write the control value, as if I use write with response, I get no response.
I’ve tried SDK 2.4 and 2.6, and both do the same with .ebl or .gbl files.
The size of the data you can write is 20,55, or 244 bytes depending on the SDK you are using, but still doesn’t work (but no write errors).
These are just a few examples, there are many confusing and contradictory statements like this, but I’m probably overthinking it, and there is some simple thing preventing my OTA update from working.
Thats why I’m looking for a simple example that will work on Sense 1 or 2 as the examples (like thermometer) don’t work on the Mighty Gecko (or Blue Gecko).
Any clarification is appreciated.
Here is what I have found out, hope it heps other people.
First, I don't know what the truth is about att.h or aat.h but I ignored it, and everything was OK (aat.h was included).
Follow the instructions in AN1045 to a point... ie include the OTA Control Point in the GATT table, OTA config in the gecko_init() structure, and include the reboot to OTA mode trigger
Now the fun begins.
What you have to do is to send anything to the control point (which could be 0x00) to trigger boot to OTA, now connect, and discover the handles for OTA contol and OTA data.
Here are the key points using tb sense 1 (legacy bootloader):
If anyone is interested I have a python program using the excellent bluepy module which does OTA updates. Working on the legacy bootloader (tb sense 1), optimistically soon to be working on the geko bootloader (tb sense 2) .
As always, YMMV.
Now have this working on the tb a Sense 2 using the gecko_bootloader.
The mechanics are the same, but the bootloader is confusing. Some documents say that the gecko_bootloader is not present by default on devices like the MG12 (as used on the Sense 2), but there is instead a “dummy bootloader”. I don’t understand the logic of this - what is the reason for a “dummy bootloader”?
Anyway, I started with a Sense 2 as is. You can set up for OTA, and can boot into OTA mode, (get the OTA version, Stack version etc) but when you upload, the device just reboots into the previous code. There are no errors, other than at the end (when you send the 0x3), which is the same as a regular failed upload.
Ok maybe this is the dummy bootloader. So I built a new bootloader, combined it with a stack and app, and flashed it.
Now when I upload OTA, it works (only with 20 bytes per transfer, every other size fails). I can tell it’s working, because if the upload fails (and it fails a lot), the board is left in OTA mode, it doesn’t reboot into the app.
In fact, I can erase the flash, just upload an app, and it still works. Even without uploading another bootloader. I can’t make it not work (for 20 bytes at a time, as I say, every other size fails).
So I’m left with some confusion. If, as I suspect, the bootloader is not initially present, but is after you flash it. How is it still there if I flash to location 0x0000 again? And why would you include a dummy bootloader which works just like the real boot loader (except for not doing anything) - with no way to tell the difference between the two? How do you tell the difference between a real OTA bootloader and a dummy one?
Unless I’m wrong about all this, and the Sense 2 app does include a bootloader (which I couldn’t get to work for other reasons) which is what I think is the case, as the other scenario doesn’t make sense.
First of all thank you for summarizing your findings. I thinks that's quite helpful for everyone.
As I see you started with the ThunderBoard Sense sample app. Indeed, this sample app is not prepared for OTA by default. It is easier to start with the SOC-empty sample app, which is prepared for OTA. But at least now you know how to add OTA to any custom application.
Answering your questions:
1) The device is shipped with a dummy bootloader, because the Gecko Bootloader can be configured in a number of ways. E.g. you can use OTA, SPI, UART BGAPI, UART XMODEM, external flash, etc. bootloaders. There is no default configuration, everybody can choose according to their needs. The dummy bootloader does nothing, just starts the application.
2) In EFR32MG1 devices the bootloader starts at 0x0000, and the application starts at 0x4000, hence if you re-flash the device, the bootloader will be overwritten as well as the application. In EFR32MG12 devices the bootloader is stored at a dedicated bootloader area, starting at 0x0FE10000, and the application starts at 0x0000. Hence if you re-flash the device with the application, only the application will be overwritten. To overwrite the bootloader area, you have to flash it separately. Or use an .s37 file that contains both bootloader and application.
3) The OTA functionality is implemented partly in the bootloader and partly in the stack. The bluetooth packets are received by the stack, as the bootloader is not capable to handle Bluetooth at all. Then the received packets are parsed and written to the flash by the bootloader, as the stack does not know where to write them. If you have a dummy bootloader, the stack may still receive the packets, but it will not write them to the flash.
On a lower level the following happens: the bootloader starts and it starts the stack. If the stack realizes that the device was rebooted in normal mode, then it starts the user application. If the stack realizes that it was rebooted in OTA mode, then it starts a custom app (called the supervisor) within itself that handles the OTA process. It connects, receives packets, etc. Then it calls the bootloader to write the packets to the proper area.
I hope this answers your questions, but if not, feel free to ask.
Thank you Arnold!
This explains a lot. I think I now have a reliable OTA host program working. I have had it working at 244 bytes/write as well, but there are a lot of variables, especially with timing, signal strength etc.
My only real question now is how do you tell if you have a dummy bootloader installed or a real one? I couldn't tell from the host end (the responses are the same), but the real one works, and the dummy one doesn't. is there some characteristic to query, or a response you can get that tells you this is just a dummy, and not a real bootloader?
The system_boot event of the BLE stack contains the bootloader version. I haven't checked it with a dummy bootloader yet, but I suppose it contains 0 value in case of dummy bootloader. I'll check.
I would be interested in your python script using bluepy for updating the Thundersense with the Gecko Bootloader. Is there any chance you could provide that script?
Thank you in advance.