Creating a Windows Universal app to talk Bluetooth LE, save to SQLite and expose a REST service
The Goal
I’ve had a couple of TI SensorTags sitting on my shelf for a couple of years. These are the original ones, which have been superseded by smaller ones that have additional sensors for light and sound.
They are wonderful devices. They last for over a year on a watch battery, they talk Bluetooth LE, and they have loads of sensors including Temperature (both spot temperature of a nearby object, and overall ambient temperature), Gyroscope, Accelerometer, Magnetometer, Barometer, Humidity, etc.
Last, but not least, they cost less than US$30. Unless you actually enjoy wiring physical sensors into an Arduino or Raspberry PI, I think Sensor Tags are a great way to start collecting all kinds of information.
Rather than have a phone sitting talking Bluetooth LE, I decided I wanted to use a Mac Mini server that I have running Windows, which I could run continuously to capture, store and serve the sensor information.
My goal was to:
- Create a Windows Universal App that talks Bluetooth LE to the Sensor Tag
- Save the captured information to an SQLite database
- Serve the captured information using REST (
/GetTemperatures?start=201501010000&end=201701010000)
At each step I hit roadblocks, and the purpose of this blog post is to try to capture what I did to overcome them, in the hope that other people may benefit from my pain.
Although I’ve been mainly writing Java/Android, C, TypeScript and JavaScript over the last three years, I still retain a soft spot for C# and the associated tooling of Visual Studio and Resharper.
I really appreciate the C# syntax and associated features such as lambdas, and LINQ.
I wanted to try my hand at create a Windows app, to see how much I’d lost over the last few years.
Bluetooth LE, SensorTag and Windows Universal
I started off creating a new Windows Universal app in Visual Studio. I browsed the documentation, and found the classes associated with using Bluetooth LE. I liked the fact that my app would be able to run on desktops down to phones.
My initial code:
_watcher = new BluetoothLEAdvertisementWatcher();
_watcher.Received += BluetoothReceived;
_watcher.Stopped += BluetoothStopped;
_watcher.Start();
When I ran this, I got the following exception: onecoreuap\drivers\wdm\bluetooth\user\winrt\common\devicehandle.cpp(100)\ Windows.Devices.Bluetooth.dll!51D26D1B: (caller: 51D273AE) Exception(1) tid(13c0) 80070005 Access is denied.
Turns out I needed to add Bluetooth to my app’s capabilities by double-clicking the Package.appxmanifest file in the Solution Explorer, going to Capabilities and checking Bluetooth.
Once that was done, I was able to look for the SensorTag’s Service UUID, and then check for the correct characteristics and enable the reception of the sensor’s data:
const string BaseUuidStart = "f000";
const string BaseUuidEnd = "-0451-4000-b000-000000000000";
const string TempData = "aa01";
const string TempConfig = "aa02";
const string AccelData = "aa11";
const string AccelConfig = "aa12";
const string HumidData = "aa21";
const string HumidConfig = "aa22";
const string MagnetData = "aa31";
const string MagnetConfig = "aa32";
const string BaromData = "aa41";
const string BaromConfig = "aa42";
const string GyroData = "aa51";
const string GyroConfig = "aa52";
private bool _attaching;
private readonly List<BluetoothLEDevice> _devices = new List<BluetoothLEDevice>();
private readonly List<GattCharacteristic> _characteristics = new List<GattCharacteristic>();
private async void BluetoothReceived(BluetoothLEAdvertisementWatcher sender,
BluetoothLEAdvertisementReceivedEventArgs args) {
if (_attaching) return;
try {
var device = await BluetoothLEDevice.FromBluetoothAddressAsync(args.BluetoothAddress);
_devices.Add(device);
_attaching = true;
device.ConnectionStatusChanged += DeviceConnectionStatusChanged;
device.GattServicesChanged += DeviceGattServicesChanged;
foreach (var service in device.GattServices) {
var serviceUuid = service.Uuid.ToString().ToLowerInvariant();
if (!serviceUuid.StartsWith(BaseUuidStart) || !serviceUuid.EndsWith(BaseUuidEnd)) {
continue;
}
foreach (var characteristic in service.GetAllCharacteristics()) {
var characteristicUuid = characteristic.Uuid.ToString().ToLowerInvariant();
if (_characteristics.Any(c => c.Uuid.ToString() == characteristicUuid)) {
continue;
}
var characteristicType = characteristicUuid.Substring(BaseUuidStart.Length, 4);
switch (characteristicType) {
case AccelData:
case BaromData:
case HumidData:
case GyroData:
case MagnetData:
case TempData: {
_characteristics.Add(characteristic);
characteristic.ValueChanged += CharacteristicChanged;
var status =
await characteristic.WriteClientCharacteristicConfigurationDescriptorAsync(
GattClientCharacteristicConfigurationDescriptorValue.Notify);
Debug.WriteLine("Subscribed .... with status " + status);
break;
}
case AccelConfig:
case BaromConfig:
case HumidConfig:
case GyroConfig:
case MagnetConfig:
case TempConfig: {
var status = await characteristic.WriteValueAsync(new byte[] {1}.AsBuffer());
break;
}
default:
Debug.WriteLine("Ignoring characteristic: " + characteristicType);
break;
}
}
}
sender.Stop();
}
catch (Exception ex) {
Debug.WriteLine("got " + ex);
}
}
I used the Sensor Tag documentation to know about the GUIDs used for the services and characteristics.
I found I needed to press Advertise the button on the side of my Sensor Tag to get it to be seen.
Capturing the values was pretty easy, but I did hit one stumbling block which was the temperature. There is an algorithm described in the documentation as to how to transform the series of bytes received into the spot and ambient temperature in degrees Celsius. When I tried using it I got garbage values, but eventually found this C# example showing how they can be calculated:
private async Task ProcessTempData(string bluetoothId, byte[] rawData) {
// Extract ambiant temperature
var ambTemp = BitConverter.ToUInt16(rawData, 2)/128.0;
// Extract object temperature
int twoByteValue = BitConverter.ToInt16(rawData, 0);
var vobj2 = twoByteValue*0.00000015625;
var tdie = ambTemp + 273.15;
const double s0 = 5.593E-14; // Calibration factor
const double a1 = 1.75E-3;
const double a2 = -1.678E-5;
const double b0 = -2.94E-5;
const double b1 = -5.7E-7;
const double b2 = 4.63E-9;
const double c2 = 13.4;
const double tref = 298.15;
var s = s0*(1 + a1*(tdie - tref) + a2*Math.Pow(tdie - tref, 2));
var vos = b0 + b1*(tdie - tref) + b2*Math.Pow(tdie - tref, 2);
var fObj = vobj2 - vos + c2*Math.Pow(vobj2 - vos, 2);
var tObj = Math.Pow(Math.Pow(tdie, 4) + (fObj/s), .25);
var objTemp = tObj - 273.15;
await SaveTemperature(bluetoothId, ambTemp, objTemp);
}
SQLite and Windows Universal
Installing SQLite for Windows was pretty easy, but I couldn’t find clear, complete instructions. In short I used NuGet to install
- SQLite.Net-PCL
- SQLite.Net.Async-PCL
- SQLite.Net.Core-PCL
Once I had this installed, I could define classes corresponding to the tables I wanted to create, such as:
public class Temperature
{
public int Id { get; set; }
public DateTime Timestamp { get; set; }
public string BluetoothId { get; set; }
public double Ambient { get; set; }
public double Spot { get; set; }
}
Then I could initialize the database:
private SQLiteAsyncConnection _asyncConnection;
private async Task InitializeDatabase() {
Debug.WriteLine("Initializing database");
var databasePath = Path.Combine(Windows.Storage.ApplicationData.Current.LocalFolder.Path, "sensortag.db");
var connectionFactory = new Func<SQLiteConnectionWithLock>(() => new SQLiteConnectionWithLock(new SQLitePlatformWinRT(), new SQLiteConnectionString(databasePath, true)));
_asyncConnection = new SQLiteAsyncConnection(connectionFactory);
await _asyncConnection.CreateTablesAsync(typeof (Temperature));
Debug.WriteLine("Initialized database");
}
And then write the data:
private async Task SaveTemperature(string bluetoothId, double ambTemp, double objTemp) {
var temperature = new Temperature {
Timestamp = DateTime.Now,
BluetoothId = bluetoothId,
Ambient = ambTemp,
Spot = objTemp
};
Debug.WriteLine("Writing temperature");
await _asyncConnection.InsertAsync(temperature);
Debug.WriteLine("Wrote temperature");
}
It turns out this was wrong, though it is what was shown in the Stack Overflow posts I found. The reason that it is wrong is that it is creating a new database connection each time the factory lambda is invoked. When I used this code all would run fine for a while, until eventually I hit an SQLite Busy exception:
Exception thrown: &'SQLite.Net.SQLiteException&' in mscorlib.ni.dll
SQLite.Net.SQLiteException: Busy
at SQLite.Net.PreparedSqlLiteInsertCommand.ExecuteNonQuery(Object[] source)
at SQLite.Net.SQLiteConnection.Insert(Object obj, String extra, Type objType)
at SQLite.Net.SQLiteConnection.Insert(Object obj)
at SQLite.Net.Async.SQLiteAsyncConnection.<>c__DisplayClass14_0.<InsertAsync>b__0()
at System.Threading.Tasks.Task`1.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
The simple solution was to create a single database connection instance, and serve that, rather than continually serving new ones:
private SQLiteAsyncConnection _asyncConnection;
private SQLiteConnectionWithLock _sqliteConnectionWithLock;
private async Task InitializeDatabase() {
Debug.WriteLine("Initializing database");
var databasePath = Path.Combine(Windows.Storage.ApplicationData.Current.LocalFolder.Path, "sensortag.db");
_sqliteConnectionWithLock = new SQLiteConnectionWithLock(new SQLitePlatformWinRT(), new SQLiteConnectionString(databasePath, true));
var connectionFactory = new Func<SQLiteConnectionWithLock>(() => _sqliteConnectionWithLock);
_asyncConnection = new SQLiteAsyncConnection(connectionFactory);
await _asyncConnection.CreateTablesAsync(typeof (Temperature));
Debug.WriteLine("Initialized database");
}
Exposing a REST Service from Windows Universal
This was supposed to be trivially easy. I’ve done plenty of WCF in the past, and know how ridiculously straightforward it should be to expose a REST service from an app. Except that Windows Universal doesn’t currently support WCF.
I went searching and found Restup, currently in Beta, which aims to expose REST endpoints for Windows Universal apps.
I used NuGet to install it. I had to check the Include prerelease option because it was currently in beta.
Setting up was pretty easy:
private async Task InitializeWebServer() {
await InitializeDatabase();
var webserver = new RestWebServer(); //defaults to 8800
webserver.RegisterController<SensorTagService>(_asyncConnection);
await webserver.StartServerAsync();
}
[RestController(InstanceCreationType.Singleton)]
class SensorTagService {
private readonly SQLiteAsyncConnection _connection;
public SensorTagService(SQLiteAsyncConnection sqLiteAsyncConnection) {
_connection = sqLiteAsyncConnection;
}
[UriFormat("/GetTemperatures\\?start={start}&end={end}")]
public async Task<GetResponse> GetTemperatures(string start, string end) {
Debug.WriteLine("got temp request");
...
}
}
Note the escaping of the question mark in the UriFormat? I wanted to pass parameters to my endpoint, rather than use values that are part of the path, but all the RestUP examples showed values in the path. I eventually came up with this solution, however it may be unnecessary by the time you read this.
Once again the security model bit me, and I got the following exception:
An exception of type &'System.UnauthorizedAccessException&' occurred in mscorlib.ni.dll but was not handled in user code
WinRT information: At least one of either InternetClientServer or PrivateNetworkClientServer capabilities is required to listen for or receive traffic
Additional information: Access is denied.
Once again I edited the app’s capabilities by double-clicking the Package.appxmanifest file in the Solution Explorer, going to Capabilities and checking
Internet (Client),Internet (Client & Server)andPrivate Networks (Client & Server)(so that I could use my service on my home network).
Accessing a local Windows Universal app from your web browser
Try as I might, I was not able to use my local Chrome browser to access my service. I resorted to using a totally separate machine to invoke my service. I used the CheckNetIsolation tool. I ensure that the Allow Network Loopback option was set for my project Visual Studio. I turned off my firewalls. Nothing!
Conclusions
The Bluetooth side of things was quite easy, but exposing a REST API was far too hard, despite the sterling work of Tom Kuijsten and the Restup project. Not being able to access my service locally was a complete pain - the Windows Universal restrictions on being able to be accessed from the local host seem strange - almost as though they are trying to stop you from building traditional apps that talk to Windows Universal apps …
In the end I’ll likely use the Windows Universal app to capture the SensorTag data via Bluetooth LE, and then create a Node.JS app to serve it over REST, sharing the same SQLite database, with code to handle retrying if the database is busy when inserting new values.
I’ll also push the data to a Node-RED instance to act on the data.