Bluetooth link

Bluetooth as method to communicate between devices has fairly big stigma of being unreliable. This is not completely undeserved, but based on my observations it is (slowly) getting better. Unfortunately there's still a long way to go.

In my case the link is typically RFCOMM (serial emulation over bluetooth) where a device communicates with PDA or phone. On device side I've used BlueGiga's modules (not the cheapest option) but so far they have never failed me, working perfectly. Not to mention the their iWRAP firmware/command system that is simply pleasure to work with as it does all the hard work for you.

Phones, on the other hand, have been pretty much hopeless. The major issue seems to be their operating systems' bluetooth stack (and/or chipset). After some connect/disconnect cycles (like phone being moved out of range, temporary power losses at device end resetting links and so on) phones traditionally always crap out, requiring at best manual disconnect/reconnect, or at worst power cycling the phone. Not something that I'd like to tell customers to do.

The older Windows Mobile devices are the worst, and unfortunately I still have to deal with them. Daily reboots all the time. Older Nokias (Symbian) were unpredictable. Some models worked fine, others less so. I would have expected them to have common chipset/BT stack but apparently this wasn't the case.
So far I have no experience on Windows Phone so I can't comment on them and Apple's devices still don't support RFCOMM at all (or so I've been told) so they're already lost cause for my use.

Then there's Android. Like Symbian it seems to be hit-or-miss. Older Samsung phone I had (with Android 2.3) was hopeless - connect() would just block forever if it couldn't connect to remote device immediately (and I do mean forever - even if device became available it would never connect and never fail, just block until process was killed). Newer ones seem to be better, for example I surprisingly haven't been able to get Trend Plus (Android 4.1.x if my memory serves) to fail even once (and trust me, I've been abusing it a lot while testing, just like the customers would abuse the link).  I don't have much experience on other manufacturers' devices, but unfortunately the conclusion here still is this:

For BlueTooth use every single phone must be tested and "certified" as usable in our systems. I wish I wouldn't have to do that but until situation on phone end gets better that's just how it must be.

(Toy) train wreck

Year or two ago we got our kid a Brio toy train set with some length of track and remotely controllable train. Back then it wasn't very interesting (to him) so that got buried for a while, but recently he got interested again, especially after we got him some more track (total length now being around 4 metres or so - fortunately we had one spare room to lay it out).

It was expected that the train would not last long as he likes to stop it by holding it when it's run on motor and pushing it forcefully, straining the gears and the motor. So it was not very surprising that it started to fail relatively quickly, eventually not moving anymore at all (but it still makes usual sounds and so on it electrics wasn't dead).

New ones are a bit expensive so I decided to see what could be done. Some one night the train mysteriously vanished from home (event that didn't go unnoticed by the small one) and was taken apart the next day.

The text on train says Brio RC-101, I tried to look it up but couldn't find it anymore so it might be no longer available (or maybe it's just sold as a set I couldn't find on quick search). I'd expect that newer models are similar in construction, prices for those (just train, no remote) were around 20-30 euros.

First problem was that it was closed with those damn triangle-shaped screws, four of them total. 2mm flat screwdriver can be used to open them, but you need to be very very careful - if the blade slips you will very likely damage the screw head and make rest of the process more difficult, if not impossible (note also that there apparently are several triangle sizes, others may need larger blade).

Inside was motor mechanism - nothing too fancy. I didn't bother to open it further to see the electronics (IR receptor as train is remote controllable, speakers and such) so no pictures of that, sorry. I don't expect them to be too interesting anyway, main chip most likely a COB with glob on top, a few passives, transistors and others and nothing more.

Gearbox comes out easily enough, aside the cover it isn't held in place at all. Fortunately no more fancy screws, gearbox was closed with two phillips-type screws. After popping it open the motor comes out easily enough. And yes, it was the motor that was broken, not the plastic wheels. Those would've been impossible to repair.

Here I have replaced the motor already, original motor is the one that not connected. It was annoyingly small - 8x10x15mm, excluding the shaft, so finding replacement was a bit difficult. I ended up ordering one from dx.com, sku 275090. It isn't exact replacement and most likely runs slower than original but no matter, important thing is that it runs. You can also see that new one is a bit shorter so I had to put in a bit glue to hold it in place. 

And of course despite trying to decipher markings on the motors the first time leads were the wrong and train reversed when it should be driving forward. Not difficult to fix but annoying.

One more picture of motor/gearbox mechanism while open. Note few small metal plates in compartment where the gearbox should be - four of them are used as spacers, placed between the train frame and gearbox. No idea why - maybe they have several gearbox designs, some of which are slightly larger.

Putting it all together was simple enough, and it's alive!

On how long it will last this time, well, your guess is as good as mine...


And somehow I predict that I have train track building duty this evening.

Later addition: After some observation train clearly (to me) is slower and weaker than before, but no matter. It would need 3v motor (train uses 2 AA batteries - replacement motor was 4,5v), but those seem to be even harder to find. Kid doesn't care so no matter, it's fine (and taking abuse just like before, maybe I should start looking for better replacement motor immediately...)


Post mortem seven months later: This train does not a good submarine make. In fact I just found out that water pretty much kills it permanently (not very surprising, really, but apparently this is a very difficult concept for small kids to grasp). All's not lost though, take one plastic gear off from the drive mechanism and you have pushable train instead.
I don't really feel like taking it apart to examine electronics there, since I pretty much expect to find just a small, cheap PCB with single (glob-covered) chip on it with maybe some passives and a speaker. Not very exciting.

Serial protocols (part 2)

In previous part I showed few simple cases of serial protocol. Now we'll improve them a bit by adding bit of buffering and simple checksum validation.

Checksum is implemented only on downlink (to Arduino) messages but extending calculation to uplink messages is trivial. This program does nothing but acknowledge messages with valid checksum; invalid checksum results NAK and too short message is ignored.

Side note on code comments; The code here is definitely not example on how you should comment your code. Since this is a tutorial I've chosen to focus most explanations and discussion on the main text, mostly because the topic itself requires way more explanation than I would normally put in code (typically I add "why" comments, but explanations here would add major "how exactly" part too -  of which the latter is in my opinion bad in living production code)

char recvBuff[4];
char recvLen = 0;

// Function to calculate checksum. Simple XOR sum is used.
unsigned char calcMsgSum(char *buff, int len)
{
  unsigned char sum = 0x77; 
  unsigned int i;
  for (i = 0; i < len; ++i)
    sum ^= buff[i];
  if ((sum == 13) || (sum == 10)) // Trivia for reader: Why is this here?
    sum += 10;
  return sum;
}
 
void loop()
{
  if (Serial.available() > 0) {
    char c = Serial.read();
    
    if (c == 10) {
      // LF ignored
      
    } else if (c == 13) {
      // CR used as end of line
      
      // Minimum of two characters required; data and checksum
      if (recvLen > 1) {
         // Verify received checksum against calculated
        unsigned char msgSum = calcMsgSum(recvBuff, recvLen-1);
        if (msgSum == recvBuff[recvLen-1]) {
          // For this example we do nothing but acknowledge message.
          Serial.write("A\r");  
        } else {
          Serial.write("N\r");  
        }
        
      }       
      recvLen = 0; // clear buffer for next command      
      
    } else {
      // Not CR or LF; store received char to buffer. 
      // There is never, I repeat, *NEVER* an excuse to not check your buffers. Always do it.
      if (recvLen < sizeof(recvBuff)) { 
        recvBuff[recvLen] = c;
        ++recvLen;
      }
    }
  }
}

Some test strings, including checksum character: "D3", "VER6" (calculated manually so I'm not 100% sure of correctness)

The checksum used here is simple XOR-sum. As a checksum algorithm XOR sum is fairly weak compared to, say, CRC but it is simple and quick to write so it is used as an example. Naturally the sender must have (functionally) equal checksum calculation routine to calculate it when building messages. Checksums in general are not topic here so I'll keep using this same XOR algorithm in my examples to keep them (more) simple.

Data itself is simple format; first data (max 3 bytes), then checksum (1 byte) and message is terminated with CR. LF character (which may be added by some terminal programs) is ignored.
This of course means that CR or LF characters may not appear in main message data. Using CR allows easy testing with simple terminal program (aside checksum calculation).

Note the buffer checking; if you don't do it already, build a habit of ALWAYS - with absolutely no exceptions - checking your buffers when dealing with incoming data. It does not matter where the data is coming from - even data internal to your system can get corrupted. C gets bad enough rep on this already so do your part to keep things tidy and safe. </rant>

When code were to receive too many characters you'd normally want to mark message invalid without bothering with further processing (received message will be invalid anyway), this code doesn't do that (again, to keep things simpler) but that isn't difficult to do.


But what if you want to transfer arbitrary binary data - likely including CR/LF characters - over the link? Escape characters will help you. In this example we map character 127 (0x7F) to mean that following character is data and not control data (changed part in blue).

unsigned char nextIsEscaped = 0; // We need flag for signalling this
  
void loop()
{
  if (Serial.available() > 0) {
    char c = Serial.read();
    
    if (nextIsEscaped) {
      if (recvLen < sizeof(recvBuff)) { // escaped data character - add to buffer
        recvBuff[recvLen] = c;
        ++recvLen;
      }
      nextIsEscaped = 0;
      
    } else if (c == 0x7F) {
      nextIsEscaped = 1;
      
    } else if (c == 10) {
      // LF ignored
      
    } else if (c == 13) {
      // CR used as end of line
      
      // Minimum of two characters required; data and checksum
      if (recvLen > 1) {
         // Verify received checksum against calculated
        unsigned char msgSum = calcMsgSum(recvBuff, recvLen-1);
        if (msgSum == recvBuff[recvLen-1]) {
          // For this example we do nothing but acknowledge message.
          Serial.write("A\r");  
        } else {
          Serial.write("N\r");  
        }
        
      }       
      recvLen = 0; // clear buffer for next command      
    } else {
      // Not escape char, CR or LF; store received char to buffer. 
      // There is never, I repeat, *NEVER* an excuse to not check your buffers. Always do it.
      if (recvLen < sizeof(recvBuff)) { 
        recvBuff[recvLen] = c;
        ++recvLen;
      }
    }
  }
}

So, now if we receive 0x7F character the system state is updated so that next incoming character will always be interpreted as data and not control character. Since our control characters here are CR, LF and 0x7F the sending system must always escape these characters before sending them as part of data. Checksum byte is also data and needs to be escaped. Note however that transmit side must calculate the checksum before doing the escaping to the message.

Also, remember those two lines in earlier checksum calculation marked as "trivia"? Those aren't needed anymore here (but they don't hurt much either if left there).

Unfortunately this change actually reduces out checksum's effectiveness a bit since now the checksum verifies the unescaped string and not the actually received bytes. In practice the reduced effectiveness most likely does not matter, but is still is worth noting.

Another thing to note is that neither lack of ignore-message-on-overflow and weak checksum calculations are security (as in "blackhat-looking-to-0wn-device") issues; it's up to next level message parser to make sure that invalid messages (be it accidental or intentional) are detected and rejected.

So I leave you here for a while. Hopefully I'll get to writing next part quicker than this, and there's still a lot to improve here and so far we've only tackled one part - transmit errors - of the communication protocol.

Serial protocols

I know I promised examples of serial protocols but I've been way too busy to write lately. Sorry about that. I managed to start with the most simple protocols so here is the first part.

For simplicity I am writing these to Arduino using its built-in USB (and yes, I'm aware that this messes with my point of reliability -  but these are simple examples for learning that can be copied and tested easily and quickly so bear with me here).

I'm focusing here on three-wire (ground, transmit, receive) type serial link with parameters 9600 8N1 (9600bps, 8 data bits, no parity, 1 stop bit) since that is the type I use the most. Usually MCUs can also provide hardware flow control and some other control mechanisms (like parity for simple bit-error detection) that can be used to make link more reliable and controllable, depending on your exact situation.

After loading the sketch to Arduino you can use your favorite serial terminal program to send commands to it - I've written custom one that handles some custom protocols I often use but there are many terminals available for free, for example termite that I'm linking here because happened to be on top of quick google search and it seemed actually useful, unlike few other top links.
( http://www.compuphase.com/software_termite.htm ).

Basic Arduino doesn't have much in way of easily visualized I/O so I'm using the "L" led and serial return channel as example of work done. In case of serial return channel I'll make note when data is part of protocol or just example (typically written on paper/screen/whatever and not seen by controller).

So to the code. First basic setup phase; use built-in routines to initialize serial port and set pin 13 (tied to L led) to output. Led will turn on when pin is driven high.

void setup() 
{
   // Init serial communication, 9600 8N1
  Serial.begin(9600);

   // Init L led, set off
  pinMode(13, OUTPUT);
  digitalWrite(13, LOW);
}

Then the main loop, starting with as simple code as possible.

void loop()
{
  if (Serial.available() > 0) {
    int c = Serial.read();
    
    if (c == '1') {
      digitalWrite(13, HIGH);
    }
    if (c == '0') {
      digitalWrite(13, LOW);
    }
  }
}

Arduino is simply listening to incoming data and acting on it by setting LED on when '1' is received or off when '0' is received. There is no error checking or acknowledgements of any kind, so controller can't know if device actually receives the commands.
For controlling a simple lamp (or relay or really anything that is either on or off) this may actually be "good enough". If link is "lossy" controller can simply send the same command multiple times, hoping the at least one is received successfully.

This protocol works (kinda-sorta) but is quite fragile. What happens if wire is cut? So let's add simple acknowledge mechanism; received send A when command is received and N if command is bad.

void loop()
{
  if (Serial.available() > 0) {
    int c = Serial.read();
    
    if (c == '1') {
      digitalWrite(13, HIGH);
      Serial.write('A');
    } else if (c == '0') {
      digitalWrite(13, LOW);
      Serial.write('A');
    } else {
      Serial.write('N');
    }
  }
}

Now controller can listen to returning data and react on it. If no reply or NAK ("negative acknowledge") is received the command is can be sent again. Now we can detect more error situations and even report such cases to user.

Note that the difference between '1' and '0' commands used above is only single bit. One bit can get corrupted easily, so unless other kind of error detection is used, you'd want to use command bytes with more difference. For example 0x30 ('0') for off, and 0x55 ('U') for on (four-bit difference; transfer errors should usually result NAK)

(excercise: what is the difference between 'A' and 'N', in bits, and is there need to use other bytes for those replies?)

Now, when you want to control LED these can be used, but how about when you want to check if the LED controller is alive? Let's add quick query command;

void loop()
{
  if (Serial.available() > 0) {
    int c = Serial.read();
    
    if (c == '?') {
      Serial.write('A'); 
    } else if (c == '1') {
      digitalWrite(13, HIGH);
      Serial.write('A');
    } else if (c == '0') {
      digitalWrite(13, LOW);
      Serial.write('A');
    } else {
      Serial.write('N');
    }
  }
}

So now controller can probe the device ("are you there?" "yup") when say starting up and report status to user without affecting the system state. Simple addition but so very useful.

On next post I'll be continuing from here, when I find some time of course. That unfortunately might take a while, since I'm (still) quite busy and the protocols introduced next will take some time to write and test too.

Courier services

I've never been too much a fan of DHL or UPS, now even more so. This time
it was DHL that managed to pretty much destroy this meter.

Surprisingly, after I managed to rip open the enclosure (mind you, that is 1mm thick aluminium so it has taken some serious beating to bend it this badly) and get to the electronics it still works. The PCB is however so badly bent it's unusable for any real use.

Update: DHL denies any responsibility since I apparently didn't report it damaged on arrival. Well, no, the box the meter came in was only slightly deformed, like something not-that-heavy (just 20kg or so) had been on top of it. Also the box was way larger than the meter so there should have been plenty of space for meter to be move around inside so that is what I expected to have happened (and yes, it was wrapped on bubble sheets).

I have pretty good idea on what it will take to damage the meter and really, arrived package didn't look that bad. Just for fun I tried to stand on the empty aluminium enclosure diagonally (on part that was bent - no PCB inside so it's even weaker than when assembled). I weigh around 70kg but that only bent the enclosure about 2mm inward. Not even close to damage seen here. Pretty much what I expected to happen.

They must have driven a friggin' forklift over the package, with meter caught just right inside it to cause this kind of damage.

While they're technically right - I did accept the damaged package - with that denial they also just lost any change of ever getting my business ever again.