tiistai 24. helmikuuta 2015

Serial protocols; frames and sequence numbers

At last I got to writing last (for now) chapter of serial protocols. Reason for the delay was that I was actually implementing this system for first time for new board I was making: a "simple" test generator that can generate pulses, generate voltage drop sequences, calculate incoming pulses and so on. This also means that this isn't Arduino code anymore; the board I made uses PIC24 series MCU so code is a bit different, but adapting it shouldn't be too difficult.

This was actually version three of pulse generator I made; second version is now in use but has some problems. First being that since it's modified production board with hacked-on circuitry it isn't very scalable (thus specifically designed new board for third iteration), second problem being its serial protocol.

The protocol used in v2 was loosely something I presented in previous chapter, but the problem is that send-timeout-re-send -sequence sometimes does things twice; not very fun when you are trying to generate very specific pulse sequences. Due to this the tester actually fails more often than the product being tested. Annoying but not fatal - run test again and it's good - thus the third revision.

But I digress here, to the protocol itself now.

Data frames. I'm defining data frame here as sequence of bytes where Start Of Frame (SOF) character always starts data reception and End of Frame (EOF) character ends it. Data between these two characters is the actual payload. This makes synchronization of communications easy; all you need to do is listen to SOF and start processing data when it is received.
Of course this means that you can't use SOF and EOF characters in payload data; so we assign third character, ESCape. When ever data stream has SOF, EOF or ESC, it is changed to ESC-SOF, ESC-EOF or ESC-ESC so that situation can be detected.

Then the sequence numbers. While those are not related to framing above, I'm adding both topics here because both were implemented in the code I made. So, by adding simple sequence number to commands we can detect situations where command is send multiple times. If commands are always run to the completion before starting new the system doesn't even need to cache them, just keeping track of latest one would be enough, but I've added short history to the example here. With slight modification this also allows state tracking of long-running commands (say, "generate 1000 pulses @ 50 Hz"); when starting execution you send "Executing" response, and when done you change it to "Done". So master can just send same request (with same seq id) again and again until "Done" is received -- assuming of course that command parser is run in middle of execution too.

In following code I'm using frame format "SOF SeqID Data0...DataN BCC EOF". Command's SeqID is in range of 1..15 (0x01..0x0F) (since we have only four results stored larger range is not needed), and reply's SeqID is command's SeqID ORed by 0x80, so 0x81 .. 0x8F. And just before EOF comes checksum (I'm used of using short form "BCC" which is technically not correct but bear with me here).

 // Control characters
#define SOF 0xA8
#define EOF 0xD5
#define ESC 0xF0

// Command receive buffer
unsigned char cmdBuff[128];
unsigned char cmdBuffLen, cmdBuffPrevChar;

// Buffer for last 4 results
#define MAX_CMD_RESULTS 4
// [0]=command ID, [1]=length, rest=raw payload data
unsigned char cmdResults[MAX_CMD_RESULTS][10]; 

We don't want checksum to match any of the control characters so slight modification to previous calculation (and again, this is not in any sense "secure" method, just a quick and dirty way to catch basic bit flips and such during communication)

unsigned char calcBcc(unsigned char *buff, unsigned char len)
{
  unsigned char i;
  unsigned char sum = 0x12;
  for (i = 0; i < len; ++i)
    { sum = sum ^ buff[i];
    }
   // checksum can't be any of special chars, so change it.
  if ((sum == SOF) || (sum == EOF) || (sum == ESC)) 
    --sum;
  return sum;
}

Main received that handles frame reception. This assumes that actual receive part is handled elsewhere (typically interrupt). This should typically be run in main loop.

  while (serialGetRxBufferSize())
    { unsigned char c = serialGetRxChar();
       // Start of Frame and not escaped?
      if ((c == SOF) && (cmdBuffPrevChar != ESC))
        { cmdBuff[0] = c;
          cmdBuffLen = 1;
          cmdBuffPrevChar = 0;
        }
        // EOF and not escaped?
      else if ((c == EOF) && (cmdBuffPrevChar != ESC))
        {
          processFrame();
          cmdBuffLen = 0; // frame finished
          cmdBuffPrevChar = 0;
        }
      else if (cmdBuffLen)
        { if (cmdBuffLen < sizeof(cmdBuff)-1)
            {  // BCC is calculated over escaped data
              cmdBuff[cmdBuffLen++] = c;
              cmdBuffPrevChar = c;
            }
          else
            { cmdBuffLen = 0; // overflow - quetly ignore
            }
        }
      else
        { // data outside frame - ignored
        } 
    }

So above handles receiving the frame, so next is the parser with its helpers. This only sends back ACK with "Test" payload sting on correct frames.

/* -----------------------------------------------------
 * Remove escaping within the command buffer.
 * This is done as in-place operation by moving data towards
 * the start of buffer when escape characters are found.
 * 
 * I actually have to admit that I haven't really tested
 * escaping so far; data I've used has been ASCII only so far.
 * 
 */
void unescapeCmdBuff()
{
  unsigned char iptr = 1; // "in pointer"
  unsigned char optr = 1; // "out pointer"
  while (iptr <= cmdBuffLen)
    {
      if (cmdBuff[iptr] == ESC) // escape char; skip it
        { ++iptr;
        }
      cmdBuff[optr++] = cmdBuff[iptr++]; // copy
    }
}

/* ---------------------------------------------------------
 * Process full received input frame.
 * cmdBuff[0] has SOF, [cmdBuffLen] has BCC and rest is data, 
 * excluding EOF that is not stored. 
 */
void processFrame()
{
  unsigned int i;

  if (cmdBuffLen < 4) // minimum for frame: SOF ID DATA0 BCC (eof); 
    return;           // anything less is invalid

  i = calcBcc(cmdBuff+1, cmdBuffLen-2);

  if (i != cmdBuff[cmdBuffLen-1])
    { return; // bad checksum, ignored/dropped; controller will send again.
    }

  if (sendResult(cmdBuff[1], 0, 0)) // check if result is cached already
    { return; // had result; it was sent.
    }

   // first un-escape the buffer
  unescapeCmdBuff();

   // So that result for that request was not stored, generate it.

  sendAck( (unsigned char*)"Test", 4); // send something
}

Frame sending. Internally we store results in non-framed format, so whenever anything is sent back we have to build entire frame again. Reply payloads are limited to 8 bytes by sendFrame function's buffer allocation.

/* -------------------------------------------------------------
 * Send reply data back as frame, adding SOF, EOF, BCC and escaping.
 */
void sendFrame(unsigned char id, unsigned char *data, unsigned char dataLen)
{
   // Build frame
  unsigned char frame[20], fptr, i;
  frame[0] = SOF;
  frame[1] = id;

   // Add data to frame, escaping special characters.
  fptr = 2;
  for (i = 0; i < dataLen; ++i)
    { unsigned char c = data[i];
      if ((c == SOF) || (c == EOF) || (c == ESC))
        { frame[fptr++] = ESC;
        }
      frame[fptr++] = c;
       // Note that there are no guards on overflow here. Assumption is that
       // max send data len is 8, so escaped length is max 16 and frame overhead
       // four more, so 20 is just enough even in worst-case.
    }

  frame[fptr] = calcBcc(frame+1, fptr-1); // SOF excluded from BCC calculation
  frame[fptr+1] = EOF;
  serialWriteData(frame, fptr+2); // fptr+2 = total length
}
 

Then the result sending function that actually serves dual purpose. If called with NULL data pointer (by processFrame) it checks if result for given request is already stored and if so, send it again (i.e. handling duplicate request from controller).

If called with non-NULL data, above is done again (granted, unnecessarily) and when there is no old result, new one is stored in result FIFO and sent to controller.
Yes, this could (and should) be split in two functions, first doing "check if we have result" part and second doing "add new result" part for clearer functional split. This just is how I wrote it so I'm leaving it.

/* ----------------------------------------------------------------
 * Send result (new, or copy of previous).
 * Returns 1 if result was sent (either copy of old, or new), 
 *   or 0 if no result was sent (ie there was no result stored for given ID)
 */
unsigned char sendResult(unsigned char id, unsigned char *data, unsigned char len)
{
   // First check if we already have cached result for given command.
   // If so, send it.
  unsigned int i;
  for (i = 0; i < MAX_CMD_RESULTS; ++i)
    { if (cmdResults[i][0] == id)
        {
          sendFrame(id | 0x80, &cmdResults[i][2], cmdResults[i][1]);
          return 1;
        }
    }
   // Was not cached and no new data -- don't do anything.
  if (data == 0)
    return 0; // no cached data and no new data given -- fail

   // Newest result is stored as first entry in FIFO, so push older
   // results back.
  for (i = MAX_CMD_RESULTS-1; i > 0; --i)
    { memcpy(&cmdResults[i-1], &cmdResults[i], sizeof(cmdResults[i]) );
    }
   // Then store new result data and send it.
  cmdResults[0][0] = id;
  cmdResults[0][1] = len;
  memcpy(&cmdResults[0][2], data, 8);
  sendFrame(id | 0x80, data, len);
  return 1;
}
 
/* ------------------------------------------------
 * Shortcut functions; Ack (A) with payload data and NAK with no data.
 */
void sendAck(unsigned int id, unsigned char *data, unsigned char len)
{
  unsigned char abuff[10];
  abuff[0] = 'A';  
  memcpy(abuff+1, data, len);
  sendResult(id, abuff, len+1);
}

void sendNak(unsigned int id)
{
  sendResult(id, (unsigned char*)"N", 1);
} 

This was receiver side; on master side same code can mostly be used. This unfortunately isn't exactly very clear example, but then again, protocol here is getting quite complex - functionality doesn't come without cost in this case either.

I've left out initial setup (all variables should be cleared to zero) and initial handshake between controller and receiver. Latter is actually quite interesting, as when controller is starting up it has no idea of the receiver's state (assuming receiver has been previously used and not turned off in between) - any sequence ID it uses might already be cached in receiver so results to commands are initially nonsense. I'm leaving that as exercise for reader for now -- there are multiple different solutions, none of which are clearly superior to others and vary in complexity.





Ei kommentteja:

Lähetä kommentti