The Evils of Arduino Strings

Everyone, when they’re starting out on the Arduino and similar boards, learns to use the String object for working with text. Or they think they do.

Well, you should forget all you think you have learned about using Strings on the Arduino, because it is all wrong.

Strings are a bit of a tricky area on the Arduino. The String object was created to make working with blocks of text easier for people that don’t really know what they are doing when it comes to low level C++ programming. However, if you don’t know what you’re doing when it comes to low level C++ programming then it is very easy to abuse the String object in such a way that it makes everything about your sketch fragile and unstable.

So I’m here to tell you firstly why you shouldn’t use String objects, and secondly, if you really must use them, how you go about using them properly.

Ok. First a little bit of computer theory – especially to do with memory.

The Arduino’s RAM is split up into different chunks for different purposes. There’s a chunk where all the global and static variables are stored (aka BSS and data areas). There’s the stack where local variables created within function are stored, and finally there’s the heap, which is where dynamic variables are stored (more on those in a moment).

If you want to know more about how these chunks of memory relate to each other you can read more on Wikipedia here: https://en.wikipedia.org/wiki/Data_segment

The main part we are concerned with here is the heap.

So let’s describe the stack and the heap so you can grasp the difference.

Imagine you have a load of coins. Each one represents a variable in your program. Local variables in functions are placed on the stack. As its name suggests this is like a single stack of coins. You place a coin on the top of the stack, and you can remove a coin from the top of the stack. Variables are placed on the stack, and the stack grows. Variables are removed from the stack, and the stack shrinks. Just like placing coins on top of the stack of coins and then removing the, You have to remove then in the opposite order that you placed them on. If you tried taking a coin out from the middle of the stack the whole stack would fall over.

Now the heap is completely different. Again, you have a load of coins. The coins are of different sizes. Some big, some small, and maybe you even have some notes (if you’re lucky). The heap is more like you are laying the coins on the table side by side in a line. You can place a coin at the end of the line, and you can remove coins from the middle of the line. If there is a gap in the line big enough for you to fit a new coin in then you can place it in there. If the coin is smaller than the gap you end up with a tiny gap between that new coin and the next one. That gap’s too small to fit another coin into, so the next time you want to place a coin down it has to go on the end of the line. After a while of adding and removing coins of different sizes you end up with a line of coins full of gaps that’s much longer than it needs to be.

This is called Heap Fragmentation and is a real problem when you only have a very limited amount of memory. All those gaps in the heap are wasted memory that is very hard to re-use.

Dynamic variables are variables that are created by asking the memory management routines to give you a chunk of memory to work with. Once you have finished working with that memory you are supposed to hand it back to the memory management routines so that it can be re-used by another part of your program. The main functions used here are malloc(), free() and realloc(). The first asks for a block of memory, the second gives it back again. The third says “This block of memory I have is too small. Give me a bigger one instead”.

When you ask for a block of memory it tries to find a hole in the heap that it can use for your request. If there is one it will use part or all of it for your request. If there isn’t it will add it to the end of the heap. If you ask for a bigger amount of memory with realloc() and there isn’t room where your current allocation is, it will move your allocation elsewhere, leaving a hole behind.

So you can see the problem of why dynamic memory allocation in systems with very small amounts of RAM can be a problem.

And the String class uses dynamic memory in abundance.

The biggest problem is when you perform many common operations with Strings you inadvertently create new String objects that you don’t need to. Every one of those potentially creates holes in the heap.

For instance, take this simple snippet:

String hi = "Hello";
String name = "Fred";
String greeting = hi + " " + name;

How many Strings do you count there? Nope, there’s four, not three. You have the String “hi” which has allocated RAM to store the word “Hello”, a second one that stores the name “Fred”, and a third that stores the results of adding the others together. But there is a fourth. You see, to build up the results of “greeting” it has to do it in stages. First it takes the string “hi” and adds a space to the end of it. That is placed into a new String object. Then to that new String object it adds the contents of the “name” object. So you have what is called a temporary. Just that, a temporary. Not a temporary variable or anything, just simply a temporary. It’s created in the process of doing the work and then thrown away again afterwards. And of course that has the potential of leaving behind it a hole.

So what should you do if you want to avoid these temporaries? Well, the String class has a handy function concat() which will add things to the end of an existing string.

So you could have something more like:

String greeting = hi;
greeting.concat(" ");
greeting.concat(name);

BUT there is another little gotcha there. Every time you add to the end of the String it has to make more space to store the extra text in it. If the hole it’s currently in is too small it will end up moving elsewhere and leaving behind it a hole, and the heap will grow. So even that is not a good solution.

So the best solution is not to use the String class at all, but to do all the work in proper native “C strings”. I’ll cover working with those a little later on.

So let’s look at another surprising little example. Take this bit of code:

void PrintVal(String tag, String value) {
    Serial.print(tag);
    Serial.print(" = ");
    Serial.println(value);
}

A little function which takes two Strings and prints them to the serial port separated by =.

You call the function with two Strings. Say you call it with:

String tempname = "Temperature";
String temp = "23C
PrintVal(tempname, temp)

You have two Strings already, tempname and temp. You then call the function, and in doing so you inadvertently create copies of both tempname and temp. Those copies are called tag and value in the function. So suddenly you have twice as much heap used by Strings as you did before. So you have made, without thinking about it, 2 extra Strings that you didn’t really mean to. And that has a real noticeable effect on the integrity of your heap.

By now it’s looking like Swiss cheese.

So how do you avoid those inadvertent copies all over the place? Well, the trick here is to pass the String as references instead of copies. That calls for the & reference operator:

void PrintVal(String &tag, String &value) {
    Serial.print(tag);
    Serial.print(" = ");
    Serial.println(value);
}

Now when you call the function the Strings tempname and tag are the exact same Strings. The same for temp and value. You have simply given the Strings new names for the function use. That’s saved two whole extra copies.

So now you can see why the String class, which was created for use by people who don’t have advanced programming skills, is not a good thing for people who don’t have advanced programming skills to use.

But even with advanced programming skills you can never work around all the shortcomings of the String class.

So instead you really need to learn how to do without the String class altogether. And that means using “C strings”.

First a little bit of anatomy.

A C string is simply an array of characters, but it is an array of characters that must obey certain rules.

The biggest rule of C strings is that they are NULL terminated. That means that the very last character of every C string must be ASCII character 0.

The internal C string manipulation functions I will be introducing in a moment all look for that final NULL character as a marker to show where the end of the string is. The reason is because in C an array, although you may have specified a size at compile time, doesn’t have that size stored as part of it, and neither does a C string. It is perfectly possible to say you’re working with a string of 10 characters and then fill it with 20 characters instead. That is a very bad thing to do, so you must learn to take care of these things. The result of that is known as a buffer overrun and is one of the most prevalent hack attacks used by cybercriminals – fill up an input buffer with more data than it can handle until you end up writing your data over part of the program that is being run – and then your data (which could quite happily be instructions for a program) would get executed, thus compromising the device. So care must really be taken to avoid that.

Creating a C string is as simple as:

char string[30];

That will create an array of up to 30 characters. If you do it globally it will be stored in the BSS area mentioned above. If you do it in a function it will be stored in the stack. Not a hint of it even going near the heap.

Don’t forget that the 30 character space that you have reserved includes the NULL character, so actually you only have room for 29 characters if you are to be able to follow the rules for a C string.

Getting things into C strings is a little more tricky though. You can specify some content right at the start if you like:

char string[30] = "This is a string";

And that’s simple enough. But what about if you want to change the content on the fly? Unlike with the String class you can’t just do this:

string = "New content";

Instead you have to change each of the characters in the array individually. You see string doesn’t contain the test, it just points to where it is in memory. So you need to manipulate the memory that it’s pointing to, not the pointer itself.

So to change what is in the string you can use the strcpy() function:

strcpy(string, "New content");

That will iterate character by character over the second string and place those characters into the first string’s memory.

Another thing you can’t do with C strings is adding them together. This will not work:

char hi[7] = "Hello ";
char name[5] = "Fred";
char all[14] = hi + name;

Remember, the variables hi and name just point to locations in memory where those strings are stored. So in fact what you are doing there is adding to addresses (numbers) together and ending up with some bigger number which you then try to assign to an array (which doesn’t work).

Instead you need to use the handy strcat() function:

char hi[7] = "Hello ";
char name[5] = "Fred";
char all[14] = "";
strcat(all, hi);
strcat(all, name);

The strcat function, like the strcpy function, copies the memory content character by character from the right hand string to the left hand one. Unlike strcpy though, strcat starts from the end of the first string, not the start.

Now, when it comes to buffer overruns, there is a special variation of all the string handling functions available. Every C string function has an n variant available, for instance strcpy has strncpy. These variations will perform on up to n characters. That allows you to limit the maximum number of characters you will work with, and thus help you to prevent buffer overruns from existing.

One of the hardest parts of working with C strings, though, is that of working out what is in a string. You can’t just compare strings:

char a[10] = "Part A";
char b[10] = "Part B";

if (a == b) {
....
}

That kind of comparison is merely comparing the pointers to the memory where the strings are stored. Instead you must compare the content of the memory character by character.

Fortunately, again, there are functions to do that for you. strcmp is a good one to start with.

strcmp will take two strings and compare them character by character. If the strings are the same it will return a 0. If one string is logically “less” than the other (in that “a” is less that “g”) it will return -1. If one string is greater than the other then it will return +1. Most of the time you don’t care about greater than or less than, only is it equal. So to compare the two strings above you would use:

if (strcmp(a, b) == 0) {
....
}

Another useful variation on that function is strcasecmp. This does the exact same job but it doesn’t care about upper or lowercase letters. So “Hello” would equal “hello” with strcasecmp but not with strcmp.

Again there are n variants of both those functions available, strncmp and strncasecmp.

So far I have only shown you one way of creating strings: char [num] { = “content”;}, but there are others, and they each have special meaning.

char *string;

That format just creates a pointer to a string. It doesn’t actually point to any string at all – it’s like a string with no size to it whatsoever. You may thing that’s useless, but it’s not – it’s incredibly useful. You can use it to point to any other string, and since it’s really just a number, you can move it around anywhere within a string. More on pointer manipulation later.

char *string = "This is text";

That is creating a pointer to some text in memory. That memory could well be Flash memory, or it may have been copied into RAM first, depending on your architecture. It is never safe to change the content of that string since it may be in read-only memory. The size of the string is determined by the length of the content at compile time.

char string[] = "This is text";

Just like above this will create a block of memory whose size is equal to the length of the content with the content in it and point the string at it. However, this differed from the one above in that it will always be copied into RAM first. It is perfectly safe to change the content of the string.

Note the subtle difference between those last two. The first is supposed to be read only and could cause untold problems if you try to change it – the second, though it looks almost identical, is safe to change. In the former it’s normal to prefix the declaration with const which tells the compiler “I will never change this. Don’t let me even try”.

const char *string = "This is text";

Now onto that aforementioned pointer manipulation. Passing C strings to functions.

Unlike a String object you cannot pass the entire string to a function as a parameter. Instead all you pass is the pointer to the memory where the string data is located. So the example function PrintVal that we looked at way up the page would look like this:

void PrintVal(char *tag, char *value) {
    Serial.print(tag);
    Serial.print(" = ");
    Serial.println(value);
}

Just a tiny change, but one with lots to it.

Now you’re passing the address in memory of where the two strings are stored, and those are then passed on to the print function which knows how to handle them. It then, staring at the address given, works character by character printing each one in turn up until it reaches that all important NULL character.

However, all is not well there. Imagine you call the function as:

char temp[] = "23C";
PrintVal("Temperature", temp);

You’d think that would be fine. And most of the time it would be. However, the compiler isn’t completely happy with it. If you allow the compiler to show warnings (turned off by default in most Arduino and Ardiuno-like IDEs) you would see it moaning. Simply because you have specified the parameters as “char *”, which means, as parameters, “Pointers to memory that I can edit”, yet you are passing the first parameter as a string literal “Temperature”. That’s not a string in memory that you can edit, it’s a literal string in Flash memory. So it moans.

So the rule of thumb, any char pointers that you are passing into a function that you know will not be modified by the function must be done as const char pointers:

void PrintVal(const char *tag, const char *value) {
    Serial.print(tag);
    Serial.print(" = ");
    Serial.println(value);
}

Now the compiler knows that you’re never going to modify the memory pointed to by the pointers you pass, and so it is perfectly happy for you to pass a read-only string.

I have mentioned a few times phrases like “iterates over each character until it reaches the NULL character”. But what do I mean and, more importantly, how does it do it? Well, let’s take a little look at an example – printing a string to Serial character by character.

There’s a number of ways this can be done, but I’m going to show you the pointer way of doing it. Here’s a little function to whet your appetite. I have purposely broken it down into lots of small steps so we can analyse what is going on:

void PrintString(const char *str) {
    char *p;
    p = str;
    while (*p) {
        Serial.print(*p);
        p++;
    }
}

I know, you’re thinking “WTF?!”, right? Well, don’t worry, it’s all quite simple.

Firstly you should recognise that this is a function to which we’re passing a pointer to a block of memory that we know we won’t be modifying (const char str). So we have a string pointer *str to work with.

Next we are creating a new string pointer, but it’s not pointing anywhere and not got any size. That “useless” one from before, remember? (char *p)

Next we are pointing that new variable to the same area of memory that str is pointing to (p = str) – so p now contains the same address as str does, and they both point to the same piece of memory – that is, the start of the string we want to print.

Now comes a while loop, with the enigmatic test “p”. The * in front of the *p means “Give me the value that is stored in the memory that p is pointing at”. Initially, then, that means “Give me the first character from the string” since p is pointing to the start of the string. Now the magic here is that the NULL character at the end of the string is 0, which is the same as FALSE, so when *p equates to 0 the while loop will finish.

Next we use that same * operator again to get the character that is currently pointed to and print it.

Finally, the last operation in the while loop, is to increment p (p++). Because p is just a number (the memory address), incrementing p causes it to point to the next address in memory. That means, p is now pointing to the next character in the string.

And so it continues until the character pointed to by p is NULL.

So you can now see the importance of that NULL character – without it how would functions like these know when to stop?

There are far more functions for working with C strings than I can cover here, but hopefully I have shown you the basics and you can now delve in and discover more for yourself and finally do away with that String class that causes so many problems.

For further reading you might like to check out these links:

http://www.tutorialspoint.com/c_standard_library/string_h.htm
http://www.tutorialspoint.com/c_standard_library/c_function_strtok.htm
https://en.wikibooks.org/wiki/C_Programming/Strings

Making accurate ADC readings on the Arduino

There are many sensors out there which output a voltage as a function of the supply voltage as their sensed value. Temperature sensors, light sensors, all sorts.

Measuring that voltage, and converting it in to real figures for whatever is being sensed is not actually as simple as you might at first think.

There are many examples on the internet for ocnverting an ADC value into a voltage, but basically it boils down to:

  • Divide the ADC value by the ADC maximum value
  • Multiply by the supply voltage

And that sounds simple enough, doesn’t it?

unsigned int ADCValue;
double Voltage;

ADCValue = analogRead(0);
Voltage = (ADCValue / 1023.0) * 5.0;

Surely that looks OK, yes? You’ve got your Arduino plugged into the USB, which is supposedly 5 volts – after all, all the examples on the web just say 5v.

Wrong!

What you have there is a rough approximation. Nothing more.

If you want to make ACCURATE readings you have to know exactly what your supply voltage is.

Measuring the 5V connection on my Arduino while plugged in to the USB is actually reading 5.12V. That makes a big difference to the results of the conversion from ADC to voltage value. And it fluctuates. Sometimes it’s 5.12V, sometimes it’s 5.14V. so, you really need to know the supply voltage at the time you are doing your ADC reading.

Sounds tricky, yes?

Yes.

However, if you have a known precise voltage you can measure using the ADC, then it is possible to calculate what your supply voltage is. Fortunately, some of the AVR chips used on Arduinos have just such a voltage available, and can be measured with the ADC. Any Arduino based on the 328 or 168 chips has this facility.

I came across this nice piece of code on the TinkerIt site. It measures this 1.1V reference voltage, and uses the resultant ADV value to work out what the supply voltage must be.

long readVcc() {
  long result;
  // Read 1.1V reference against AVcc
  ADMUX = _BV(REFS0) | _BV(MUX3) | _BV(MUX2) | _BV(MUX1);
  delay(2); // Wait for Vref to settle
  ADCSRA |= _BV(ADSC); // Convert
  while (bit_is_set(ADCSRA,ADSC));
  result = ADCL;
  result |= ADCH<<8;
  result = 1125300L / result; // Back-calculate AVcc in mV
  return result;
}

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.println( readVcc(), DEC );
  delay(1000);
}

Very nice. very elegant. And, more importantly, very useful.

So now, using that, your ADC code could now look like this:

unsigned int ADCValue;
double Voltage;
double Vcc;

Vcc = readVcc()/1000.0;
ADCValue = analogRead(0);
Voltage = (ADCValue / 1023.0) * Vcc;

And it will be a whole lot more accurate.

The Finite State Machine

Many of the programming questions on the Arduino forum can be answered with one simple response:

Implement a “Finite State Machine.”

But what actually is a “Finite State Machine”?

Put simply, a Finite State Machine (I’m bored with typing that now, let’s call it an FSM – not to be confused with the Flying Spaghetti Monster, of course) is a computer program that, at any point in time, can be in one of a pre-determined number of states. A state is a specific action, such as “Waiting for button 3 to be pressed”, or “Moving the servo to 69°”, for example.

An FSM can replace most of your while, for and similar loops, as well as annoying functions such as delay which get in the way if you want to do more than one thing at a time.

Let’s take a simple example to help illustrate how to make an FSM. Let’s say you have an automatic gate controlling access to a car park. The gate wants to stay closed until a car drives up to it. When the car arrives you want to open the gate and flash a warning light while it opens. Once the gate is open, you need to wait for the car to go through, then give a little extra time, and close the gate, flashing the light while you do.

Sounds pretty straight forward, but how would you go about programming that on, for example, an Arduino?

Well, your first thought is probably “That’s pretty simple. Wait for the signal that the car has arrived, open the gate, wait for the signal that the car has gone through, wait a little extra time, then close the gate. Return to start.

But what about the flashing light? While we’re opening the gate we want to flash a light. Yes, you could go out and buy a light that flashes, but that’s not the point. We want our program to do it. Sounds like a job for an FSM!

Ok, so let’s first sort out a few definitions, shall we? These aren’t “official” terms, but ones I like to use:

  • Stimulus

This is any input coming in from outside the FSM, be it a light sensor, a button, some other part of the program, or whatever. More importantly, and this is where it differs from just an “input”, it is a change in a value from a sensor. Such as a button changing from “not pressed” to “pressed”, or a light sensor changing from “light” to “dark”.

  • Action

An instruction for something to actually happen, such as opening the gate, turning the light on, etc.

  • Static State

A state where the FSM is waiting for stimulus, and is taking no actions.

  • Transitional State

A state which causes actions, but doesn’t look for stimulus. A Transitional State runs once and immediately moves on to a Static State.

Let’s take a look at those last two definitions in a little more detail, shall we?

The FSM is, at any one time, in a state. That state is one of two types of state: static or transitional. A static state can be though of one where you are waiting for something to happen. The FSM lingers in that state until whatever it is waiting for has happened. A typical static state is such as “waiting for car to arrive”. The state doesn’t change until the stimulus it is waiting for arrives.

Conversely, a transitional state is one which the FSM is only in very briefly while it actually does something. Atransitional state immediately moves on to another state, typically a static state. An example would be “open gate”, which would immediately move on to “waiting for gate to open”.

Ok, so let’s have a think about what states our FSM might want. States marked with (S) are static, and (T) aretransitional.

  1. Wait for car to arrive. (S)
  2. Open barrier & start light flashing (T)
  3. Wait for barrier to open (S)
  4. Stop light flashing (T)
  5. Wait for car to drive through (S)
  6. Make a note of the current time (T)
  7. Wait for a certain number of seconds to pass (S)
  8. Close barrier and start light flashing (T)
  9. Wait for barrier to close (S)
  10. Stop light flashing and go back to step 1 (T)

But hang on, you may say, what about the flashing light? All we have said there is “start light flashing” and “stop light flashing”. How do we actually flash the light? Well, that’s quite simple. The light has its own little FSM that is separate to the barrier FSM.

That one might look like this:

  1. Wait for an instruction to start flashing. (S)
  2. Turn the light on and remember the current time (T)
  3. Wait for time to pass (S)
  4. Turn the light off and remember the current time (T)
  5. Wait for time to pass (S)
  6. Go to step 2 (T)
  7. Turn off the light and go to step 1 (T)

Step 1 doesn’t actually have to do anything at all. It can be thought of an “idle” state, where the FSM doesn’t actually do anything. The beauty of Finite State Machines is that one FSM can change the state of another. So, when we say “start the light flashing” in the gate FSM, what we actually mean is “set the state of the light FSM to 2”, and for “stop the light flashing” we mean “set the state of the light FSM to 7. Once the light has started flashing it will continue flashing forever unless something outside the FSM changes the state. This can be thought of as a forced state changebecause something outside the FSM is forcing it to change state, rather than one of the states waiting for a signal to change state. The great thing with a forced state change is that no matter what state the FSM is in it always goes to the same new state. Whether the light is on or off it’s going to turn off the light and go to the idle state.

But how do we actually implement an FSM in code? Well, the simplest way is the switch construct (covered in an earlier tutorial). A simple numeric variable can be used to store the current state, and each case of the switch construct can be one of the states.

For example, this might be how you implement the flashing light FSM:

void setup()
{
  pinMode(13,OUTPUT);
}

void loop()
{
  static int state = 1; // initial state is 1, the "idle" state.
  static unsigned long ts;  // To store the "current" time in for delays.

  switch(state)
  {
    case 1:
      // We don't need to do anything here, waiting for a forced state change.
      break;
    case 2:
      digitalWrite(13,HIGH);  // Turn on the light
      ts = millis();  // Remember the current time
      state = 3;  // Move to the next state
      break;
    case 3:
      // If one second has passed, then move on to the next state.
     if(millis() > ts + 1000)
      {
        state = 4;
      }
      break;
    case 4:
      digitalWrite(13,LOW);  // Turn off the light
      ts = millis();  // Remember the current time
      state = 5;
      break;
    case 5:
      // If one second has passed, then go back to state 2.
      if(millis() > ts + 1000)
      {
        state = 2;
      }
      break;
    case 6:
      // We only get here when forced from outside.
      digitalWrite(13,LOW);  // Turn off the light
      state = 1;  // Return to the "idle" state.
      break;
    default:
      state = 1;
      break;
  }
}

You notice we have combined steps 5 and 6 into state 5, and step 7 has become state 6. This is just for cleanliness. We could have done it exactly the same, but we’d have effectively had a redundant state where it just went straight on to another state.

Also, this snippet won’t actually do anything, as there is no way to force a state change out of the idle state. You might want to experiment by setting the initial state to 2 instead of 1.

So, each time through the loop() function it examines the current state. That state value is uses to perform a selected small snipped of code. If the conditions are right, then the state is changed, so next time through the loop() function a different snippet of code is run.

Implementing multiple FSMs that run at the same time is as simple as having multiple switch() constructs in the loop() function. Notice how there isn’t a single loop of any form in there, except the main loop() function that wraps it all? Delays are all done by comparing the return value of millis() to the stored “ts” variable. for loops can be done by simply incrementing a variable until it reaches a certain value then moving on to another state. Of course, if your for loop is only a couple of iterations and doesn’t cause enough delay to be a problem to the operation of the FSMs you can still use a traditional for loop for small tasks.

One more tip before you go off and make your own FSM. The numbers assigned to the states are purely arbitrary. They could be anything. Anything as long as each state has a unique identifier. If you have more than just a few states it can become quite confusing and hard to remember what number means what state. Make good use of #define to create meaningful names to label your states with. For example, the flashing light FSM above could be rewritten using #definemacros like such:

#define S_IDLE 1
#define S_LEDON 2
#define S_WAITON 3
#define S_LEDOFF 4
#define S_WAITOFF 5
#define S_TURNOFF 6

void setup()
{
  pinMode(13,OUTPUT);
}

void loop()
{
  static int state = S_IDLE; // initial state is 1, the "idle" state.
  static unsigned long ts;  // To store the "current" time in for delays.

  switch(state)
  {
    case S_IDLE:
      // We don't need to do anything here, waiting for a forced state change.
      break;
    case S_LEDON:
      digitalWrite(13,HIGH);  // Turn on the light
      ts = millis();  // Remember the current time
      state = S_WAITON;  // Move to the next state
      break;
    case S_WAITON:
      // If one second has passed, then move on to the next state.
      if(millis() > ts + 1000)
      {
        state = S_LEDOFF;
      }
      break;
    case S_LEDOFF:
      digitalWrite(13,LOW);  // Turn off the light
      ts = millis();  // Remember the current time
      state = S_WAITOFF;
      break;
    case S_WAITOFF:
      // If one second has passed, then go back to state 2.
      if(millis() > ts + 1000)
      {
        state = S_LEDON;
      }
      break;
    case S_TURNOFF:
      // We only get here when forced from outside.
      digitalWrite(13,LOW);  // Turn off the light
      state = S_IDLE;  // Return to the "idle" state.
      break;
    default:
      state = S_IDLE;
      break;
  }
}

Makes it much more readable, yes?

So go and play. Make your next Arduino sketch a Finite State Machine.

Reading Serial on the Arduino

I see many many questions on the Arduino forums from people trying to read data from a serial connection and not fully understanding how it works – and hence failing.

So, how should you read from serial?

Well, what a lot of new users don’t realise is that serial data arrives one character at a time, and you have little or no control over just when that data arrives.

The Arduino has a handy function: Serial.available(), which tells you how many characters are in the serial device’s receive buffer. If you know beforehand how many characters you are going to be receiving this can be a very handy and simple way of managing your receiving. However, it has to be done right.

Too many times I have seen the following:

if (Serial.available() > 0) {
  for (int i=0; i<8; i++) {
    buffer[i] = Serial.read();
  }
}

What that is intended to do is wait for the message to arrive, then read all 8 characters of it into a buffer. What it actually does, is wait for the first character of the message to arrive and then try and read in 8 characters into the buffer, whether or not they have actually arrived.

What you should be doing is waiting for the internal serial buffer to have all 8 characters in it, and only then do you read them in:

if (Serial.available() >= 8) {
  for (int i=0; i<8; i++) {
    buffer[i] = Serial.read();
  }
}

See the subtle difference there?

Another problem is what to do if you don’t know how many characters you will be receiving. A common misconception seems to be that if you send 5 characters from one end of the serial link you will instantly receive 5 characters at the other end, and that those 5 characters will form a single coherent lump that the receiver somehow knows are one transmission. That is not the case. The receiver just receives the characters one at a time and adds them to its internal buffer. It has no concept at all about how many characters were sent, and how long the message is meant to be.

For that you need to have some marker that tells the receiver when the whole message has arrived. The normal marker to use is character 13, or the carriage return character. This is what your keyboard sends when you press the RETURN or ENTER key, so it’s a logical choice.

The receiver needs to just keep receiving characters and adding them to its buffer up until it receives this terminatingcharacter. Only then can you actually do anything with the message.

Remember – Serial.read() just returns one character (if it is available). So you will need to keep calling it over and over again until the whole message has arrived.

Take the following little sketch for example:

int readline(int readch, char *buffer, int len)
{
  static int pos = 0;
  int rpos;

  if (readch > 0) {
    switch (readch) {
      case '\n': // Ignore new-lines
        break;
      case '\r': // Return on CR
        rpos = pos;
        pos = 0;  // Reset position index ready for next time
        return rpos;
      default:
        if (pos < len-1) {
          buffer[pos++] = readch;
          buffer[pos] = 0;
        }
    }
  }
  // No end of line has been found, so return -1.
  return -1;
}

void setup()
{
  Serial.begin(9600);
}

void loop()
{
  static char buffer[80];
  if (readline(Serial.read(), buffer, 80) > 0) {
    Serial.print("You entered: >");
    Serial.print(buffer);
    Serial.println("<");
  }
}

So, what are we doing here? Well, we have a little function called readline(). This takes an incoming character (provided by Serial.read()), and decides what to do with it. If it’s a new line character (‘\n’) then it gets ignored and thrown away. This is to make it handle “CRLF” terminated lines properly without leaving a mess. If it’s a carriage return, then we decide that the message is complete and return the number of characters in the message. If it’s none of those, then just add the character to the buffer (if there’s room).

Only when the message has been received (the return value > 0) will we actually look at the contents of the buffer and do something with it – in this case just print it back to the serial device. you’ll notice that this method has several advantages:

  • You’re not blocking while waiting for a character to arrive, so you can continue doing other things at the same time as receiving your message.
  • The terminating carriage return (and possible line-feed) characters are automatically discarded, which makes string comparisons simpler.
  • Your program can continue doing other things while the message is being received – it’s very simple to know if it’s all there or not.
  • The buffer will always be properly null-character terminated.

So as soon as the return value of readline() is greater than 0 you know you have a full message of more that 0 characters in length. Now you can go and do whatever you want with that message – be it convert it to an integer withatoi(), compare it with other strings with strcmp(), etc.

New Blog Site

I have decided to ditch the old, ropey, incredibly slow Drupal based blog that I used to have and am now running a WordPress site free of charge thanks to good old WordPress themselves.

Unfortunately it seems I may have inadvertently managed to lose the old site down the back of the sofa.  Still, that means I now get to re-write the good articles that are worth re-writing, and get to completely ignore all the old rubbish that no one wants to read.

So wish me luck with trying to remember just what it was I had been writing in the past…