Due to WordPress’s abysmal handling of code blocks this blog post is now hosted at https://majenko.co.uk/blog/
I see many many questions on the Arduino forums from people trying to read data from a serial connection and not fully understanding how it works – and hence failing.
So, how should you read from serial?
Well, what a lot of new users don’t realise is that serial data arrives one character at a time, and you have little or no control over just when that data arrives.
The Arduino has a handy function: Serial.available(), which tells you how many characters are in the serial device’s receive buffer. If you know beforehand how many characters you are going to be receiving this can be a very handy and simple way of managing your receiving. However, it has to be done right.
Too many times I have seen the following:
if (Serial.available() > 0) {
for (int i=0; i<8; i++) {
buffer[i] = Serial.read();
}
}
What that is intended to do is wait for the message to arrive, then read all 8 characters of it into a buffer. What it actually does, is wait for the first character of the message to arrive and then try and read in 8 characters into the buffer, whether or not they have actually arrived.
What you should be doing is waiting for the internal serial buffer to have all 8 characters in it, and only then do you read them in:
if (Serial.available() >= 8) {
for (int i=0; i<8; i++) {
buffer[i] = Serial.read();
}
}
See the subtle difference there?
Another problem is what to do if you don’t know how many characters you will be receiving. A common misconception seems to be that if you send 5 characters from one end of the serial link you will instantly receive 5 characters at the other end, and that those 5 characters will form a single coherent lump that the receiver somehow knows are one transmission. That is not the case. The receiver just receives the characters one at a time and adds them to its internal buffer. It has no concept at all about how many characters were sent, and how long the message is meant to be.
For that you need to have some marker that tells the receiver when the whole message has arrived. The normal marker to use is character 13, or the carriage return character. This is what your keyboard sends when you press the RETURN or ENTER key, so it’s a logical choice.
The receiver needs to just keep receiving characters and adding them to its buffer up until it receives this terminatingcharacter. Only then can you actually do anything with the message.
Remember – Serial.read() just returns one character (if it is available). So you will need to keep calling it over and over again until the whole message has arrived.
Take the following little sketch for example:
char buf[80]; int readline(int readch, char *buffer, int len) { static int pos = 0; int rpos; if (readch > 0) { switch (readch) { case '\r': // Ignore CR break; case '\n': // Return on new-line rpos = pos; pos = 0; // Reset position index ready for next time return rpos; default: if (pos < len-1) { buffer[pos++] = readch; buffer[pos] = 0; } } } return 0; } void setup() { Serial.begin(115200); } void loop() { if (readline(Serial.read(), buf, 80) > 0) { Serial.print("You entered: >"); Serial.print(buf); Serial.println("<"); } }
Trouble reading this snippet? Get it on BitBucket.
So, what are we doing here? Well, we have a little function called readline(). This takes an incoming character (provided by Serial.read()), and decides what to do with it. If it’s a carriage return character (‘\r’) then it gets ignored and thrown away. This is to make it handle “CRLF” terminated lines properly without leaving a mess. If it’s a line feed, then we decide that the message is complete and return the number of characters in the message. If it’s none of those, then just add the character to the buffer (if there’s room).
Only when the message has been received (the return value > 0) will we actually look at the contents of the buffer and do something with it – in this case just print it back to the serial device. you’ll notice that this method has several advantages:
- You’re not blocking while waiting for a character to arrive, so you can continue doing other things at the same time as receiving your message.
- The terminating line feed (and possible carriage return) characters are automatically discarded, which makes string comparisons simpler.
- Your program can continue doing other things while the message is being received – it’s very simple to know if it’s all there or not.
- The buffer will always be properly null-character terminated.
So as soon as the return value of readline() is greater than 0 you know you have a full message of more that 0 characters in length. Now you can go and do whatever you want with that message – be it convert it to an integer withatoi(), compare it with other strings with strcmp(), etc.
I’m working on a serial communication protocol for the Arduino and I knew most of the example code I was seeing on forums wasn’t designed well. Your code illustrates proper design principles and understanding of serial buffers. Thank you for taking the time to share it, I can now code in confidence!
LikeLike
I really appreciate your taking time to share this post. I have recently built some projects that require communication with several serial devices. An adaptation of this scheme allows me to build up buffers for each in parallel instead of putting the rest of the sketch on hold while listening to a one device.
LikeLike
Brilliant and simple, this has helped me a lot!
many thanks
LikeLiked by 1 person
Pingback: Splitting Up Text in C | Majenko's Hardware Hacking Blog
How would I get this to react based on what is received over serial? I’ve been trying if, do/while, and switch but can’t quite get it to behave as desired.
LikeLike
These posts can probably help you: https://hackingmajenkoblog.wordpress.com/2016/02/04/the-evils-of-arduino-strings/ and https://hackingmajenkoblog.wordpress.com/2017/04/08/splitting-up-text-in-c/
LikeLike
hey thank you for the great article it give me a great aproach to seria;
i have a little problem and i would appreciate if you help
i have sending 2 data on Serial1 by Mega and receiving these two data by your last sketch on node bu it seems like data streams doubled on nodeMcu !
LikeLike
I am sorry, I can’t provide technical support through blog comments. It’s just not practical. I suggest you post a question on arduino.stackexchange.com where I will most likely find it. Remember to post your full code (or an example code that demonstrates the problem) along with your wiring.
LikeLike
Hi.
Firstly thank you for taking time to explain this. What im struggling with being a novice in arduino is the bit about the parameters we are are passing to the readline function. Can you provide a code snipet about how call out readline in the main loop please
LikeLike
First parameter is the character you have read from serial. Second is a char * buffer to store the string in. Third is the size of that buffer in bytes.
LikeLike
Thanks for writing this up, I’ve had a hard time finding more information about how Serial.Available() and Serial.Read() work. This is a great conceptual grounding.
I’m having a hard time following your code snippet, I think some important characters may have gotten dropped while pasting into wordpress. Here are a couple of things I have questions on (assuming the lines in your code snippet are numbered starting at 1):
-Line 10 ‘rpos = pos;’, it doesn’t look like pos is declared at 0, but inside the function, it doesn’t look like the value of pos is ever changed to anything other than 0. In the next line it’s value is then set to 0, but I’m not following how the value would have changed before that, or how rpos will be anything other than 0 as well.
-Line 14 ‘if (pos 0) {‘, I’m assuming this should be ‘if (pos = 0) {‘ ? I can’t get the current code snippet to compile.
-Line 14 ‘if (pos 0) {‘, is this if statement supposed to be inside the ‘default:’ code block (which starts on line 13) or follow after the end of the switch statement? I assume it supposed to follow the switch statement, but there are some ‘}’ brackets missing and it’s not clear what code should be the default, or what should be the end of the function. Also, are the Serial.print… lines (lines 15-17) supposed to be nested inside that if statement? Again it’s hard to tell without all of the close brackets.
Thanks!
LikeLike
Yes, WordPress is complete jank when it comes to posting code. It’s forever trashing my post. I need to post the code elsewhere so it can be properly viewed.
LikeLike
Github might be a good option if your willing to set up an account.
Any chance you could paste your example into a reply here? Not a problem if the white space gets messed up, if all of the brackets make it through your readers will be able to figure it out.
LikeLike
Nevermind, didn’t see it had been edited. Thanks!
LikeLike
I am in the process of shipping all my blog posts over to my main website where I have much better control over the formatting.
LikeLike
I think your definition of:
static int pos = 0;
needs to be global, rather than within your readline function, otherwise pos will just stay as zero surely….
LikeLike
No. It’s “static”.
LikeLiked by 1 person
Crazy, Ive never questioned how you get local persistent variables in Arduino and tend to just make them global. Thanks!
LikeLike
I do believe the test in the default branch should be:
if (pos < len-1) {
buffer[pos++] = readch;
buffer[pos] = 0;
}
Please correct me if I am wrong.
Thank you
LikeLike
Indeed. Or pos <= len – 2. An artefact of the many rewrites I have had to do since WP trashes the code.
LikeLike
Thank you for this article. I have tried to do a if statement with the buf variable to make things happen but nothing does. This compiles with no error. If I say it does not equal (!=) then the print command executes. I am guessing because buf is not a string but I have tried converting from DEC and anything else I could search for. Any guidance would be appreciated thank you.
if (readline(Serial.read(), buf, 80) > 0) {
Serial.print(“You entered: >”);
Serial.print(buf);
Serial.println(“<");
if (buf == "A10") {
Serial.println("Good Command ");}
}
LikeLike
Never mind I found your link in a reply above. Thank you
if (readline(Serial.read(), buf, 80) > 0) {
Serial.print(“You entered: >”);
Serial.print(buf);
Serial.println(“<");
char A10[80] = "A10";
if (strcmp(buf, A10) == 0) {
Serial.println("Good Command ");}
}
LikeLike
That’s not how you compare strings. You need to use the strcmp function. This post may help you: https://hackingmajenkoblog.wordpress.com/2016/02/04/the-evils-of-arduino-strings/
LikeLike
Tks ! But how I can change your sketch for not need to use the RETURN or ENTER key ? I want detect only want chacarter without press RETURN or ENTER key. Thank you in advance F.C.
LikeLike