Removing carriage returns
When working with shell scripts, it's very easy to get tripped up by line returns and get output which looks something like this:
$ ./example.sh -bash: ./example.sh: /bin/sh^M: bad interpreter: No such file or directory
The first time you run into this problem, it can be a little confusing. Running the script as an argument to bash will work as expected:
$ bash example.sh Hello world
And displaying the contents of the file with cat won't show any obvious problems:
$ cat example.sh #!/bin/sh echo 'Hello world'
What's actually going on?
A brief history of new lines
Most modern computers use ASCII, or a derivative like Unicode. ASCII characters can be split into two groups, printable characters and control characters. Printable characters are characters which can be represented visually, such as alphanumeric characters or punctuation.
Control characters on the other hand are not normally displayed. Instead they
provide instructions intended for the device displaying the text (e.g. a
printer). Many of the control characters have their roots in actions you would
perform on a manual typewriter. For example when moving to a new line on a
typewriter, you have to move the carriage back to the starting position and
then feed the paper so you don't overwrite the previous line. These movements
map to carriage returns (
\r) and line feeds (
\n) in ASCII.
Unfortunately not all operating systems use the same sequence of control
characters to represent a new line. Unix-like systems (Linux, OS X, FreeBSD
etc) all use a single line feed (
\n) to represent a new line. DOS and windows
systems on the other hand use a carriage return followed by a line feed
Looking at the first line
When you execute a shell script, the first line contains a shebang, followed by the path to the interpreter which should be used. If you're using windows-style new lines you will have an extra carriage return before the line feed character.
$ hexdump -c example.sh 0000000 # ! / b i n / s h \r \n e c h o 0000010 ' H e l l o w o r l d ' \r \n 000001f
As a result bash will try to use
/bin/sh\r as the interpreter and fail. You
can easily tell if you're using windows-style new lines using the
$ file example.sh example.sh: POSIX shell script, ASCII text executable, with CRLF line terminators
Removing extra carriage returns
To fix this issue, any extra carriage returns before a line feed need to be removed. There is a command called dos2unix for doing this:
$ dos2unix example.sh dos2unix: converting file example.sh to Unix format ...
If you don't have
dos2unix installed you can use sed:
$ sed -i 's/\r$//' example.sh
If for some reason you don't have
sed, you can use tr:
$ tr -d '\r' < example.sh > example_fixed.sh
After doing one of the above you should be able to run your script:
$ ./example.sh Hello world
Converting with Vim
Alternatively editors like Vim can also be used to convert scripts. Typing the following in normal mode will set the new line style and rewrite the script without any carriage returns:
:set ff=unix :w
ff is short for
w is short for write.