Sed: a 5 minute practical guide

Sed, which stands for stream editor, is probably most commonly used to find and replace strings in bash scripts. Something like this:

sed 's/foo/bar/g' foo.txt

It’s a powerful tool, but one that’s a little confusing. For me, at first glance sed looked about as confusing as regex was before I learned it – just a bunch of characters and symbols haphazardly mashed together. Sed is actually pretty straightforward once you know it’s simple syntax. Once learned, it enables you to do powerful things like programmatically substitute, delete, or insert text in a file or stream.

How the command works

The sed command itself is straightforward and follows this form:

sed [options/flags...] [sed script] [input file(s)]

Note that you’ll generally want to wrap what you pass to sed in single quotes. This prevents bash from interpreting the content and replacing special characters. Also, as we’ll see more of later, macOS sed and GNU/Linux sed are different. macOS sed is pretty old, so it’s not as capable. So only GNU sed can handle multiple input files or globs.

How the syntax works

Here’s how sed’s script syntax works:

[address]command[options]

• Address: can be a line number, range of line numbers, or a regex
• Command: just one letter, like s, which stands for “substitute”
• Options: depends on the command

🤯I wish I had known that a lot sooner.

Note that sed is not big on spaces between each of those things. On macOS (BSD) sed, a space between the address and command is okay though.

Let’s see how that works with some examples.

Substitute, s

sed 's/target/replacement/g' file.txt

This will globally substitute the word target with replacement. No address is provided, so it applies globally.

The command is s, which stands for “substitute”. For the options, s takes a regex, which is /target/replacement/g. The g flag is the regex global flag. This ensures that every match is replaced.

So if we wanted to provide an address to constrain what lines the command applies to, we could do something like this:

sed '1,5 s/target/replacement/g' file.txt

Same as before but only lines 1 through 5 will be affected by the command.

A really common use of substitute is to delete something in a line, like so:

sed 's/remove//g' file.txt

This will remove all instances of the word “remove” by simply replacing it with nothing.

Delete, d

The delete command will remove an entire line. Let’s say we wanted to use a regex to determine which lines we want to delete. We pass that regex to the address, like so:

sed '/^[[:space:]]/ d' file.txt

This will delete (d in sed) all lines that start with a space, newline, or tab. d doesn’t have any options. (Note [[:space:]] is a bash regex character class thing. There are some differences between JavaScript regex and bash’s regex.)

Modify in place, -i

If you’ve tried any of the above examples out, you may have noticed that they all output the modified text to stdout and leave the original source unmodified. If we wanted to preserve the original but create a new file from the modifications we could do that using plain bash redirection:

sed '/^[[:space:]]/ d' file.txt > file-modified.txt

But what if we want to modify it in place?

sed -i '/^[[:space:]]/ d' file.txt

The -i flag, which stands for “in-place”, will do just that. But here’s a caveat–while this works as is on Linux/GNU sed (which can be installed on macOS if you so desire), on default macOS sed, this fails. That’s because you can pass an argument to the -i flag which is optional on newer versions of sed but required on older ones. That argument specifies the extension of a backup file.

sed -i '.bak' '/^[[:space:]]/ d' file.txt

This command creates file.txt.bak with the original source, and sets the modified text into file.txt. You can pass an empty string to -i to make it not create a backup. This is what I do on macOS’s older sed when I want to edit in place without a backup.

sed -i '' '/^[[:space:]]/ d' file.txt

Insert, i

Let’s say I have a bunch of files that I want to insert a generic header on. Because GNU sed is newer and much easier than macOS’s, we’ll start with that one:

sed -i '1 i // Copyright 2019 Cameron Nokes' *.js

So we pass -i flag to sed with no arguments to tell it to modify in place. Then in our sed script, we want to insert on the first line, so the address is 1. Then i, for insert. Then the content we want inserted comes after. After the sed script, we pass a glob *.js to tell it modify all .js files in this directory. That’s it, pretty easy.

On macOS, things are more involved and I personally struggled to do this just on the command line, so I created a script:

for file in $(ls *.js); do
  sed -i '' '1 i\
  // Copyright 2019 Cameron Nokes
  ' $file
done

There’s a few important differences to point out here:

1. macOS sed can’t operate on multiple files, so I loop through them
2. sed’s i command on macOS sed has to be followed by a \ and a newline

Unfortunately the syntax gets a little awkward on macOS.

Multiple commands

You can do multiple sed commands in one script by separating them with a semicolon.

sed '1 d; 2,5 s/target/replacement/g' file.txt

This will delete the first line, then do a substitute on lines 2 through 5.

---

That’s it for sed. Sed can even do quite a bit more! You can read the full manual here https://www.gnu.org/software/sed/manual/sed.html.

Do as I sed, not as I sudo

That heading has no real meaning here, it was just the best bash pun I could come up with 😀. And lastly, a sed gif for your enjoyment:

via GIPHY

I don’t know what this has to do with sed, but it is tagged “sed” on giphy, and seems strangely fitting.