Understand Basic Command-Line Concepts – Take Control of the Mac Command Line with Terminal, 3rd Edition

Understand Basic Command-Line Concepts

In order to make sense of what you read about the command line, you should know a bit of background material. This chapter explains the ideas and terminology I use throughout the book, providing context for everything I discuss later.

What’s Unix?

Unix is a computer operating system with roots going back to 1969. Back then, Unix referred to one specific operating system running on certain expensive minicomputers (which weren’t “mini” at all; they were enormous!). Over time, quite a few companies, educational institutions, and other groups have developed their own variants of Unix—some were offshoots from the original version and others were built from scratch.

After many branches, splits, mergers, and parallel projects, there are now more than a dozen distinct families of Unix and Unix-like operating systems. Within each family, such as Linux (a Unix-like system), there may be many individual variants, or distributions.

macOS is a version of Unix that nicely illustrates this process of branching and merging. On the one hand, you had the classic Macintosh OS, which developed on its own path between 1984 and 2002. On the other hand, you had NeXTSTEP, an operating system based on a variety of Unix called BSD (Berkeley Software Distribution). NeXT, the developer of NeXTSTEP, was the company that Steve Jobs founded after leaving Apple in 1985.

When Apple bought NeXT in 1996, they began building a new operating system that extended and enhanced NeXTSTEP while layering on capabilities (and some of the user interface) of the classic Mac OS. The result was Mac OS X (later renamed OS X and now macOS): it’s Unix underneath, but with lots of extra stuff that’s not in other versions of Unix. If you took macOS and stripped off the graphical interface, the Cocoa application programming interfaces (APIs), and all the built-in apps such as Mail and Safari, you’d get the Unix core of macOS. This core has its own name: Darwin. When you work in the command-line environment, you’ll encounter this term from time to time.

Darwin is itself a complete operating system, and though Apple doesn’t sell computers that run only Darwin, it is available as open source so anyone with sufficient technical skill can download, compile, and run Darwin as an operating system on their own computer—for free.

What’s a Command Line?

A command-line interface is a way of giving instructions to a computer and getting results back. You type a command (a word or other sequence of characters) and press Return or Enter. The computer then processes that command and displays the result (often in a list or other chunk of text). In most cases, all your input and output remains on the screen, scrolling up as more appears. But only one line—usually the last line of text in the window, and usually designated by a blinking cursor—is the actual command line, the one where commands appear when you type them.

What’s a Shell?

A shell is a program that creates a user interface of one kind or another, enabling you to interact with a computer. In macOS, the Finder is a type of shell—a graphical shell—and there are still other varieties with other interfaces. But for the purposes of this book, I use the term “shell” to refer only to programs that create a command-line interface.

macOS includes up to seven different shells, which means that your Mac has not just one command-line interface, but seven! These shells share many attributes—in fact, they’re more alike than different. Most commands work the same way in all the shells, and produce similar results. The shells in macOS are all standard Unix shells, and at least one of them is on pretty much any computer running any Unix or Unix-like operating system.

The original Unix shell was called the Bourne shell (after its creator, Stephen Bourne). The actual program that runs the Bourne shell has a much shorter name: sh. The other Unix shells included with macOS are:

  • csh: the C shell, named for similarities to the C programming language (Unix folks love names with puns, too, as you’ll see)

  • tcsh: the Tenex C shell, which adds features to csh

  • ksh: the Korn shell, a variant of sh (with some csh features) developed by David Korn

  • bash: the Bourne-again shell (yet another superset of sh)

  • zsh: the Z shell, an advanced shell named after Yale professor Zhong Shao that incorporates features from tcsh, ksh, and bash, plus other capabilities

  • dash: the Debian Almquist shell, a lightweight shell that’s been around for more than two decades, but was not included with macOS until Catalina

In Mac OS X 10.2 Jaguar and earlier versions, tcsh was the default shell. Starting with 10.3 Panther, bash became the new default. And in 10.15 Catalina, zsh replaced bash as the default shell (see Zsh Becomes the New Default Shell). However, if you’ve upgraded your Mac (or transferred your user account) from an earlier version of macOS that used a different default shell, your account may still be configured to use the earlier default shell.

In this book, I discuss only the zsh and bash shells (which, as you’ll see, are similar enough that most of what I say applies equally to both of them). Almost everything you learn in this book will serve you well regardless of which shell you use—zsh, bash, or any of the others.

A bit later in the book, in Set a Default Shell, I show you how to confirm which shell you’re currently using and how to change your default, if you like.

What’s Terminal?

So, how do you run a shell in order to use a command-line interface on your Mac? You use an app called a terminal emulator.

As the name suggests, a terminal emulator simulates a terminal—the devices people used to interact with computers back in the days of monolithic mainframes. A terminal consisted of little more than a display (or, even earlier, a printer), a keyboard, and a network connection. Terminals may have looked like computers, but all they did was receive input from users, send it along to the actual computer (which was likely in a different room or even a different building), and display any results that came back.

A modern terminal emulator program provides a terminal-like connection to a shell running either on the same computer or on a different computer over a network.

Quite a few terminal emulators run on macOS, but the one you’re most likely to use is called—you guessed it—Terminal, and it’s included as part of macOS. Some third-party terminal emulators, such as iTerm 2, have fancy and impressive features that will save you time and effort if you do a lot of work on the command line. (Read about even more options in 10 Best Alternatives for the MacOS Terminal App at FOSSMint.) However, in order to keep this book to a reasonable length, I discuss only Terminal here.

So, to summarize: you use Terminal to run a shell, which provides a command-line interface to macOS—a variety of Unix (of which the non-graphical portion is known as Darwin). You can use the macOS command line successfully without having all those facts entirely clear in your mind, but a rough grasp of the hierarchy makes the process a bit more comprehensible.

What Are Commands, Arguments, and Flags?

The last piece of background information I want to provide has to do with the kinds of things you type into a Terminal window. I provide extensive examples of all these items ahead, but I want to give you an introduction to three important terms: commands, arguments, and flags. If you don’t fully understand this stuff right now, don’t worry: it will become clearer after some examples.

Commands

Commands are straightforward; they’re the verbs of the command line (even though they may look nothing like English verbs). When you enter a command, you tell the computer to do something, such as run a program. Very often, entering a command—a single word or abbreviation—is sufficient to get something done.

For example—not to get ahead of myself but just to illustrate—if you enter the command date, your Terminal window shows the current date and time.

Arguments

Along with commands (verbs), we have arguments, which you can think of as nouns—or, in grammatical terms, direct objects. For example, I could say to you, “Eat!,” and you could follow that command by consuming any food at hand. However, if I want you to eat something in particular, I might say, “Eat cereal!” Here, cereal is the direct object, or what we’d call an argument in a command-line interface.

On the command line, you must frequently specify the file, directory, or other item to which you want a command applied. In general, you simply type the command, a space, and then the argument. For example, the command nano, by itself, opens a text editor called nano. (In other words, entering nano means “run nano”—you tell the shell to execute a command simply by entering its name.) But enter nano file1 and the command instead opens the file file1 using the nano text editor. Here, file1 is the argument to the command nano.

Some commands accept no arguments. Some take optional arguments. And some commands require one or even several arguments. For example, to change the modification date of three files—file1, file2, and file3—I can enter touch file1 file2 file3. But other commands require multiple arguments that have different meanings (as in “Process file1 with the information found in file2 and store the output in file3”). In these cases, the order in which the arguments appear is critical. I detail which commands in this book take arguments, the order of those arguments, and the circumstances when you need to use those arguments.

Flags

Besides verbs and nouns, we have adverbs! In English, I could say, “Eat cereal quickly!” or “Watch TV quietly.” The adverbs quickly and quietly don’t tell you what to do, but rather how to do it. By analogy, an expression in a command-line statement that specifies how a command should be accomplished is called a flag, though you may also hear it referred to as an option or switch. (Some people consider a flag to be a type of argument, but I’m going to ignore that technicality.)

Suppose I want to list the files in a directory. I could enter the ls (“list”) command, which would do just that. But if I want to list the files in a particular way—say, in a way that included their sizes and modification dates—I could add a flag to the ls command.

The flag that ls uses to indicate a “long” listing (including sizes and dates) is -l. So if I enter ls -l (note the space before the flag), I get the kind of listing I want.

Some commands require both arguments and flags. In general, the order is command flag(s) argument(s), which is unlike usual English word order—it would be comparable to saying, “Eat quickly cereal!” For example, if you want to use the ls (“list”) command to show you only the names of files beginning with the letter r (r*), in long (-l) format, you’d put it like this: ls -l r*.

What Changed in Catalina?

For most of the history of macOS, changes to the command-line environment (that is, the Terminal app, the shells themselves, and the preinstalled Unix programs) have been minor and infrequent. Indeed, if you were to use the very first version of this book (from 2009) and try the commands in today’s version of macOS, the vast majority of them would still work, just as most of the examples in this book work as far back as the very first release of Mac OS X.

However, with the 2019 release of macOS 10.15 Catalina, several noteworthy changes occurred, and depending on how you use the command line, you may find some of these to be quite significant. Here’s what you need to know.

Zsh Becomes the New Default Shell

The first thing to know about Catalina is that it changes the default shell, for the first time since Panther in 2003: zsh is now the default, rather than bash.

Based on articles and blog posts I’ve read, and emails I’ve received from readers, it seems that this change is widely misunderstood. So I want to set the record straight here. To make a long story short, this change will have almost zero effect on the overwhelming majority of users. Almost everyone can ignore it, almost all the time.

The facts are these:

  • Zsh has been in macOS a long time. As I hinted earlier in What’s a Shell?, both zsh and bash have been part of macOS for many years. You could have used zsh in Mac OS X 10.1 in 2001 if you wanted to! It’s not a new shell, and it’s not new to the Mac.

  • Bash is still there. The bash shell is still included with Catalina, and it still works just fine. Apple didn’t remove it—they just changed the default.

  • Your default stays in place when you upgrade. If bash was your default shell in Mojave (or earlier) and you upgraded your Mac to Catalina (or later), bash is still your default. Zsh is the default only for new installations of Catalina (or later), or newly created user accounts.

  • Zsh and bash are extremely similar. Nearly everything you can do in bash works exactly the same way in zsh. Sure, each shell has its idiosyncrasies and zsh has some nice additional features; I discuss some of them later in Zsh Tips and Shortcuts. But unless you’re a hard-core command-line user, you may never notice these changes.

  • “Default” just means what runs automatically. When you open the Terminal app (or you open a new tab or window in Terminal), the shell you’ve set as the default runs. That’s all “default” means.

  • You can change your default easily. I describe multiple ways to do this later on, in Set a Default Shell.

  • You can switch shells anytime, on the fly. Regardless of your default, you can switch shells instantly, at any time, simply by typing the other shell’s name. So, if you’re currently in a bash shell and you decide you want to use zsh, just enter zsh. If you’re in a zsh shell and you want to use bash, just enter bash. Done!

  • Scripts aren’t necessarily affected at all. If you write shell scripts (see Create Your Own Shell Script), or use shell scripts other people have written, those scripts should start with a line that says which shell should perform the script’s actions. A script that specifies the bash shell will continue to work even if you’re running it in zsh, and vice versa. But for that matter, script syntax between the two shells is so similar that most scripts could switch their specified shell and still work exactly the same.

  • You can follow (or ignore) prompts to switch. If you’ve upgraded to Catalina or later from an earlier version of macOS that used a different default, you’ll see something like this in every new Terminal window:

    The default interactive shell is now zsh.
    To update your account to use zsh, please run `chsh -s /bin/zsh`.
    For more details, please visit https://support.apple.com/kb/HT208050.

    If you want to go ahead and switch your shell to zsh, you can follow those instructions. But if you want to keep using your old shell—and not be constantly nagged about it—you can suppress that message. Read How to hide the ‘default interactive shell is now zsh’ message in Terminal on macOS at AddictiveTips for instructions.

  • macOS Recovery still uses bash. If you press ⌘-Shift-R to reboot your Mac into macOS Recovery and open the Terminal app there, you’ll find that bash is still the default in that environment (and zsh isn’t even available as an option, though sh and dash are).

All that said, since zsh and bash are so similar, I include instructions for both of them in this book. Except as noted, everything I say here works exactly the same way regardless of which of those two shells you use.

The Startup Volume is Read-Only

In order to reduce the possibility that malware (or, ahem, user error) could damage, delete, or replace essential system files, Apple made a big change in Catalina. The startup volume is now split into two volumes behind the scenes—one (still called Macintosh HD by default) that’s read-only and contains the core components of macOS, and another that’s writable (called Macintosh HD - Data by default) and contains all your optional and third-party apps, user-generated data, and so on. The upshot of this arrangement is that it’s next to impossible to make changes to your startup volume (intentionally or otherwise) that would prevent your Mac from booting—and that’s a good thing!

Apple does some clever tricks to hide the complexity of the two-volume arrangement. In the Finder as well as on the command line, this split is normally invisible—ordinarily, you’ll never see the “Data” volume separately, and the /Applications folder (for example) shows both Apple’s apps and third-party apps as though they were all located on the same volume. However, in some situations, especially when working on the command line, you have to be able to tell which of the two underlying volumes a certain file or folder is really on, and you have to know how to make changes to the writeable “Data” volume. I discuss this situation in the context of the ls (“list”) command—refer to See What’s Here—but the instructions there can apply to any command.

Scripting Languages Are Deprecated

Scripting languages such as Perl, Python, and Ruby have been built into macOS from the very beginning, and they’re almost invariably found on other Unix and Unix-like operating systems (see Run a Script). Their presence not only enables you to run and write scripts in these languages, but also makes it possible for lots of important third-party software to run. (For example, the popular Homebrew system, which I discuss in Use a Package Manager, is written entirely in Ruby, and therefore requires Ruby to be present in order to run.)

In Catalina, Apple has deprecated these languages, which means, basically, that although they are still present and still work for now, Apple is declaring those languages have essentially been abandoned in place and will be removed in the future (likely in macOS 10.16, expected in late 2020. Developers whose software depends on them are being urged to bundle the relevant scripting languages with their apps.

I don’t know why Apple is doing this (though I have some theories), and I wish they wouldn’t, because this move is going to cause a considerable amount of confusion and frustration. However, it’s ultimately not a huge issue, for two reasons:

  • You can still download and install these languages yourself. If you find yourself on a Mac without one of these languages (or with an older version), you can download and install the latest version yourself, using the links below or a package manager (see Use a Package Manager):

    • Perl: Download the ActivePerl installer, or (on the same page) even customize an installation for your specific needs.

    • Python: Download the current release (or earlier releases) from python.org.

    • Ruby: Download any of several different installers, or download the source code and compile it yourself, at ruby-lang.org.

  • macOS shipped with old versions anyway. For reasons only Apple knows, the versions of these languages that have shipped with macOS were always significantly behind the current release versions. For that reason, serious scripters often downloaded newer releases anyway—either supplementing or replacing the ones Apple provided.

If you’re running Catalina or earlier, you can basically ignore all this for now—unless you want to install a newer version of one of these languages for some reason. Just be aware that once these languages stop shipping with macOS, you may have to jump through some extra hoops to be able to use certain command-line software that depends on them.

Apple Enforces Notarization Requirement

In Apple’s ongoing quest to reduce the likelihood of malware infecting your Mac, the company has been imposing progressively stricter requirements on developers. Now, I could spend several pages here explaining all the complexities of terms like code-signing, hardening, and notarization, but I’ll spare you. After all, if you’re an Apple developer, you undoubtedly know about all that stuff already, and if you aren’t a developer, almost none of those details will have any effect on your day-to-day computer usage. Ordinary users should not have to lose sleep over whether some software they downloaded was notarized or not—or even what notarization is. (OK, since you’re twisting my arm, it’s basically Apple’s stamp of approval that software has passed an automated malware check.)

Except…

On February 3, 2020, Apple began enforcing a requirement that they’d previously relaxed. For certain kinds of software built after June 1, 2019 and distributed on the internet (but outside the App Store), users running Catalina will see an error message unless the developer has gone through the extra step of notarization. That can include command-line software, though the vast majority of command-line software you download (see Install New Software) will either be old enough that this requirement doesn’t apply or new enough that developers have notarized it. You’ll hardly ever run into problems.

Nevertheless, it could occur that you download, install, and run a new command-line program only to see a warning like the one in Figure 1.

Figure 1: A non-notarized command-line app named piffle would produce an error message like this one when you try to run it.

Should this happen to you, don’t worry. As long as you are confident that you downloaded the software from a reputable source (or are willing to take the small risk of encountering malware), you can bypass this warning:

  1. Click Cancel in the initial alert—not Move to Trash!

  2. Open System Preferences > Security & Privacy > General.

  3. At the bottom, you should see a message along these lines:

    Program name” was blocked from use because it is not from an identified developer.

    Next to that is a button that says Allow Anyway. Click it!

  4. Try running the app again. This time you’ll get a slightly different warning:

    macOS cannot verify the developer of “program name”. Are you sure you want to open it?

    Click Open.

The program then runs, and macOS won’t pester you about that program in the future.