Work with Programs – Take Control of the Mac Command Line with Terminal, 3rd Edition

Work with Programs

Every command that you use on the command line, including merely listing files, involves running a program. (So, in fact, you’ve been using programs throughout this book!) However, some aspects of using programs on the command line aren’t entirely obvious or straightforward. In this chapter, I explain some of the different types of programs you may encounter and how to run them (and stop them).

I also show you how to edit files on the command line, and I talk about shell scripts, a special kind of program you can create to automate a sequence of tasks.

Learn Command-Line Program Basics

If you’ve been reading this book in order, you already know many basics of running programs on the command line. Each time you enter a command such as ls or cp or pwd, you’re running a program—and we saw how to change program options and supply additional parameters with arguments and flags earlier (in What Are Commands, Arguments, and Flags?). However, I think you should know a few other important facts about running programs.

Command-line programs come in a few varieties, which I’ll lump together in three broad categories. (These are my own terms, by the way; other people may categorize them differently.) You’ll have an easier time using the command line if you’re aware of the differences.

Basic Programs

Most command-line programs you use simply do their thing and then quit automatically. Enter ls, for instance, and you instantly get a list of files, after which point ls is no longer running. Some of these single-shot programs produce visible output (date, ls, pwd, etc.); some normally provide no feedback at all unless they encounter an error (cp, mv, rm, etc.). But the point is: they run only as long as is needed to complete their task, without requiring any interaction with you other than the original command (with any flags and arguments).

Interactive Programs

A second type of program asks you for an ongoing series of new commands, and in some cases doesn’t quit until you tell it to. For example, the command-line program used to change your password is passwd. If you enter passwd, you see something like the following:

Changing password for jk.

Old password:█

You type your old password and press Return, and then the program gives you another prompt:

New password:█

Type in a new password and you get yet another prompt:

Retype new password:█

Reenter your new password, as long as it matches the first one, the program changes your password and exits without any further output.

Programs of this sort include ssh, which lets you Log In to Another Computer, and sftp, which lets you transfer files between computers, among many others. If you’re running an interactive program, want to quit it, and can’t find an obvious way to do so, try pressing Control-C (see Stop a Program for more possibilities).

Full-Window Programs

The third broad category of programs is full-window programs—those that are interactive but, instead of handling input and output on a line-by-line basis, take over the entire Terminal window (or tab). You’ve already tried a few of these—less and man are examples. Some full-window programs helpfully display hints at the top or bottom of the window showing what commands you can use; others require that you’ve memorized them (or can look them up in a man page, perhaps in another window). As with other interactive programs, pressing Control-C usually lets you exit a full-window program if you can’t find another way to do so.

Run a Program or Script

Often, running a program requires nothing more than typing its name and pressing Return. However, the process can be a bit trickier in certain cases. Read on to discover how to run programs located in unusual places, as well as scripts (programs packaged in a somewhat different form).

How Your PATH Works

As you know already (see Understand How Paths Work), each file on your Mac has a path—a location within the hierarchy of directories. So a path of /Users/jk/Documents/file1 means file1 is in the Documents directory, which is in turn in the jk directory, which is in Users, which is at the top, or root, level of the disk (signified by the initial /).

But there’s another, specialized use of the term PATH: when capitalized like this, it refers to a special variable your shell uses that contains a list of all the default locations in which a shell can look for programs.

To run a program, your shell must be able to find it. But so far, all the commands you’ve entered have been “bare” program names without specific paths given. For example, to run less, you simply enter less, but in reality the program you’re running is stored in /usr/bin. Looking everywhere on your disk for a program would be time-consuming, so how can your shell find it in order to run it? The answer is that when you enter a command without an explicit path, the shell automatically looks in several predetermined locations. That list of locations, which happens to include /usr/bin, is your PATH.

By default, your PATH includes all of the following directories:

/bin
/sbin
/usr/bin
/usr/local/bin
/usr/sbin

A program in any of these locations is said to be “in your PATH.” You can run a program in your PATH, regardless of your current location in the directory structure, simply by entering its name. I encourage you to look through these directories (try ls -l /bin, ls -l /sbin, and so on) to get an idea of the hundreds of built-in programs and where they’re located.

Most programs you’ll need to run are already in your PATH, and if you download or create new programs, you can put them in one of these locations to make sure you can run them just by entering their names. But what about programs that aren’t in your PATH? You can either enter the program’s full or relative path (for example, /usr/local/bin/stuff or ../software/myprogram), or you can expand your PATH to include other directories (I explain how in Modify Your PATH).

Run a Program

To summarize, you can run a program in any of three ways, depending on where the program is located, your current position in the directory structure, and what’s in your PATH:

  • By relative or absolute path: You can always run a program by entering its complete path, such as /usr/bin/less, or its relative path from the current location, for example apples/oranges/program.

  • In the current directory: If you’re in the same directory as the program you want to run, you might think you could just enter the program’s name, but that doesn’t work. Instead, you must precede the name with ./ (and no space). For example, to run a program named counter in the current directory, enter ./counter.

  • In your PATH: To run a program anywhere in your PATH, simply enter the program’s name—for example, less, mkdir, or man.

Run a Script

In macOS, as in other varieties of Unix, the programs you run are usually compiled binary files. If you were to open them in a text editor, they’d look like nothing but garbage characters, because they’ve been put into an optimized, self-contained, machine-friendly format for maximum performance. However, another broad category of programs consists of human-readable text that’s interpreted by the computer as it runs instead of being compiled in advance. Programs in this category are often referred to as scripts, and they’re often used to automate or simplify repetitive activities. Just as AppleScript provides a way of writing human-readable programs that run in the Mac’s graphical environment, scripts of various kinds can run from the command line.

A shell script is a series of instructions interpreted, or run, by the shell itself. So, a shell script could consist of little more than a list of commands, just as you would type them manually in a Terminal window. Run the script, and the shell executes all those commands one after the other. (In fact, shell scripts can use variables, conditional tests, loops, math, and much more—I introduce you to these items later, in Add Logic to Shell Scripts.) I explain the basics of creating a simple script ahead in Create Your Own Shell Script. By convention, shell scripts usually have an extension of .sh (regardless of which shell they use); see Run a Shell Script from the Finder for an exception.

Other kinds of scripts are written in scripting languages such as Perl, Python, and Ruby, and run by the corresponding interpreter. Perl scripts, by convention, end in the .pl extension, Python scripts in .py, and Ruby scripts in .rb.

Regardless of a script’s extension, it’s considered good programming practice to include the name and location of the interpreter that should process it on the first line of the script. For example, if a shell script is intended to be interpreted by the sh shell, the first line should be:

#!/bin/sh

The #! at the beginning of this line, called a “shebang,” is a marker indicating that what follows it is the path to the interpreter. (You can examine a script using, say, less or cat to see if it has such a line.)

Because the interpreter is spelled out right in the script, you can run the script just as you would any other program, by entering its name (if it’s in your PATH) or its path, and the shell you’re currently using need not be the same as the shell specified in the script.

However, if a script doesn’t include that line, you must tell it explicitly which shell or other interpreter to run it with. You do that by entering the interpreter’s name with the path to the script as an argument. For example:

sh ~/Documents/my-shell-script.sh

perl ~/Documents/my-perl-script.pl

python ~/Documents/my-python-script.py

ruby ~/Documents/my-ruby-script.rb

Run a Program in the Background

Most of the time when you run a program, it does its thing, and then you quit it (or it quits by itself). While it is running—whether that takes a second or an hour—it takes over your shell and thus the Terminal window or tab in which the shell is running. If you expect a program to take some time to complete its task, or if you want it to keep running even after you exit the shell, you can run it in the background. Background programs let you do other tasks in the same Terminal window or tab, and, if necessary, they keep going even after you quit Terminal.

To run a program in the background, you simply put a space and an ampersand (&) after the program name (and any flags or arguments). For example, suppose you want to compress a folder containing hundreds of large files. Ordinarily, you might use a command like zip -r archive.zip apples. To run that command in the background instead, enter this:

zip -r archive.zip apples &

While a program is running in the background, you’ll see no feedback or output. If it’s a program that simply accomplishes a task (such as copying or compressing files) and then quits automatically, then you’ll see a message stating that it’s finished—not immediately afterward, but the next time you execute a command or even just press Return to create a new command line. The message saying a process is finished looks something like this:

[1]+  Done          zip -r archive.zip apples

See What Programs Are Running

Here’s a thought question: How many programs are running on your Mac right now? If you glance at the active icons in your Dock and conclude that the number is, say, a dozen, you haven’t even scratched the surface. For example, as I type these words, my Dock tells me I have 16 programs running, but in reality the total is over 500! Besides the visible programs like Mail and Safari, that figure includes background programs that are part of macOS—the Spotlight indexer, Time Machine, logging tools, and many others that perform important but little-noticed functions behind the scenes. It also includes my zsh shell running in Terminal, and every program running in that shell.

You may be aware of Activity Monitor (in /Applications/Utilities), which lists all this information and more. In the command-line environment, too, you can list all your Mac’s processes (visible and invisible) and get a variety of useful facts about them. The two most commonly used command-line programs for discovering what’s running on your Mac are top and ps.

Top

The top command is the nearest command-line equivalent to Activity Monitor. Enter top and you get a full-window list of all your running processes, updated dynamically. Figure 5 shows an example.

Figure 5: In the top window, you get a list of all the processes currently running on your Mac.

By default, the top command lists several pieces of information for each process, including the following particularly interesting ones: PID (process ID), COMMAND (the process name), %CPU (how much CPU power the process is using), TIME (how long the process has been running), and MEM (how much RAM the process is using).

I won’t go into great detail about everything you see here (try man top to learn more), but I do want to call your attention to a few salient points and offer some top tips:

  • Pruning the list: You almost certainly have many more processes than can fit in your window at one time, even if you make your window very large. So you can restrict the number of items top shows at a time using the -n (number) flag, followed by the number of items to show (top -n).

  • Sorting the list: By default in recent versions of macOS, top lists processes in reverse order of %CPU, which means the processes at the top of the list are the ones that have been using the most CPU power recently. You can adjust the sort order with the -o (order) flag—for example, enter top -o pid to list processes in order of process ID (roughly, in order of how recently they were launched), or enter top -o rsize to list processes in order of RAM usage.

  • Top at the top: Depending on what else is running on your Mac at the moment, top itself may be at or near the top of the list, even when sorted by CPU usage. Don’t be alarmed: the effect is caused by the way top gathers its data.

  • Customizing the list: You can combine flags to customize your display. For example, enter top -n 20 -o cpu to list only the top 20 processes by CPU usage.

  • Quitting: To quit top, just type the Q key (by itself).

Ps

Whereas top is dynamic, you may want simply to get a static snapshot of the processes running at any moment. For that, the best command is ps (“process status”). If you enter ps by itself, you get a list of your processes running in terminals—which usually means the Terminal app. In all likelihood, this is just zsh or bash itself.

The list includes the PID, the TTY (or terminal name), time since launch, and command name for each process:

  PID TTY           TIME CMD

22635 ttys001    0:00.06 zsh

You can expand the amount of information that ps provides using flags. For example, to include not only processes in the current shell session but also those from other sessions (yours or other users’), enter ps -a. To show processes that aren’t running in a shell at all (including regular macOS apps and background processes), enter ps -x. Combine the two (ps -ax) to show all the processes running on your Mac.

Of course, although ps -ax provides lots of information, it might be too much to be useful. You can filter the output from the ps command by using a couple of spiffy Unix tricks. First, add the pipe (|) character (type Shift-\) to channel the output from ps into another program. (For more on the pipe, see Pipe and Redirect Data, later.) The other program, in this example, is grep, a powerful pattern-matching tool we’ll see again in Get a Grip on grep. So, enter ps -ax | grep followed by a space and some text, and what you get is a list of all and only the running processes whose listing includes that text. For example, to list all processes running from inside your /Applications directory, enter:

ps -ax | grep /Applications

Or, to show only the processes whose names include the characters sys (in any combination of upper- and lowercase), try this:

ps -ax | grep -i sys

Stop a Program

As we’ve seen, most command-line programs quit automatically when they finish performing their functions, and full-window programs usually have a fairly obvious way of quitting them (for example, pressing Q in the case of less or man. However, if a program doesn’t quit on its own, or if you need to unstick one that’s stuck (even if it’s a graphical macOS app!), you can use one of several techniques.

Ask Politely

If a command-line program won’t quit on its own, the first thing to try is pressing Control-C. In this context, it’s approximately like pressing ⌘-Q in a regular macOS app—it tells the process to quit, but to do so in a controlled way (saving open files and performing any other necessary cleanup operations).

Kill (Humanely)

What if you want to stop a program that’s not running in the current shell? If it’s a graphical macOS app, or an invisible background process, or a program running in another shell, you can send it a “Quit” signal remotely. The command you use to do this is kill. That sounds extreme, but, in fact, when kill is used on its own, it sends a program the same sort of polite request to terminate that Control-C does.

The catch is that you have to know how to refer to the program you want to kill. Here there are two options:

  • By PID: If you can find the process’s PID (process ID)—using top, ps, or even Activity Monitor—you can simply enter kill followed by that number. For example: kill 1342

  • By name: If you don’t know the process’s PID, or can’t be bothered to find out—but do know its name—you can quit it by name using a variant of kill called killall. Simply follow killall with the program’s name. For example: killall Safari

    You must enter the name exactly as it appears in top, ps, or Activity Monitor. For example, if you want to quit Excel, you must enter killall "Microsoft Excel" (quotation marks added because there’s a space in the name).

Kill (with Extreme Prejudice)

If a program fails to respond to Control-C or to the standard kill or killall command, it’s time to pull out the big guns. By adding the -9 flag to the kill command, you turn a polite request into a brutal clobbering that can terminate almost any process.

When you use the kill -9 command, you must give it the process’s PID; the -9 flag doesn’t work with killall to force-quit a process by name. For example:

kill -9 1342

If even kill -9 doesn’t stop a process, and I’ve seen that happen more than once, it is likely stuck beyond the power of any software command, and if logging out of your user account doesn’t solve the problem, your only choice is to restart the computer.

Edit a Text File

Earlier I showed you how to view the contents of text files, but you may also need to modify them. For that, you can work with any of several command-line text editors. Using a command-line text editor is often quicker and easier than opening a text file in a program like TextEdit—especially for files that don’t appear in the Finder—and is less likely to cause problems with file formats or permissions.

If you ask a hardcore Unix geek what text editor they use, they will probably answer vi. (That’s “vee-eye,” not “vie,” by the way.) It’s a very powerful text editor that’s been around forever, and because a lot of programmers cut their teeth on vi and then proselytized future generations, it’s become a sort of badge of honor to be skilled in using vi.

macOS includes vi, but I’m not going to tell you how to use it. As command-line programs go, vi has the most opaque user interface I’ve seen. Until you get used to vi’s oddities and memorize its commands, you can’t even change a letter in a text document without referring to a manual. Powerful or not, from a usability standpoint, vi is hideous. I just want you to know about vi so that when someone asks you why you don’t use it, you can give the correct response: “Life is too short.”

Happily, you can use several other fine text editors. There’s the venerable emacs, which is less obnoxious than vi while still being fabulously flexible. But I’m going to recommend what you might think of as the TextEdit of command-line text editors: a simple, straightforward, and adequately powerful program called nano.

To edit a text file in nano, use a command like the following:

nano file1

If file1 is already present, nano opens it; otherwise, it opens a blank file that will be called file1. Figure 6 shows a text file open in nano.

Figure 6: A text file open in the nano text editor. The menu of keyboard controls is at the bottom of the window.

One of the reasons nano is easy to use is that editing is straightforward. To insert text at the cursor location, simply type—or paste the contents of your clipboard by choosing Edit > Paste or pressing ⌘-V. To delete the character to the left of the cursor, press the Delete key; to delete the character at the cursor, press the Forward Delete key (if your keyboard has one). To delete the entire current line, press Control-K.

Other than those basics, here are the most important things you should know how to do in nano:

  • Save: To save the current file, press Control-O (WriteOut).

  • Quit: To quit nano, press Control-X (Exit). If you’ve made any changes to the document that you haven’t yet saved, nano prompts you to save the file before exiting. Press N to discard changes and quit immediately, C to cancel and stay in nano, or Y to save changes and exit. If you do save changes, nano verifies that you want to keep the existing filename (if you don’t, you can type a new one). Press Return after verifying the filename.

  • Find: To find text within the file, press Control-W (Where Is). Type the text you’re searching for (case doesn’t matter) and press Return. The cursor jumps to the next spot in the document where that string appears. Repeat this procedure to do additional searches.

Those commands alone should enable you to do almost everything you need to do in nano. To learn about additional nano features and shortcuts, press Control-G to view its online help.

Create Your Own Shell Script

Before I wrap up this discussion of running programs, I want to give you a tiny taste of creating your own shell scripts. Scripting is a bit like learning chess: you can pick up the basics in a few minutes, but it may take years to master all the subtleties. So I’m not going to teach you anything about programming as such, just the mechanics of creating and using a simple script. I want you to have enough familiarity with the process that you can successfully reproduce and run shell scripts you may run across in magazines, on websites, or even in this book (see Command-Line Recipes, which includes a couple of shell scripts). Later on, for those who are interested in learning a bit more, I’ve included instructions on how to Add Logic to Shell Scripts.

You can create and run a shell script in six easy steps; in fact, you can arguably combine the first four into a single process. But one way or another, you must make sure you’ve done everything in the list ahead.

Step 1: Start with an Empty Text File

Scripts are plain text files, so you should begin by creating one in a text editor. You can make a shell script in TextEdit, BBEdit, or even Word, but that requires extra steps. So I suggest using nano, as described in Edit a Text File. For the purpose of demonstration, name your script test.sh. (Remember from Run a Script that the .sh extension isn’t mandatory, but it can help you keep track of which files are shell scripts.)

Before you create this file, I suggest using cd (all by itself!) to ensure that you’re in your home directory. (You can put scripts anywhere you want, but for now, this is a convenient location.) That done, enter nano test.sh. (As a reminder, the extension doesn’t really matter—and it certainly doesn’t depend on what shell you’re using. It’s just an arbitrary convention to end shell scripts in .sh.) The nano text editor opens with a blank file.

Step 2: Insert the Shebang

The first line of your script should include the “shebang” (#!) special pointer (see Run a Script) to the shell it will use. Since we’re focusing mainly on the zsh shell, we’ll use that one. Type the following line:

#!/bin/zsh

Step 3: Add One or More Commands

Below the shebang line, you enter the commands your script will run, in the order you want them executed. Your script can be anything from a single one-word command to thousands of lines of complex logic.

For now, let’s keep things simple. Starting on the next line, type this:

echo "Hello! The current date and time is:"
date
echo "And the current directory is:"
pwd

The echo command simply puts text on the screen—and you’ve seen the date and pwd commands. So, this script displays four lines of text, two of which are static (the echo lines) and two of which are variable.

Step 4: Close and Save the File

To save the file, press Control-O and press Return to confirm the filename. Then press Control-X to exit nano.

Step 5: Enable Execute Permission

The only slightly tricky thing about running scripts—and the step people forget most often—is adding execute (run) permission to the file. (I say more about this later, in Understand Permission Basics.)

To do this, enter:

chmod u+x test.sh

Step 6: Run the Script

That’s it! To run the script, enter ./test.sh. It should display something like this:

Hello! The current date and time is:
Wed Feb 5 19:58:21 PST 2020
And the current directory is:
/Users/jk

For fun, try switching to a different directory (say, /Library/Preferences) and then run the script again by entering ~/test.sh. You’ll see that it shows your new location.

Any time you need to put a new script on your system, follow these same steps. You may want to store the scripts you create somewhere in your PATH (see How Your PATH Works), or add to your PATH (see Modify Your PATH), to make them easier to run.

Shell scripts can be much more than simple lists of commands. If you want to explore more advanced capabilities, skip ahead to Add Logic to Shell Scripts.