1.0) Overview
2.0) Using the Shell
    2.1 Command Prompt
    2.2 Environment Variables
    2.3 Standard Input Redirection
    2.4 Standard Output Redirection
    2.5 Pipeline Commands
    2.6 File Globbing
3.0) Common Commands
    3.1 Filesystem Commands
    3.2 Data Manipulation Commands
    3.3 Process-Related Commands
    3.4 Internet-Related Commands
    3.5 Commands That Teach or Inform
4.0) Filesystem Permissions
    4.1) Working With Permissions
5.0 Shell Scripts
6.0) Links for Further Study and Reference

1.0 Overview

You can access a huge variety of commands and services from the unix command line using the Terminal application, found in:

        /Applications/Utilities/Terminal.app

When you launch Terminal it gives you a window with a command prompt.

You type in commands and press the Return or Enter key and the system executes the commands. The results are typically displayed in the Terminal. When the command is finished you will typically get another prompt.

Commands begin with a program name and then you may provide "arguments" to specify what the program should do or on what files or other resources it should operate. Every program has its own arguments, but there are some common conventions. Options often begin with a hyphen and often appear before any files or directories. The order of the arguments often matters.

You can learn more about any command using "man command" where command is a command you want to learn about. For example "man sort" or "man more".

Manual pages shown by the man command can be painfully terse and surprisingly free of useful examples.

You can have any number of Terminal windows open. In Leopard, Terminal windows can have tabs, like an internet browser.

You can certainly use OS X and never actually use Terminal. But some of OS X's more advanced capabilities and diagnostic tools are only available via the Terminal application. And, sometimes the fix for a problem is most easily accomplished via Terminal. So, some basic knowledge about it can be very useful.

2.0 Using the Shell

The Terminal window is running a program that collects commands from you and hands them to the operating system to be executed. That program is called a shell. There are several popular shells: sh, ksh, bash, csh, tcsh

All shells have some things in common:

• Command Prompt
• Environment Variables
• Standard Input Redirection
• Standard Output Redirection
• Pipeline Commands
• File Globbing (matching filenames with simple patterns)

The examples are from my computer, which is called "Toybox" (from the anime series "Planettes", which I highly recommend by the way).

2.1 Command Prompt

The last thing a new terminal window will display before accepting your first command is its shell's command prompt. In all the examples below, the command prompt is a number followed by 'Toybox% '. You never type the prompt -- it is just telling you that the shell is ready for your next command.

2.2 Environment Variables

Like variables in any language, environment variables have a name and a value. They store useful pieces of information that affect the behavior of the shell and some of the programs or commands you may run from the shell.

Example 1:

        1 Toybox% echo $SHELL
        /bin/csh

Example 2:

        2 toybox% echo $PATH
        /bin:/usr/bin:/usr/sbin:/Users/hoco/bin:.:/sbin:/usr/local/bin:/bin:

2.3 Standard Input Redirection

Unix systems are based on some relatively ancient concepts in computer science, steming from the days when everyone used a 24x80 character terminal tied to a central computer.

At that time, pretty much all programs you might care to run would read text input and provided text output. They also provided text error messages.

Unix has standardized concepts called "Standard In", "Standard Out" and "Standard Error" to reflect the notion that all programs had at least these standard places to read and write information.

This is still true today. If you do nothing special, all input comes from the your keyboard (when the Terminal Window is in the foreground), and all output (both standard out and standard error) go to the terminal window as well.

Try it!

        Example 3:
        4 toybox% cat

The example above runs the cat command (short for concatenate). Cat copies whatever it receives on its standard input to its standard output.

Type whatever you want and press the Return Key. You will see that it is printed right back to you. That happens because it is reading from the keyboard. When standard in is your keyboard, unless special steps have been taken by the program, you will see whatever you type.

You are seeing your own input because after you press the Return key the data is written to the standard output.

When you are done seeing your own output, press Control-D (hold down the control key and press the D key). This is how you signal "end of data on standard input" to the shell.

Redirecting Standard Input means taking the data from a file on disk or some other source rather than from the keyboard. Example 4:

        4 toybox% cat < /etc/shells
        # List of acceptable shells for chpass(1).
        # Ftpd will not allow users to connect who are not using
        # one of these shells.

        /bin/bash
        /bin/csh
        /bin/ksh
        /bin/sh
        /bin/tcsh
        /bin/zsh

2.4 Standard Output Redirection

Like Standard Input, you can also redirect Standard Output. This simply means that what would have been written to your terminal will instead be written to wherever you redirect it.

Example 5:

        5 toybox% cat < /etc/shells > /dev/null

Nothing has happend to /etc/shells. The file was not affected.

2.5 Pipeline Commands

One of the important concepts in unix is that the output of one program can serve as the input of another. When programs are connected this way we call the combination a pipeline. Sometimes we say we will pipe one program to another program to indicate we will arrange them so that one program gets its input from the other.

Example 6:

	6 toybox% ls $HOME/library/preferences | more
	AddressBookMe.plist
	ByHost
	CD Info.cidb
	CDDB Preferences
	Explorer
	Inspiration Software
	JPEGDeux.plist
	JpegJet.plist
	Macromedia
	Microsoft
	NeoOffice-2.2 
	OfficeSync Prefs
	QuickTime Preferences
	Reason Preferences
	VLC
	[byte 213]

The ls command lists the files in whatever directory is specified, or the current directory if none is specified. It sends it data to the standard output.

But, the command above uses the pipe character to connect the output of the ls command to the input of the more command.

The more command reads one "screenfull" of data and then pauses for you to press the space bar before continuing with the next screenful.

It reports the byte number within the file, because it doesn't know how much data will be coming through the pipeline. You could also use the more command to look at a long text file.

Example 7:

	7 toybox% more /etc/protocols

	# 
	# Internet protocols                
	#                 
	# $FreeBSD: src/etc/protocols,v 1.14 2000/09/24 11:20:27 asmodai Exp $          
	#       from: @(#)protocols     5.1 (Berkeley) 4/17/89    
	#                 
	# See also http://www.isi.edu/in-notes/iana/assignments/...
	#                
	ip      0       IP              # internet protocol, pseudo protocol number
	#hopopt 0       HOPOPT          # hop-by-hop options for ipv6                
	icmp    1       ICMP            # internet control message protocol
	igmp    2       IGMP            # internet group management protocol
	ggp     3       GGP             # gateway-gateway protocol
	ipencap 4       IP-ENCAP        # IP encapsulated in IP (officially ``IP'')
	st2     5       ST2             # ST2 datagram mode (RFC 1819)
	tcp     6       TCP             # transmission control protocol
	cbt     7       CBT             # CBT, Tony Ballardie 
	[/etc/protocols 11%]

2.6 File Globbing

One of the most common things you do in Terminal is go looking for files and directories. Sometimes you know right where to look and sometimes you look in a few different places, or even use the find command to look through potentially vast numbers of directories.

You can also ask the shell to find files for you given a pattern. If you are in a directory with many images whose names are descriptive, you could use a command like this to find all files that include the word birds:

	ls *bird*

The * character matches zero or more characters. The ? character matches exactly 1 character.

So, you could match all files with a three-letter extension like this:

	ls *.???

Patterns cannot span directories. So, a pattern like *.jpg won't match files in a subdirectory of the current directory. But, you can include slashes in a pattern if you want to look in subdirectories:

	ls */*dirpat*/*.jpg

You could also use a find and grep pipeline like this:

	find . -name '*.jpg' -print | grep dirpat

3.0 Common Commands

Commands fall into many categories, only a few of which are probably useful to you until you have a lot more experience.

• Filesystem Commands (change directories, list directory, rename, copy files, change permissions, find files)
• Data Manipulation Commands (sort data, compare files, match patterns, spellcheck, gunzip)
• Process-related commands (view process list, kill process)
• internet-related commands (ftp, get web page, secure shell, secure copy)
• Commands that teach or inform (man, date, cal)

3.1 Filesystem Commands

command	description
cd	change directories to your home directory
cd ..	change to the directory "above" the current directory
cd somedir	change to the "somedir" directory
ls	list contents of the current directory
ls *.jpg	list contents of the current directory that end in .jpg
ls -l *.jpg	long listing of the contents of the current directory that end in .jpg
ls -t *.jpg	list contents of the current directory that end in .jpg, from most recently modified through the file modified longest ago
ls -rt *.jpg	list contents of the current directory that end in .jpg, from the file modified longest ago through the most recently modified
mv somefile newname	rename the file whose name is "somename" so that its new name is "newname"
cp somefile copyofsomefile	copy the file "somefile" so that the copy's name is "copyofsomefile"
chmod ugo+rw somefile	make "somefile" readable and writable by everyone
chmod ugo+r somefile	make "somefile" readable by everyone
chmod u+x somefile	make "somefile" executable by you
chmod go-rwx somefile	make "somefile" not readable, writable or executable by anyone but you
find . -name '*.jpg' -print	find all files with the extension .jpg beginning in the current directory (.) and continuing in all subdirectories
find /Library /System -name '*.plist' -print	find all files with the extension .plist in the /Library and /System directories, or any subdirectories of those.
rm somefile	remove a file. You can also use patterns, of course, to remove multiple files.
rm -i somefile	ask whether to remove the file -- answer Y to remove it. Very useful when you use patterns, so you can confirm each removal
rm -f somefile	force removal of a readonly file.
rm -rf somedir	recursively remove a directory and all of its contents. DANGER! Be certain you want to remove it all, because it is *so* gone...
rmdir somedir	remove an empty directory

3.2 Data Manipulation Commands

command	description
echo some string	print some string into the terminal window. This is also a useful way to look at the values of an environment variable -- just put $VARNAME onto the command line of echo (where VARNAME is the environment variable you want to see the value of)
cat somefile	print the contents of somefile to the terminal window
more somefile	print the contents of somefile to the terminal window, but pause for every pagefull
less somefile	print the contents of somefile to the terminal window, but pause for every pagefull (uses vi-like keybindings)
vi somefile	invoke the vi editor on somefile
open somefile	invoke whatever application is associated with somefile. E.g, for an image it would invoke Preview.app
head somefile	show the first 10 lines of a file or pipeline. You can pass -20 (or any other number) to show that many lines.
tail somefile	show the last 10 lines of a file or pipeline. You can pass -20 (or any other number) to show that many lines.
tail -f somefile	show the last 10 lines of a file or pipeline and then show any new files that are written to the file
Example: tail -f /var/log/apache2/error_log	Follow the apache error log file, showing any new lines that appear
sort file	sort contents of file
Example: cd $HOME/Music/iTunes/iTunes' 'Music ls | sort --ignore-case /tmp/musiclist.txt
grep pattern file	look for a pattern in a file: note, you need to put your pattern inside quotes if it contains any special characters
grep pattern file1 file2 file3	look for a pattern in three files
Example: grep Subject: $HOME/Library/Mail/POP-*/INBOX.mbox/Messages/*
diff file1 file2	show differences between two files
Example: ls /volumes/adisk/adir/asubdir > /tmp/files.in.adisk.txt ls /volumes/adisk_mirror/adir/asubdir > /tmp/files.in.adisk_mirror.txt diff /tmp/files.in.adisk.txt /tmp/files.in.adisk_mirror.txt
comm file1 file2	compare two sorted files. Shows lines only in file 1, only in file2 and common to both files
spell file1.txt	find spelling errors in a text file.
gunzip archive.gz	unzip a .gz or .zip file

3.3 Process-Related Commands

command	description
ps	short process listing
ps auxww	view complete process listing
top	dynamic process listing, like activity monitor
kill -15 1234	send signal 15 to process 1234 -- graceful shutdown
kill -9 1234	send signal 9 to process 1234 -- ungraceful, immediate shutdown
sudo kill -9 1234	send signal 9 to process 1234 -- ungraceful, immediate shutdown, done as the most priviiged user

3.4 Internet-Related Commands

command	description
ping www.timefold.com	find out if www.timefold.com is connected and responding
ping 192.168.1.1	find out if 192.168.1.1 is connected and responding (e.g. your router)
ftp ftp.mozilla.org	cleartext ftp command line tool
curl http://www.timefold.com	retrieve a web page
curl http://www.timefold.com/timefold/Timefold.html | grep img	retrieve web page and extract only lines that contain images
ssh somehost.com -l username	secure shell connection to somehost.com, as user username
scp somefile username@somehost.com:/tmp	securly copy file to the /tmp directory of somehost.com, as user username

3.5 Commands That Teach or Inform

command	description
man sort	show manual page for the sort command. Practically every command in unix has an associated manual page, but they may be too terse to comprehend.
date	show current date
env	show the value of all environment variables
id	show the current user and all their groups
cal	show calendar for current month
cal 2008	show complete calendar for 2008

4.0 File Permissions

The Unix filesystem is a hierarchical collection of directories that may contain directories and files. The root of the filesystem is / and the forward slash character is also the separator between directories. So, a complete pathname for a file or directory always begins with a slash, like this:

	/Library/WebServer/Documents/PoweredByMacOSX.gif

Every file, directory, etc. has its own set of permissions that control who may read, write and execute (for files) or find through (for directories).

Reading and writing are intuitive.

For regular files, the execute permission turns the file into a program that can be run by the system, or at least, it convinces the system to allow the attempt to run it. Making a file executable is necessary, but not sufficient for a program to actually run.

For directories, the execute permission allows the user to descend through it with the find command, or enter it with the cd command.

Each file, directory, etc. actually has three separate sets of permissions, one each for: user, group, other.

The permissions for "user" apply to the owner of the file. Typically, if you create a file you own it. You probably also own the vast majority of all files you find in your $HOME directory or any subdirectory thereof.

The permissions for "group" apply to all users that belong to the group associated with the file. For a home computer, this isn't very meaningful. For computers in a business environment, it is common for members of a department to all be in the same group, so there can be permissions that apply to the department and that still exclude people not in that department. For example, people in your own department may be able to read all the files of other people in the same department, but people outside the department may not even be able to read the files.

The permissions for "other" apply to everyone else -- that is, users who are not you and also are not in the group associated with the file.

	ls -l /Library/WebServer/Documents/PoweredByMacOSX.gif
	-rw-rw-r--  1 root  admin  3726 Sep 23  2007
		/Library/WebServer/Documents/PoweredByMacOSX.gif

The permissions string is -rw-rw-r--. This is actually four separate parts:

• The first letter tells us the kind of thing it is
• there are three groups of three letters each

-

says this is a regular file

d

says this is a directory

l

says this is a symbolic link (an alias)

c

says this is a character device (e.g. a tty)

b

says this is a block device (e.g. a disk)

The three-letter groups define permissions for user, group and other, respectively.

In the above example, both user and group have read and write permissions, but not execute permissions. Whereas, other has only read permission.

The ls -l command shows us that the file is owned by 'root' and the group is 'admin', and the file is 3726 bytes. The file was modified on Sep 23, 2007.

ls -l /System/Library/CoreServices/dotmacsyncclient
lrwxr-xr-x  1 root  wheel  88 Nov 14  2007
    /System/Library/CoreServices/dotmacsyncclient ->
    /System/Library/PrivateFrameworks/DotMacSyncManager.framework/Resources/dotmacsyncclient

ls -ld /Library/CoreServices
drwxr-xr-x  101 root  wheel  3434 Aug  1 07:18 /System/Library/CoreServices

The above permissions show that /System/Library/CoreServices is a directory, and that it is owned by root and is in group wheel. Root may modify the directory, but everyone else may simple read the directory and find through it.

What does it mean to write to a directory? If you have write access to a directory you may add files to the directory, or remove them. You may also change permissions (and group membership) of files you own in that directory (but not on files you don't own).

In fact, you don't need read or write permission to a file in order to remove it. You only need write permission to the directory that contains it.

4.1) Working With Permissions

You can change permissions on files you own using the chmod command. Here are some examples:

command	description
chmod go-rwx file	remove read, write and execute for group and other
chmod u+x file	make file executable, but just for me. Actually, if the file was already executable for group and/or other, it remains so. This command just adds execute permissions for yourself.
chmod ugo+x file	make the file executable for everyone, but it must also be readable, so you had really better run chmod ugo+rx if you want the file to be able to be run by anyone
chmod +x file	same as chmod ugo+x file, above
chmod go-w file	remove write permission for group and other

Note: you can specify a file or directory where it says 'file' in these commands.

5.0 Shell Scripts

Shell scripts are simply text files that contain shell commands, as if you had typed them at the command prompt. You can create them in your favorite text editor, including the TextEdit application.

A shell script needs to be made executable (chmod +x yourscriptfile). There are lots of possible scripting languages you could use: bourne shell, cshell, tcsh, ksh, bash -- even perl or python! Whatever scripting language you choose, you must give the operating system a hint about which shell to use as the interpreter. The very first line of the file should begin with one of the following sequences, with no blanks in front. It must be the very first line.

For a bourne shell script use:

#!/bin/sh

For a cshell shell script use:

#!/bin/csh

For a tcsh shell script use:

#!/bin/tcsh

For a ksh shell script use:

#!/bin/ksh

For a bash shell script use:

#!/bin/bash

For a python script use:

#!/usr/bin/python

For a perl script use:

#!/usr/bin/perl

Shell scripts may also signal their success or failure with an exit code. By convention, an exit code of 0 means success, and any other number means failure.

A script may set its exit code only when it terminates, using the exit command. For example, to signal success, put this at the end of your script:

	exit 0

Each shell has its own syntax, although for very simple commands you may find that more than one shell can interpret the commands. For example, sh, csh, tcsh ksh and bash could all run the following commands:

cd /User/$LOGNAME/birdpix
find . -name '*.jpg' -print | 
	grep -i bird | 
	sort > /tmp/home.birds.txt

cd /volumes/backup/User/$LOGNAME/birdpix 
find . -name '*.jpg' -print |
	grep -i bird | 
	sort > /tmp/backup.birds.txt

comm -23  /tmp/home.birds.txt /tmp/backup.birds.txt

Then it does the same thing for a mounted backup disk (presumably cloned from your computer's hard drive at some point in the past.) We assume the disk is called 'backup' and is mounted at /volumes/backup (which is the normal place a disk called 'backup' would be mounted.) It produces its filtered, sorted list of bird files and directories in /tmp/backup.birds.txt

Finally, it uses the comm program to compare the results. The -23 option tells comm to report only lines unique to /tmp/home.birds.txt. In other words, it reports all the bird-related files and directories you've added since the backup has been made. It doesn't report items you've deleted though. If you'd like to see those instead, use comm -13 instead.

Here is a simple bourne shell script called 'isrunning':

#!/bin/sh

prog="$1"
running=`ps auxww | grep "$prog" | grep -v grep | grep -v isrunning`
if [ "$running" = '' ]
then
    echo "$prog is not running"
    exit 1
else
    echo "$prog is running"
    exit 0
fi

You can use the isrunning script to determine if a process is already running. For example, I have another script that depends on the mysql database, which may or may not already be running on my computer. MySql is, unfortunately, not automatically started when my machine reboots.

So, I include this in my bourne shell script that needs mysql:

isrunning mysql
running=$?
if [ "$running" != 0 ]
then
    echo "dget: mysql deamon is required"
    exit 1
fi

The differences between the command-oriented shells are primarily the syntax you use for more complicated things like loops, if-the-else logic and the way you set and retrieve environment or shell variables. While $LOGNAME will always be your (short) unix login name, some shells provide more subtle ways to access data, and different shells have different syntax for setting variables and for limiting or extending the scope of variables.

perl and python are really robust programming languages -- and they do not really make very good command-oriented shells. You could not run the above commands in either perl or python. You can see an example of a perl script here: The driveio tool

The command-oriented shells like sh and csh are really not very convenient as programming languages, and you quickly run into their limitations if you try to do anything too complicated. But they are adequate or even convenient for simple tasks.

The bash, ksh and tcsh shells are better, but still, if you are a programmer you are much more likely to appreciate a programming language like perl or python for any serious software project, and you'll probably want to use shell scripting for only fairly simple tasks. You will probably also prefer bash over csh or tcsh for programming.

Each shell has its strengths and weaknesses. I prefer to write simple scripts in sh and more complicated scripts in perl. If you want to learn only one shell or language, bash is a good choice. Perl is a mature and remarkable language with extensive libaries of free software available, but it has a steep learning curve, like any significant programming language. For simple scripts, it is probably not practical to learn it. Python is easier to learn, but still overkill for simple scripts.

6.0 Links for Further Study and Reference

• http://cb.vu/unixtoolbox.xhtml or http://cb.vu/unixtoolbox.pdf
• terminal-commands-for-hidden-settings-in-leopard
• 15 useful shell commands
• ten-os-x-command-line-utilities-you-might-not-know-about
• Undersdtanding the grep command
• OS X's BSD/unix command-line
• Learning the bash Shell, O'Reilly & Associates
• bash Cookbook, O'Reilly & Associates
• Classic Shell Scripting, O'Reilly & Associates
• Mastering Regular Expressions, O'Reilly & Associates
• Mac OS X for Unix Geeks, O'Reilly & Associates