============================
Some Essential Unix Commands
============================

GETTING AROUND DIRECTORIES & FILES

Like DOS, Unix organizes information by putting *files* in *directories*. Breaking information down into directories helps to make a large body of information more manageable.

A directory is like a folder, which can contain either information (a file), or more folders (called subdirectories). Each directory can contain any number of files or subdirectories, and every file and every directory has a name, made up of letters and numbers. The full name of a file includes its "path" -- in other words, the complete directory structure, with the directories and subdirectories separated by slashes (forward slashes, "/", rather than the backslash, "\", of DOS). A file called "file1" in the directory /home4/jqpublic would have the full path /home4/jqpublic/file1.

The "working directory" is the directory you are in at the moment. When you are in a working directory such as /home4/jqublic, most commands by default apply only to that directory. To work with other files, it's often easiest to change into another directory (see "cd," below).

pwd -- Print Working Directory.

ls -- Display a list of files and directories.

cd -- Change Directory. cp -- Copy a file. mv -- Move or rename a file. rm -- Remove a file. rmdir -- Remove a Directory. mkdir -- Make a Directory.

FILE PERMISSIONS

Unix is a multi-user operating system, meaning many people are on the same system at once. To ensure security, some files are restricted: only certain people can read or write them. This is known as file permissions.

When you use "ls -l" ("ls" for list files, "-l" for "Long format"), the far left will include a pattern of letters like this:

-rwxr-xr--

The ten positions here indicate the file permissions. The first (leftmost) position can be either "d" or "-" -- "d" means the entry is a directory, while "-" means it's a file.

The remaining positions come in three groups of three. Each group consists of three positions: "r" or "-"; "w" or "-"; "x" or "-." "r" means "Read permission"; "w" means "Write permission"; "x" means "execute permission." The most important are Read and Write permissions: Read permission is what it sounds like: it means the User, Group, or Other can look at the contents of the file. Write permission means the User, Group, or Other can change or delete the file. An "r" means read permission is enabled, while a "-" means it isn't; "w" means write permission is enabled, and "-" means it isn't.

Each group of three, then, shows read, write, and execute permission. The three groups are: User, Group, and Other. The User is the person who created (or owns) the file; the Group is a collection of people (defined on the system) who have similar permissions; Other is anyone else on the system.

The example above, therefore --

-rwxr-xr--

-- can be broken down this way: The first position is a "-" instead of a "d," so it's a file, not a directory. The User's permission -- the next three characters -- is "rwx," meaning the person who created the file can read, write, and execute the file. The Group permission is "r-x," meaning the Group can read and execute, but not write. The Other permission is "r--," meaning Others can read but not write or execute.

You'll want to set permissions carefully. By default, most files you create can be read and written by you alone. If you want others to have access to the files, you'll have to set the file permissions to allow other people -- either those in your group or everyone -- to read or write the files. Be careful not to allow others to tamper with your files. Your personal mail, for instance, should give read and write permission only to you: -rw-------. If you want someone else to be able to see the contents of your file, give read permission to either the group or to other, but keep write permission only for yourself. If you give write permission to other, anyone on the system can change or delete your files.

To change file permissions, use the "chmod" command.

chmod -- Change File Mode.

LOOKING AT FILES

You can always use a text editor (such as pico, emacs, or vi) to look at a file. Some other methods are sometimes useful:

OTHER FILE OPERATIONS

grep -- Search for a String. sort -- Sort a file.

"sort" will put the contents of a file into alphabetical order. For instance, you can type

sort file1
and get a list of all the lines in file1 sorted in alphabetical order. The output will appear on the screen.

    A few flags are useful:
  1. sort -b -- Ignore leading tabs and spaces.
  2. sort -f -- Sort upper- and lowercase together.
  3. sort -r -- Sort in reverse order.
  4. sort -u -- Skip duplicates. If two lines are identical, "sort -u" will display only one of them.
cut -- Look at part of each line.

"cut" lets you select just part of the information from each line of a file. If, for instance, you have a file called "file1" with data in this format:

        0001 This is the first line
        0002 This is the second
and so on, you can look at just the numbers by typing
cut -c1-4 file1
The "-c" flag means "columns"; it will display the first four columns of each line of the file. You can also look at everything but the line numbers:
cut -c6-100 file1
will display the sixth through one hundredth column (if the line is less than a hundred characters -- and most will be -- you'll see up to the end of the line).

You can also use cut to look at fields instead of columns: for instance, if a file looks like this:

        curran:Stuart Curran
        jlynch:Jack Lynch
        afilreis:Al Filreis
        loh:Lucy Oh
you can use cut to find the full name of each person, even though it's not always in the same place on each line. Type
cut -f2 -d: file1
"-f2" means "the second field"; "-d:" means the delimiter (the character that separates the fields) is a colon. To use a space as a delimiter, put it in quotations:
cut -f2 -d" " file1
sed -- Stream Editor.

"sed" is a sophisticated tool that allows you to perform large-scale search-and-replace operations on a file. There are many possibilities; the most important are "s" ("Substitute") and "d" ("Delete").

sed "s/colour/color/" filename
will take the entire file called "filename" and replace every occurrence of the pattern "colour" with "color", then display the output to the screen. You can also use "s" to remove a pattern of letters:
sed "s/quite//" filename
will replace "quite" with "" -- in other words, remove it. You can also use sed to delete entire lines:
sed "/light/d" parlost.01
will delete every entire line of the file parlost.01 that contains the word "light."

REDIRECTION & PIPING

By default, Unix sends the output of all these commands to the screen. But this isn't necessary. Unix lets you take the output of one command and either save it in a file, or turn it into the input of another command, allowing you to chain several commands together. You can do two things with output: (1) redirect it into a file; and (2) pipe it into another command.

REDIRECTION

The ">" (greater than) symbol lets you take the output of a command and save it to a file. For instance, "ls" alone will display the contents of the directory on the screen. Typing
ls > dirfile
will save that output to a file called "dirfile" (it won't appear on the screen). Programs such as cut, sort, grep, and sed can produce very long outputs; you might want to save them to a file. For instance, to translate "colour" to "color" you can use sed, but the output will scroll past you too quickly to see. Use redirection to save it to a file:
sed "s/colour/color/" file1 > file1.out
will produce a new file called "file1.out" that has changed the spelling of "colour" to "color" in every line.

You can use redirection to join two files together with "cat":

cat file1 file2 > file3
will take file1 and file2 and combine them into a new file called file3.

WARNING: "ls > dirfile" will overwrite "dirfile" if it already exists. If you want to add the contents of the directory to the end of a pre-existing dirfile, type

ls >> dirfile
-- the ">>" means "add to the end."

PIPING

The "|" symbol lets you take the output of a command and send it to the input of another command. For instance, you take the output of the "ls" command and sort it in reverse order:
ls | sort -r
To see just the first five matches of the word "light" in a group of files, type
grep "light" * | head -5
If the search for the word "light" produces too many matches, you can see them a screen at a time by typing
grep "light" * | more
There's no limit on the size of the chain you can put together. To see all the files in a directory created in January, sort them by name, and save them to a file, you can type
ls -l | grep " Jan " | cut -c55-100 | sort > janfiles

GETTING HELP

man -- Display the manual pages for a command.

"man" is the Unix "help" -- type, for instance, "man grep" and you'll get the complete instruction manual for the grep command.

apropos -- Search for commands that relate to a particular topic.

"apropos directory" will display every command that includes the word "directory" in its brief description. You can then use "man" to get a fuller description of the command.