Space Saving Ideas

The first rule of disk usage seems to be that you never have enough. If you have 10 Megabytes, you need 20; if you have 1 Gigabyte, you need 2 Gigabytes. Note that if you run out of disk space, you cannot receive email! Learning ways to save disk space are important, then. The first step in saving space, is to know what there is and what you have. Start with a "df -k" command to see what is available on the system. You'll get a listing of information on every filesystem on Bama, but here are the 5 most interesting.

Filesystem        kbytes   used     avail    capacity Mounted on 
/dev/dsk/c3t5d0s0 24953411 1819576  20638495 9%       /st1 
/dev/dsk/c3t5d1s0 24955459 1052183  21407736 5%       /st2 
/dev/dsk/c3t5d2s0 26117763 108017   24515239 5%       /st3 
/dev/dsk/c3t5d3s0 26314216 12366081 13421851 48%      /fs 
swap              1648088  22488    1625600  2%       /tmp

The name of the filesystem is in the rightmost column. Here, "/st1", "/st2", "/st3" are the student directories, "/fs" is for faculty and staff, and "/tmp" is an in-memory filesystem. The columns labeled "kbytes", "used", and "avail", are the total amount of space, the amount used, and the amount available, respectively. The percentage of space used is given under "capacity." These are given in kilobytes, so you should picture 24953411 as 24, 953, 411 kbytes. That's almost 25 Gbytes! This listing is the actual one for Bama in early November, 1998. You can see that the faculty/staff directory area is almost half full. So, even though faculty/staff do not have quotas, they can still run out of disk space.

So now that we know what disk usage on the entire system looks like, let's look at individual usage. If you want to save disk space, it is best to find out where you are taking up the most and work from there on down to smaller files. You can see your entire disk usage by typing "du -k | page". This will list your disk usage in kilobytes, directory by directory. With this, you'll get a good idea as to which directories are taking up the most space. Once you know what directory to focus on, you can do an "ls -l" in that directory. In the resulting listing, the number in the 5 th column is the size of the file. Again, a good strategy is to start looking at the big files and move on down to the smaller files. If you have large, unnecessary files, delete them.

What can you do if you have a large file that you want to keep but don't need to use all the time? Here are a few strategies. We'll look at more in a later Tipsheet.

Turn Spaces into Tabs

Files that have lots of spaces might still be readable with the spaces turned into tabs. There is a Unix command, called "unexpand", that will take spaces and convert them to tabs. The default is to take any eight spaces in a row to make one tab. Here is how you use "unexpand":

unexpand -a myfile > newfile

where myfile is the file with lots of spaces and newfile is the file with tabs substituted in. Many terminal programs, like TeraTerm, will interpret the tabs correctly, and your file will look the same when you view it. How do you change tabs back to spaces? With "expand", of course:

expand -a newfile > myfile

Compress Your Files

We have two different compression algorithms available on Bama; compress/uncompress and gzip/gunzip. Gzip seems to make smaller compressed files so it has become a popular option. Any compressed file will be a binary file, so you won't be able to read it or edit it directly. To compress a file with gzip, type

gzip filename

where filename is the original file. The compressed file will automatically receive the suffix ".gz". To get the file back to its original form, type

gunzip filename

You don't need to put on the ".gz" suffix.

If you have an entire directory tree that you wish to compress the best way to proceed is to create a "tar file". Tar originally stood for "tape archive" but these can be written to disk, too. Since small files can take up a minimum of 1 block of space, even if they are smaller than that, lots of small files can take up lots of space. When they are "tarred" together, they are joined into one file, taking up only as much space as needed. Once a directory tree is put into a "tar" file, that file can be compress, thus taking up even less space. Here's a one-step command to tar and compress a directory tree:

tar -cvf - project | gzip > project.tar.gz

What you have done is run the tar command with the "create" option, "c", verbosely, "v", so that you can see what is happening, and the file output option, "f". The file you are writing to is STDOUT (see Tipsheet Vol. 1, No. 4), indicated by "-" and you are tarring the directory called project. This is piped, "|", to gzip and the output is sent, ">", to "project.tar.gz". In this case you must give gzip a name to use since it doesn't get a name through the pipe. Now you can remove the files and the directory "project", since it is safe in the archive. This gzipped, archive file can be transferred to other computers with FTP, Just be sure to transfer it as a binary file.

To get the directory tree back you would extract it like this:

gunzip -c project.tar | tar -xvf -

The "-c" option on gunzip let's you send the uncompressed file on to the tar command without having it create the uncompress file on disk.

Use /tmp for temporary space

There is a /tmp filesystem which anyone can use. It is a virtual filesystem. It exists only in computer memory, so that when the computer is rebooted, all files in it are lost. But it can serve to ease you space crunch while you clean up your directory. For instance, you can put your compressed file or tar archive over there while it is being created, then copy it back to your directory when you have space.

If you are using Netscape on Bama it is not recommended that you set your cache to /tmp. To save space you could set the disk cache size to be zero (0). Go to the "preferences" menu under "edit" and look at the "advanced" section. In this way, you will only cache pages in memory, making them available for that session, but they will disappear when you log out.

© 1998, The University of Alabama. The information included here is for the University of Alabama central computing facility as it was configured on the document date. It may or may not apply to other Unix systems.

November 2, 1998 Vol. 2 Index Back to Vol. 2, No. 9 Ahead to Vol. 2, No. 11		Search:

	E-mail Internet Access Software Assistance Computer Labs Software Licensing Web Publishing Calendaring Computer Security Unix Training

	Space Saving Ideas The first rule of disk usage seems to be that you never have enough. If you have 10 Megabytes, you need 20; if you have 1 Gigabyte, you need 2 Gigabytes. Note that if you run out of disk space, you cannot receive email! Learning ways to save disk space are important, then. The first step in saving space, is to know what there is and what you have. Start with a "df -k" command to see what is available on the system. You'll get a listing of information on every filesystem on Bama, but here are the 5 most interesting. Filesystem kbytes used avail capacity Mounted on /dev/dsk/c3t5d0s0 24953411 1819576 20638495 9% /st1 /dev/dsk/c3t5d1s0 24955459 1052183 21407736 5% /st2 /dev/dsk/c3t5d2s0 26117763 108017 24515239 5% /st3 /dev/dsk/c3t5d3s0 26314216 12366081 13421851 48% /fs swap 1648088 22488 1625600 2% /tmp The name of the filesystem is in the rightmost column. Here, "/st1", "/st2", "/st3" are the student directories, "/fs" is for faculty and staff, and "/tmp" is an in-memory filesystem. The columns labeled "kbytes", "used", and "avail", are the total amount of space, the amount used, and the amount available, respectively. The percentage of space used is given under "capacity." These are given in kilobytes, so you should picture 24953411 as 24, 953, 411 kbytes. That's almost 25 Gbytes! This listing is the actual one for Bama in early November, 1998. You can see that the faculty/staff directory area is almost half full. So, even though faculty/staff do not have quotas, they can still run out of disk space. So now that we know what disk usage on the entire system looks like, let's look at individual usage. If you want to save disk space, it is best to find out where you are taking up the most and work from there on down to smaller files. You can see your entire disk usage by typing "du -k \| page". This will list your disk usage in kilobytes, directory by directory. With this, you'll get a good idea as to which directories are taking up the most space. Once you know what directory to focus on, you can do an "ls -l" in that directory. In the resulting listing, the number in the 5 th column is the size of the file. Again, a good strategy is to start looking at the big files and move on down to the smaller files. If you have large, unnecessary files, delete them. What can you do if you have a large file that you want to keep but don't need to use all the time? Here are a few strategies. We'll look at more in a later Tipsheet. Turn Spaces into Tabs Files that have lots of spaces might still be readable with the spaces turned into tabs. There is a Unix command, called "unexpand", that will take spaces and convert them to tabs. The default is to take any eight spaces in a row to make one tab. Here is how you use "unexpand": `unexpand -a myfile > newfile` where `myfile` is the file with lots of spaces and `newfile` is the file with tabs substituted in. Many terminal programs, like TeraTerm, will interpret the tabs correctly, and your file will look the same when you view it. How do you change tabs back to spaces? With "expand", of course: `expand -a newfile > myfile` Compress Your Files We have two different compression algorithms available on Bama; compress/uncompress and gzip/gunzip. Gzip seems to make smaller compressed files so it has become a popular option. Any compressed file will be a binary file, so you won't be able to read it or edit it directly. To compress a file with gzip, type `gzip filename` where `filename` is the original file. The compressed file will automatically receive the suffix ".gz". To get the file back to its original form, type `gunzip filename` You don't need to put on the ".gz" suffix. If you have an entire directory tree that you wish to compress the best way to proceed is to create a "tar file". Tar originally stood for "tape archive" but these can be written to disk, too. Since small files can take up a minimum of 1 block of space, even if they are smaller than that, lots of small files can take up lots of space. When they are "tarred" together, they are joined into one file, taking up only as much space as needed. Once a directory tree is put into a "tar" file, that file can be compress, thus taking up even less space. Here's a one-step command to tar and compress a directory tree: `tar -cvf - project \| gzip > project.tar.gz` What you have done is run the tar command with the "create" option, "c", verbosely, "v", so that you can see what is happening, and the file output option, "f". The file you are writing to is STDOUT (see Tipsheet Vol. 1, No. 4), indicated by "-" and you are tarring the directory called `project`. This is piped, "\|", to gzip and the output is sent, ">", to "`project`.tar.gz". In this case you must give gzip a name to use since it doesn't get a name through the pipe. Now you can remove the files and the directory "project", since it is safe in the archive. This gzipped, archive file can be transferred to other computers with FTP, Just be sure to transfer it as a binary file. To get the directory tree back you would extract it like this: `gunzip -c project.tar \| tar -xvf -` The "-c" option on gunzip let's you send the uncompressed file on to the tar command without having it create the uncompress file on disk. Use /tmp for temporary space There is a /tmp filesystem which anyone can use. It is a virtual filesystem. It exists only in computer memory, so that when the computer is rebooted, all files in it are lost. But it can serve to ease you space crunch while you clean up your directory. For instance, you can put your compressed file or tar archive over there while it is being created, then copy it back to your directory when you have space. If you are using Netscape on Bama it is not recommended that you set your cache to /tmp. To save space you could set the disk cache size to be zero (0). Go to the "preferences" menu under "edit" and look at the "advanced" section. In this way, you will only cache pages in memory, making them available for that session, but they will disappear when you log out. © 1998, The University of Alabama. The information included here is for the University of Alabama central computing facility as it was configured on the document date. It may or may not apply to other Unix systems.
Copyright © 1998-2004 The University of Alabama \| Disclaimer \| Contact: Help.Desk@ua.edu