The Linux Filesystem Hierarchy
One of the things that made me curious when I approached Linux for the first time was the long list of different directories that are part of a basic installation. The command ‘man’ helps in figuring this out when searching for ‘hier’, which stands for hierarchy:?
[carlo@dragon ~]$ man hier
Another good source of information for the file system hierarchy of Linux is available at The Linux Documentation Project web site:
http://tldp.org/LDP/Linux-Filesystem-Hierarchy/Linux-Filesystem-Hierarchy.pdf
This is a 113 pages book in PDF format and is a good reference for those who want to know more. For the rest of us, here is a list of the most important directories in the hierarchy.
Keep in mind that different Linux distributions may use some of the directories in a slightly different way and, sometimes, even change the path of some of the contents described in this article.
However, the principle remain, and you can always type ‘man hier’ on the command line to learn how your particular distribution behaves, if something is different.
/
The so called “root” directory. Everything starts from here; all the other directory paths start from this root point.
/bin
This is a container for the programs that are used in command line mode. Mostly, these are the programs that are used in day to day chores, from creating and destroying directories, to change ownerships, to make searches, and even provide shell environments. All these programs are executable binaries, from which the name of the directory: bin[ary].
/sbin
The container for the system programs that are used to boot the system and to perform certain administrator tasks. Regular users usually don’t need to access the contents of this directory, although they are usually visible to everybody.
/boot
The boot loader directory. It contains the boot loader itself, all its related files, and the kernel binary. When the system is powered on, the BIOS goes directly into this directory and starts the boot loader. This may present a list of Operating Systems to start, like Linux or Windows or OSX, then it loads the kernel into memory and gives control to it.
/dev
This directory contains a list of the physical devices that are part of the system. In Linux, every device is represented as a file, so each device has an entry in this directory that emulates a file. Whenever you access a device, either a disk, or a USB device, or a console terminal, or any other, the OS thinks it is a file that is located in this directory and access it, like it was a real file. Low level drivers take care of converting the file access directives to actual directives for the device itself. If you have some knowledge of C++, you can easily see the concept when you think how all kind of devices can be accessed by a stream, simply using the iostream class or one of its derivatives.
/etc
The repository for most of the system configuration files. Some very large packages have their configuration file in subdirectories of /etc. Otherwise, everything is located at this exact level. Example of configuration files stored in this directory are: bashrc, crontab, host.conf, inittab, netconfig, passwd, virc, yum.conf. Whenever aprogram is launched, it usually go into this directory to see if there is a centralized configuration file it should use.
/etc/opt
This is a special subdirectory of /etc. While /etc is used to store configuration files for system applications, or applications that come with the operating system, /etc/opt is used to store configuration files for optional applications, those that can be installed from expernal repositories, or those that you can eventually create of buid yourself. These are applications that are normally stored in the /opt directory.
/home
The default home directory for all users. Each user has usually its own subdirectory allocated right below /home. For example: /home/carlo
/root
The home directory of the root user. The root user or superuser is the main administrator of the system.
/lib
The libraries directory. This is the place where all the shared libraries are stored. Shared libraries are those binary routines that are used by the programs but are not part of the programs themselves. When a specific functionality, or a set of them, can be used for different programs, it is usually put in such a library. Then, the programs don’t have to incorporate that code over and over again. Instead, they just link to the library that contains the code so they can use it. As you can imagine, this architecture has the advantage that a bug fixed for a specific functionality, that is fixed within a library, automatically and simultaneously fix the same bug in all the programs that use that functionality through the library. It is very effective and largely used.
/media
Each storage device, before it can be used, needs to be mounted, otherwise the OS and the programs cannot access it. This directory is the place where removable media is mounted. CD-ROMs and USB disks, for example, are mounted in this directory and accessed from there.
/opt
As already mentioned earlier, this is the directory where optional applications are installed. These are applications that are not specifically part of the distribution.
/proc
This is a directory containing files that describe the current status of the system and all its processes. These files are continuously updated by the OS and we can only access them to read their content. Unfortunately, although the contents are written in text format, they are kind of criptic to understand. However, there are several programs and commands available that are made just for read and decode the contents of the proc files in a more human readable format. Commands like top and iostat, or applications like System Monitor, simply access the files in this directory and present their contents to you in a nice format.
/srv
This is a repository for certain data used by the system when it is used as a server. It usually comes empty with a new installation.
/tmp
A temporary container for everybody to use. Depending on the policy defined for the system, files in this directory can be deleted without notice, either because they are odler than a certain age, or at predetermined days and times. Certain programs use this directory to hold temporary data. Users can put files in it too, but they have to be aware that they are not going to stay there forever.
/usr
This is the directory containing programs for the regular users, their documentation, everything related to X, files used by compilers, libraries, and a lot of other interesting stuff. To stalk abou tthis directory alone, it would take a lot of space so, just go browse into it; you’ll be surprised by the amount of information in it. This directory was the original home directory for all the users in the UNIX systems, but its usage has been restricted more and more over time, until today is only usage for files that are of support for the users. The home for the users has moved to /home. However, some UNIX versions still use /usr the way it was originally intended.
/var
This directory is divided in several subdirecories. It is the repository for most of the log files generated in the system, along with some configuration files, cache, users’ mailboxes, spools for various programs, and a lot more.
Case study: my home network routine backup
As I promised last time, here is a description of how I deal with backups at home.
I am a very safe person when working with computers. Well not just for that… I am a safe person in general. Anyway, when it comes to safeguard the data stored in the home computers, I become even more cautious, up to the point of storing several backup copies of the same data, in case a copy is lost. All in all, I keep up to 4 copies of my data all around the house and one more up in the Internet cloud.
Here is how it works:
each computer in the network makes its own backup on a storage repository on the Linux server, which is used for that, to handle a local family WEB site, and to handle Internet access via a proxy server; this is what I call the first level of backup.
each month, I manually transfer the backup copies on the Linux server to a pool of removable drives; this is what I call the second level of backup.
each month, the previous content of the pool of removable drives is stored on a separate partition of the other server I have in the network, which acts as a storage and printer server; and this is, as you may guess, my third level of backup.
each month, the previous content of the storage server is moved on to DVD, my forth level of backup.
periodically, every time I deem it necessary, I finally store a copy of the most important data on the cloud, for safe keeping if something should go badly wrong in the house (I hope that never happens). This is usually data coming from the first level of backup. This data, are never removed form the cloud. Instead, every time I add something, I put the date in the name itself of the backup file, so that an historical archive is safely stored off site.
You can see how each level of backup stores a older version of data than the previous one. This way, files that are deleted from a computer can always be retrieved from an old backup, even though the most recent backups don’t have them anymore. You may call this one too an historical repository, if you are there for the big words.
And finally, here is a picture that delineates, hopefully better than words, what I just described.
To the next…
Backup your data!
Many people just assume that whatever they save in their computer is going to stay there forever, unless they delete it. Never a thought mas more wrong than that!
Computers are mostly made of solid state devices, which are supposed to last for a very long time without breaking. However, sometimes something goes wrong, for example a power surge, and a component on the motherboard goes away, or the whole motherboard fries up, or…
Did you ever think of the hard drive? Yes, that device that actually holds all your data. That is the most fragile piece inside a computer. Why? Because it is a mechanical device with moving parts! Moving parts are subject to friction, and they expand and contract when temperature changes. They get old, then they break. And usually they break way before other parts of the computer reach the end of their lifetime.
Then… what happens to the important things that you stored on the hard drive? All those pictures, and songs, and important documents, and…
Don’t wait ’till the worst happens. Take action immediately. Avoid blaming yourself once the disk dies and you haven’t make a copy of your data.
Take a USB drive, or a recordable DVD or a CD-ROM, and make a copy of all your data on it. Use several of them, if one is not enough. Even use an external removable disk. Then, once you have the copy of your data, store them in a safe place, ready for that day when your disk will break, and you’ll have to recreate your data on a new device.
Pretty easy, isn’t it? Just a little time spent once to make you sleep with no worries. Well, be careful! Every now and then you are surely going to change something in your data. Maybe you are going to add new stuff, like new songs, or new documents, or that video you were looking for since a long time.
Do another backup, save all those things that have changed, or you will end up with an old copy of your original data without the most recent updates.
Yes, I know, it is a never ending chore, but believe me: it is worth it.
Finally, if you want to play really safe, establish a routine:
Do periodic backups
Store a copy of your backup off site (what about if a fire or a flood strikes right where you keep both the computer and your only copy of backup?)
Think about the possibility of storing your data over the Internet, in the so named cloud. There are several on-line services that make available a certain amount of space to store your files in their servers. And if that space is not enough, for a small fee they can give you plenty of extra space and, one day, you’ll be happy you’ve done that.
Next time I will talk again about this topic. I will describe the routine I’ve established for my own computers, so to give you an example of how you could deal with your own stuff. Remember, nothing is perfect. What works for somebody may not be OK for another. In the end you have to decide how you want to protect your data, if at all. But a good example is a place to start, something you can think about to decide how you want to handle your stuff.
Sleep well.
Adding Linux to a Windows Laptop
I have this laptop running Windows XP® but, even though it is equipped with an Intel Core Duo® at 1.66 GHz with 4 Gb of RAM, sometimes it makes me think I’m using an ancient computer from 20 years ago. So, I wanted to try Linux on it to see if I could give the PC a new life. At the same time, I didn’t want to remove the existing OS, as the primary user of the laptop is my wife, which really doesn’t know how to live without the programs she is used to.
The solution: to install Linux alongside Windows, in what is called a dual boot configuration. Here is a brief account of how I did it, and some of the challenges I had to face.
If you don’t know that yet, I’m a a big fan of the Fedora distribution, so that is what I immediately decided to install. And since release 13 was just released, I took the opportunity to test drive it, so to decide if it was suited to also upgrade my main machine, which is used as a proxy and a WEB server for the family network.
Having decided what to put on the laptop, I went to the Fedora WEB site to download a copy of the distro. I choose to download the live CD version of it, so I could test it for compatibility before actually install it.
Here is a link to the download WEB page: http://fedoraproject.org/en/get-fedora
Once I had the .iso file on my desktop computer, I burned a CD with it and started working on the laptop itself.
First, I inserted the CD in the laptop and restart it, to make it boot from the CD. From the main screen, I selected the option to boot Fedora from the live CD. Once the boot process was completed, I found myself in the Gnome desktop from where I ran a number of simple compatibility tests, to make sure that Fedora was compatible with the laptop hardware:
check that the display was capable of showing the right resolution, and that the 3D acceleration was handled correctly;
check that the network interface was working fine and that it was possible to access the LAN and the Internet;
check that the sound was working properly;
check that the extra peripherals connected to the computer were working fine (i.e. printer).
Since everything seemed OK, I went to the next step: free up some space on the hard disk to install the new OS. To do that, I used gparted while still on Linux from the live CD.
Unfortunately, when I tried to shrink the existing NTFS Windows partition , gparted told me that it didn’t have enough space to do so. Very odd, since at least 70% of the disk was empty. But, in fact, gparted had it right: my disk was 70% empty, but it was highly fragmented, so there was not enough contiguous space to shrink the partition. I had to shutdown the computer, remove the CD, and reboot the laptop with Windows, then recall the defragmentation program and have it run 3 times, before the disk was finally enough defragmented that I was able to shrink the NTFS partition. I then left about 50% of the disk available to Windows and the remaining 50% unused.
Done that, I finally selected the icon on the Gnome desktop to start the actual installation, which conducted me through a number of choices:
Language.
Basic storage device, the hard disk.
Host name, the name the computer is known in the local network.
Time zone, and here I made sure to uncheck the box “system clock uses UTC “. Reason for that is that Fedora and Windows both try to update the system time on the motherboard, but Windows always does it in local time, so I had to make sure Fedora did the same to avoid incompatibilities when switching back and forth between Windows and Linux.
Root password, which is the password used by the system administrator.
Make the Linux installation use only the free space on the hard disk. Here I also required to be able to review the partitioning layout, so to be able to have the /home file system reside on a partition of its own, which is always better to do to simplify future upgrades.
Change label used by the boot loader to identify the “other” OS as Windows XP.
Set Windows XP as the default OS, to make things easier for my wife.
The installation procedure then required the system to be rebooted. After that, the procedure went through a series of steps to complete the configuration. It basically asked to do the following:
Create one or more user, as you don’t want to use “root” for regular activities. And for that I created two users, one for my wife (in the hope that she will use Linux from time to time and get used to it) and one for myself.
Set the time, which I required to be set automatically across the network.
Send the HW profile of the PC to the Fedora Project. This one step is not necessary, but it is a nice thing to do, so that developers know what kind of PCs are used out there, and that helps them to tune up the OS in future releases and make it work better and better.
And there was it: the installation was completed and the PC was ready to use. I just made some small final little adjustments to please my taste:
Installed Thunderbird , as my primary e-mail application.
Ran the update tool to install all the latest fixes.
Setup the system to access the Internet through my proxy server (squid )
Installed the RPM Fusion repositories, which make available lots of multimedia programs and codecs.
Installed the Samba services, so to be able to access the Windows shared disks in the home network.
In conclusion, all that took me a good half day of work, but it was worth, as the laptop, with Fedora on it, now runs much faster and smoother than when it runs Windows. One more evidence that Linux is a much lighter and stable OS than Windows.
Let’s Organize Our Data
Not so long ago my wife came to me begging to help her find a file that she knew she had on her laptop, but that she could no longer find. Armed with patience I then sat at her computer and started running searches with whatever she thought was part of the title of her file. Nothing! Then I started searching by some keyword she suggested about the contents of the lost file. Nothing again! Finally I asked her what tool she used to generate that file and, surprise surprise, it was the omnipresent MS® Word.
So, I launched the program and started looking in the list of the latest opened documents. Nothing even there. The history had been wiped out because she kept opening one file after another in search for the missing one and, of course, the history only provides the last 10 opened files.
I finally started asking my wife about details of when she created the file, if she remembered where she put it, what was the purpose of that file, whether she created more than one version of it. After all that questioning, we were able to narrow down the search to a few folders until, finally, we found it.
It was hard, but it was necessary as that file contained an important document she needed at work the very next day.
I just hope this event helped my wife learn an important lesson: computers hard drives are big, and you can put a lot of stuff in it. But the more you store in it, the more it becomes difficult to retrieve your things back, unless you start thinking like a librarian.
Oh my… what am I doing now? A preach on the goodness of keeping things well organized!!!
Well, yes, that’s what I’m doing. You see, with the big hard drive sizes we have available nowadays, we have the capacity to store as much information as a real public library, maybe even more. How do you think you are going to find your stuff if you don’t design a method for storing and retrieving information? Failing to do so will inevitably bring you to the chaos, to the moment when you won’t be able to easily find anymore what you are looking for.
You know, this is a well understood concept among those companies that want you to use the computer for everything you do every day. And they help you on this task by providing for free some tools that simplify the search for something missing. Think at the search capabilities of MS Windows®, or the Google® search that you can install on your own computer to easily find stuff on it.
Are these tools really the answer to all the problems? Maybe yes, maybe not. If you use your computer only occasionally and you name your files with very creative names all quite different from each other, then yes, these tools help you out. But if you start having different files with similar names and worst, different files with similar names and similar contents, how are you going to distinguish one from another in the long run?
Organization is the only key. And all the computers have the one tool to make you succeed in your quest for the data organization. This tool is called file system . Start with your Documents , or MyDocuments , or whatever you want to call it, folder. Don’t put all your files in this folders regardless of what they refer to. Create other folders within this main folder using meaningful names. Some people use to name these folders with the year when the documents were created. Other people names them into categories. Then, in each of this second level folders, create more folders, with more specific names. And so on and so on, until you reach the point where the file name itself becomes the most meaningful object at that level, so you finally start putting files in this lower level folders. It will be easier, later, to navigate the folders depending on the thing you are looking for, and you’ll find it in no time.
Think of it as family tree. Well, this method of organizing is actually called a tree. Why? Look at the graphical representation of it. Doesn’t it look like an upside down tree?
Organized tree
Can you feel how easy it would be to find things in it? Don’t let the programs put the saved files wherever they decide. Be in control! You decide where to put a saved file. Everything becomes so easy.