The Linux file system is most often ext4. It is logging and allows you to conveniently work with data when solving the vast majority of tasks. However, there are others. Main types file systems and the principles of working with them will be considered within the framework of this material.
Distinctive features are the speed of working with files, security and parameters (such as block size) that exist by default and are set when the FS is created. Perhaps the most important characteristic is the presence of a log. The system log records data or metadata(only headers) from which information can be recovered in case of failure.
A file system can be created on any device: disk or system partition.
EXT2 is a now obsolete file system that is hardly used in modern installations. the main drawback is the lack of logging, which, accordingly, makes it impossible to restore data in case of a failure. Still applies to portable media information such as USB. A magazine is not required for them, since it takes up a certain amount of space.
It also guarantees maximum speed.
Displaced EXT2, main feature is the appearance of the magazine, is fully backward compatible with EXT2 (EXT2 can be freely converted to EXT3). Now it is also rare, EXT4 is almost always used.
The log is a special area in memory that records information about all changes.
There are three options for journaling (specified when creating the file system):
The modern version of the extended file system, it is most often used
In EXT4 logging can be turned off by setting the option data when mounted in off
The file system is created by the mk2fs command
The desired logging option is specified when mounting, for example:
mount /dev/vdc /mnt/1 -t ext3 -o data=journal
Converting from EXT2 E to XT3
ReiserFS (and the modern implementation of Reiser4 with SELinux support) perform well and are very productive - especially when working with a large number of small files. ReiserFS does not allocate inodes for each small file by processing them together, also ReiserFS uses a journal with several options available. Currently, the file system is supported by developers from Russia.
You can create an FS for a device with the command
XFS is a journaled file system. Uses RAM to store information, so data loss is possible - for example, when the power is turned off.
To use XFS in Ubuntu you will need to install packages xfsprogs And xfsdump
The Linux file system also exists in the Windows environment. It is used when it is necessary to organize joint access to certain disks and partitions of clients with different operating systems. In other cases, it is not recommended to use it because it can be difficult when working in Linux.
VLADIMIR MESHKOV
ext2 file system architecture
The article discusses the logical structure of ext2 - the file system operating system linux.
Basic components of an ext2 filesystem
As in any UNIX file system, the following components can be distinguished in the ext2 file system:
Blocks and block groups
The entire disk partition space is divided into fixed-size blocks, multiples of the sector size - 1024, 2048 and 4096 bytes. The block size is specified when creating a file system on a partition hard drive. A smaller block size saves hard disk space, but also limits the maximum file system size. All blocks have serial numbers. In order to reduce fragmentation and the number of hard disk head movements when reading large data arrays, blocks are combined into groups.
Information node
The basic concept of a file system is an information node, information node, or inode. This is a special structure that contains information about the attributes and physical location of a file. File attributes are its type (regular file, directory, etc.), access rights to it, owner ID, size, creation time. The physical location information is a sequence of absolute block numbers containing file data.
Superblock
The superblock is the main element of the ext2 file system. It contains the following information about the file system (the list is not complete):
In other words, this is the number of the block containing the super block. This number is always 0 if the file system block size is greater than 1024 bytes, and 1 if the block size is 1024 bytes.
The health of the file system directly depends on the integrity of the superblock. The operating system creates several backups superblock so that it can be restored in case of damage. The master copy is located at offset 1024 bytes from the beginning of the partition on which the file system was created (the first 1024 bytes are reserved for the operating system loader).
Early versions of the ext2 filesystem created superblock copies at the beginning of each block group. This resulted in a large loss of disk space, so later the number of superblock backups was reduced, and block groups 0, 1, 3, 5 and 7 were allocated to accommodate them.
Block group format
A generalized block diagram of the ext2 file system is shown in fig. 1.
Almost all block groups have the same format. In each group, in addition to information blocks, information about the occupancy of blocks and inodes of the group is stored in the form of a bitmap. Block group 0 also includes a super block and a group descriptor table, which we will discuss below.
The block busy bitmap is usually located in the first block of the group. If there is a backup superblock in the group, the bitmap is located in the second block of the group. The bitmap size is one block. Each bit of this map indicates the state of the block. If the bit is set (1), then the block is busy; if it is reset (0), the block is free. The first block of the group corresponds to the zero bit of the map, the second block corresponds to the first bit, and so on.
Inodes that are within the same group are collected in a table. In a group's inode occupancy bitmap, each bit represents the state of an element in the group's inode table.
Each block group is described by a block group descriptor. A group descriptor is a structure that contains information about the addresses of the block busy bitmap, busy inode bitmap, and inode table of the corresponding group. All group descriptors are collected in a group descriptor table, which is stored in block group 0. As with the superblock, the operating system backs up the group descriptor table.
File Reading Algorithm
Each inode, like a block, has a sequence number that is unique within the file system and contains information about only one file. Thus, in order to access the contents of a file, it is necessary to know the ordinal number of the inode corresponding to it.
As mentioned above, information about the physical location of the file is contained in the inode. This information is a sequence of 32-bit block numbers containing the file data (Figure 1). The first 12 numbers are direct links to information blocks (direct blocks number). The 13th number is an indirect block number. It contains the address of the block, which stores the addresses of information blocks. The 14th number is a double indirect link (double blocks number), the 15th number is a triple indirect link (triple blocks number).
The file name is not part of the inode, the mapping between file names and inode sequence numbers is done through directories.
Catalogs
Files on UNIX and POSIX systems are stored in a tree-like hierarchical file system. The root of the file system is the root directory, denoted by the "/" symbol. Each intermediate node in the file system tree is a directory. The leaf nodes of the filesystem tree are either empty directories or files. The absolute pathname of a file consists of the names of all directories leading to the specified file, starting from the root directory. For example, the pathname /home/test.file means that the file test.file is located in the home directory, which in turn is located in the root "/" directory.
A directory, like a file, is described with an inode. The contents of a directory is an array of entries, each containing information about a file that is "inside" the current directory.
The directory entry has the following format:
Searching for the inode number of a file always starts at the root directory. For example, to get the inode number of a file located in the root directory, the operating system must get the contents of the root directory, find an entry in it with the name of this file, and extract the file's inode number from this entry.
The first few inodes are reserved by the file system and are listed in the header file:
*Special inode numbers
#define EXT2_BAD_INO 1 /* Bad blocks inode */
#define EXT2_ROOT_IN 2 /* Root inode */
#define EXT2_ACL_IDX_IN 3 /* ACL inode */
#define EXT2_ACL_DATA_INO 4 /* ACL inode */
#define EXT2_BOOT_LOADER_INO 5 /* Boot loader inode */
#define EXT2_UNDEL_DIR_INO 6 /* Undelete directory inode */
Inode number 2 (root inode) is reserved for the root directory entry. This inode is in block group 0 and is the second position in the group's inode table. The number of the first unreserved inode is stored in the super block.
Having determined the sequence number of the file's inode, the kernel calculates the number of the group in which this inode is located and its position in the group's inode table. By reading from this inode position, the operating system obtains complete information about the file, including the addresses of the blocks in which the contents of the file are stored.
The block group number in which the inode is located is calculated by the formula:
group = (inode_num - 1) / inodes_per_group
Where:
The position of the inode in the group's inode table is determined by the formula:
index = (inode_num - 1) % inodes_per_groupe
where index is the inode position in the table.
Consider an example of getting the contents of the test.file file located in the root directory. To read the /test.file file, you need to:
On fig. Figure 2 shows the steps for reading the /test.file in detail.
Steps 1-6 - reading the root directory:
By specifying the inode number, the information blocks of the file can be accessed (steps 7-11):
Software implementation of the file reading algorithm
Initial data: there is a hard disk partition on which the ext2 file system is created. This partition corresponds to the /dev/hda3 device file. The home subdirectory has been created in the root directory of the partition, and it contains the test.file file with the following content:
Would citrus live in the thickets of the south?
Yes, but a fake copy!
1234567890-=
Do not think bad, this is not nonsense, but a test exercise from the training course for telegraph operators in the signal troops of the former USSR!
Attention! One should take into account important point. The created file will not be immediately written to the disk, but will first go to the disk buffer. An attempt to immediately get the contents of the file using the above algorithm will not lead to anything, since there is no information about this file physically on the disk. You need to "force" the system to write the disk buffer to disk. The easiest way to do this is to perform a reboot operation. Therefore, after the file is created, reboot the system.
Our task is to use the /dev/hda3 device file to read the /home/test.file file by directly accessing its information blocks.
Consider the software implementation of the module that performs this operation.
Header files:
#include
#include
#include
#include
#include
#include
The header file defines structural types that describe the main components of the ext2 file system - superblock, block group descriptor, information node, directory entry.
Let's briefly consider the fields that are included in each of these structures:
The value of EXT2_N_BLOCKS is defined in the file:
* Constants relative to the data blocks
#define EXT2_NDIR_BLOCKS 12
#define EXT2_IND_BLOCK EXT2_NDIR_BLOCKS
#define EXT2_DIND_BLOCK (EXT2_IND_BLOCK + 1)
#define EXT2_TIND_BLOCK (EXT2_DIND_BLOCK + 1)
#define EXT2_N_BLOCKS(EXT2_TIND_BLOCK + 1)
#define EXT2_NAME_LEN 255
Let's determine the name of the partition on which the file system was created, global structures and variables.
#define PART_NAME "/dev/hda3"
struct ext2_super_block sb;
/* buffer to store group descriptor table */
unsigned char buff_grp;
unsigned char buff; /* information buffer */
int indev; /* device file descriptor */
int BLKSIZE; /* filesystem block size */
Let's define a few functions that we need to work:
Superblock read function:
void read_sb()
memset(&sb,0,1024);
We shift 1024 bytes from the beginning of the section and read the superblock into the struct ext2_super_block sb structure:
If(lseek(indev,1024,0)< 0) {
error("lseek");
Exit(-1);
If(read(indev,(char *)&sb,sizeof(sb))< 0) {
error("read");
Exit(-1);
Checking the file system ID:
If(sb.s_magic != EXT2_SUPER_MAGIC) (
Printf("Unknown file system type!");
Exit(-1);
The EXT2_SUPER_MAGIC value is defined in the header file.
We display information about the file system that is in the superblock:
printf(" Superblock info ----------- ");
Printf("Inodes count - %u ",sb.s_inodes_count);
Printf("Blocks count - %u ",sb.s_blocks_count);
Printf("Block size - %u ",1024<< sb.s_log_block_size);
Printf("First inode - %d ",sb.s_first_ino);
Printf("Magic - 0x%X ",sb.s_magic);
Printf("Inode size - %d ",sb.s_inode_size);
Printf("Inodes per group - %u ",sb.s_inodes_per_group);
Printf("Blosks per group - %u ",sb.s_blocks_per_group);
Printf("First data block - %u ",sb.s_first_data_block);
return;
Group descriptor table read function:
void read_gdt()
Calculate the block size of the file system:
BLKSIZE = 1024<< sb.s_log_block_size
The group descriptor table is located in the block immediately following the first data block (behind the superblock).
Reading the table:
If(lseek(indev, (sb.s_first_data_block + 1) * BLKSIZE, 0)< 0) {
error("lseek");
Exit(-1);
If(read(indev,buff_grp,BLKSIZE)< 0) {
error("read");
Exit(-1);
return;
The function to get the contents of an inode by its number:
void get_inode(int inode_num, struct ext2_inode *in)
The input parameters of the function are the inode number and the struct ext2_inode structure.
Struct ext2_group_desc gd;
U64 group, index, pos;
We calculate the number of the group of blocks in which the inode with the ordinal number inode_num is located:
Group = (inode_num - 1) / sb.s_inodes_per_group;
Extract the group descriptor from the group descriptor table and copy it to the struct ext2_group_desc gd structure:
Memset((void *)&gd, 0, sizeof(gd));
Memcpy((void *)&gd, buff_grp + (group * (sizeof(gd))), sizeof(gd));
We calculate the position of the inode with the ordinal number inode_num in the inode table of the group group and read this inode into the struct ext2_inode structure:
index = (inode_num - 1) % sb.s_inodes_per_group;
Pos = ((__u64)gd.bg_inode_table) * BLKSIZE + (index * sb.s_inode_size);
Preread64(indev, in, sb.s_inode_size, pos);
return;
Data Block Read Function:
void read_iblock(struct ext2_inode *in, int blk_num)
U64pos;
The input parameters of the function are the inode structure and the block number (meaning the number from the sequence of address blocks located in the inode).
We calculate the offset to the information block on the partition and read this block into the global buff buffer:
Pos = ((__u64)in->i_block) * BLKSIZE;
Preread64(indev, buff, BLKSIZE, pos);
return;
The function to get the contents of the root directory:
void get_root_dentry()
Struct ext2_inode in;
The root directory inode number is known, so we get the contents of the root directory inode and read its contents into the buff buffer:
get_inode(EXT2_ROOT_INO, &in);
Read_iblock(&in, 0);
The buff buffer will contain the contents of the root directory.
return;
Function to get inode number from file name:
int get_i_num(char *name)
The input parameters of the function are the file name. The return value is the inode number of the file.
int i = 0, rec_len = 0;
Struct ext2_dir_entry_2dent;
The buff buffer contains an array of directory entries. To determine the inode number of a file, you need to find an entry in this array with the name of this file:
For(;i< 700; i++) {
Memcpy((void *)&dent, (buff + rec_len), sizeof(dent));
If(!memcmp(dent.name, name, dent.name_len)) break;
Rec_len += dent.rec_len;
Return dent.inode;
Now let's write the main function:
int main()
Variables and structures:
struct ext2_inode in;
// absolute pathname of the file
Unsigned char *full_path = "/home/test.file";
unsigned char buff1;
Static int i = 1;
int n, i_num, outf, type;
The first character in an absolute pathname of a file must be a forward slash (/). Let's check it out:
If(full_path != "/") (
error("slash");
Exit(-1);
Open the device file, read the superblock and group descriptor table:
indev = open(PART_NAME,O_RDONLY);
if(indev< 0) {
error("open");
Exit(-1);
Read_sb();
Read_gdt();
Get the contents of the root directory:
get_root_dentry();
The buff buffer now contains all entries in the root directory (you can save them in a separate file if you like). Now that we have root directory entries, we can get to the contents of test.file using the file reading algorithm above. To this end, we organize a cycle. In the body of the loop, we will parse the absolute path name of the file, highlighting its elements - subdirectories (we have one, home) and the name of the desired file (test.file). For each element, we determine the ordinal number of the inode, read this inode and then get the contents of the zero block (from the sequence of address blocks located in the inode):
while(1) (
memset(buff1,0,sizeof(buff1));
For(n = 0 ; n< EXT2_NAME_LEN; n++, i++) {
Buff1[n] = full_path[i];
If((buff1[n] == "/") || (buff1[n] == "?")) (
I++;
break;
buff1[n] = "?";
For each element of the absolute pathname of the file, we determine the sequence number of the inode, read this inode into memory, and then get the contents of the zero block:
I_num = get_i_num(buff1);
Get_inode(i_num, &in);
Read_iblock(&in, 0);
Let's display information about the file (name, inode number, file size and type):
Printf("Inode number - %u ", i_num);
Printf("File name - %s ", buff1);
Printf("File size - %u ",in.i_size);
The file type is determined by the upper four bits of the i_mode field of the struct ext2_inode structure:
type = ((in.i_mode & 0xF000) >> 12);
Printf("Type - %d ",type);
switch(type) (
Case(0x04) :
Printf("(directory)");
break;
Case(0x08) :
Printf("(regular file)");
break;
Case(0x06) :
Printf("(block device file)");
break;
Case(0x02) :
Printf("(char device file) ");
break;
default:
Printf("(unknown type)");
break;
Checking the file type. If this is a regular file, we break the loop:
If(type & 0x08) (
The buff buffer will contain the information read from the information blocks of the /home/test.file file. Let's write this information to a file:
Outf = open("out",O_CREAT|O_RDWR,0600);
Write(outf, buff, sizeof(buff));
close(outf);
break;
Exit:
close(indev);
return 0;
This concludes our consideration of the logical structure of the ext2 file system.
We will now describe the most popular Linux disk file system, ext2. The first release of Linux used the MINIX 1 file system, which had short file names and a maximum file size of 64 MB. The MINIX 1 file system was eventually replaced by the first extended file system, ext, which allowed for longer file names and larger file sizes. Due to its low efficiency (in terms of performance), the ext system was replaced by its successor ext2, which is still widely used today.
The disk partition with ext2 contains the file system shown in fig. 10.17 layout. Block 0 is not used by the Linux system and contains the computer's boot code. Following block 0, the disk partition is divided into groups of blocks (ignoring disk cylinder boundaries). Each group is organized as follows.
The first block is the superblock, which stores information about the layout of the file system, including the number of i-nodes, the number of disk blocks, the beginning of the list of free disk blocks (usually several hundred elements). This is followed by a group descriptor containing information about the location of the bitmaps, the number of free blocks and i-nodes in the group, and the number of directories in the group. This information is important because the ext2 filesystem attempts to distribute directories evenly across the disk.
The two bitmaps keep track of free blocks and free i-nodes (this is also inherited from the MINIX 1 file system and distinguishes it from most UNIX file systems, which use a list for free blocks). The size of each bitmap is one block. With a block size of 1 KB, this scheme limits the block group size to 8192 blocks and 8192 i-nodes. The first number is a real limit, and the second is practically none. With 4 KB blocks, the numbers are four times larger.
Then the i-nodes themselves are located. They are numbered from 1 to some maximum. Each i-node is 128 bytes in size and describes exactly one file. The i-node contains accounting information (including everything returned by the stat call, which simply takes it from the i-node), as well as enough information to locate all disk blocks that contain file data.
The i-nodes are followed by data blocks. All files and directories are stored here. If a file or directory consists of more than one block, then those blocks need not be contiguous on disk. In reality, blocks of a large file are likely to be scattered all over the disk.
The i-nodes corresponding to the directories are scattered across all disk block groups. Ext2 tries to place regular files in the same block group as the parent directory, and data files in the same block as the i-node of the source file (assuming there is enough space there). This idea was borrowed from the Berkeley Fast file system. File System(McKusick et al., 1984). Bitmaps are used to receive quick fixes regarding the selection
places for new file system data.
When new file blocks are allocated, ext2 also preallocates a few (eight) additional blocks for the same file (to minimize file fragmentation due to future writes). This scheme spreads the file system across the disk. It also has good performance (due to its contiguous nature and reduced fragmentation).
To access a file, you must first use one of the Linux system calls (such as open), which requires you to specify the path to the file. This path is parsed and its constituent directories are extracted from it. If a relative path is specified, then the search starts from the current directory of the process, otherwise, from the root directory. In any case, the i-node for the first directory is easy to find: there is a pointer to it in the process descriptor, or (in the case of the root directory) it is stored in a specific block on disk.
The directory allows filenames up to 255 characters long (Figure 10.18). Each directory consists of a number of disk blocks (so that the directory can be written to disk atomically). In a directory, the elements for files and directories are in unsorted order (each element immediately follows the previous one). Elements cannot cross block boundaries, so there is usually some amount of unused bytes at the end of each disk block.
Each directory entry in Fig. 10.18 consists of four fixed-length fields and one variable-length field. The first field is the i-node number, which is 19 for the colossal file, 42 for the voluminous file, and 88 for the bigdir directory. Next comes the rec_len field, which gives the size of the entire directory entry in bytes (possibly with additional padding bytes after the name). This field is needed to find the next entry (in the case where the file name is padded with an unknown number of bytes). In the figure, this field is indicated by an arrow. Then there is a field of type file, directory, etc. The last fixed-length field contains the length of the file name in bytes (8, 10 and 6 for this example). Finally, there is the filename itself, terminated by a null byte and padded to a 32-bit boundary. It may be followed by additional padding bytes.
On fig. 10.18b shows the same directory after the element for voluminous has been removed. All that is done in the directory is to increase the number in the record size field. previous file colossal, and the directory entry bytes for remote file voluminous turn into placeholders for the first record. Subsequently, these bytes can be used for writing when creating a new file.
Since directories are searched linearly, it can take a long time to find an entry that is at the end of a large directory. Therefore, the system maintains a cache of recently accessed directories. The cache is searched by file name, and if it is found, then the expensive linear search is no longer needed. A dentry object is entered into the directory entry cache for each of the path components, and (via its i-node) the directory is searched for subsequent path entries (until the actual i-node of the file is found).
For example, to find a file specified by an absolute path (such as /usr/ast/file), the following steps must be followed. First of all, the system finds the root directory that i-node number 2 usually uses (especially when i-node number 1 is reserved for bad blocks). It places the appropriate entry into the directory entry cache (for future searches of the root directory). It then searches the root directory for the string "usr" to obtain the i-node number for the /usr directory (which is also entered into the directory's item cache). This i-node is then read and disk blocks are extracted from it, so that the /usr directory can be read and searched for the string "ast". Once a matching entry has been found, the i-node number for the /usr/ast directory can be determined from it. Given this i-node number, it can be read and directory blocks found. Finally, we search for "file" and find its i-node number. Thus, using a relative path is not only more convenient for the user, but also reduces the amount of work for the system.
If the file is present, the system retrieves the i-node number and uses it as an index of the i-node table (on disk) to look up the corresponding i-node and read it into memory. This i-node is placed in the i-node table, a kernel data structure that contains all i-nodes for open this moment files and directories. The i-node element format must contain (at a minimum) all the fields returned by the stat system call for the stat call to work (see Table 10.10). In table. Figure 10-13 shows some of the fields in the i-node structure supported by the Linux file system. The actual i-node structure contains many more fields, since the same structure is used to represent directories, devices, and other special files. The i-node structure also contains fields reserved for future use. History has shown that unused bits don't stay idle for long.
Now let's see how the system reads the file. Remember that a typical library procedure call to run the read system call looks like this:
n = read(fd, buffer, nbytes);
When the kernel takes control, all it can start with are these three parameters and the information in its internal tables (pertaining to the user). One of the elements of these internal tables is an array of file descriptors. It is indexed by file descriptors and contains one element per open file (up to some maximum number, usually 32 by default).
The idea is to start at this file descriptor and end with the corresponding bnode. Let's look at one possible scheme: put a pointer to a node in a table of file descriptors. Despite the simplicity this method(unfortunately) doesn't work. The problem is this. Each file descriptor must have an associated file pointer that specifies the byte in the file where the next read or write operation will begin. Where should this pointer be stored? One option is to place it in the node table. However, this approach will not work if several unrelated processes open the same file at the same time, because each process must have its own pointer.
The second solution is to place a pointer in the file descriptor table. Each process that opens a file has its own position in the file. Unfortunately, this scheme also does not work, but the reason for the failure is this case not so obvious and related to nature sharing files on a Linux system. Consider shell script 5, which consists of two commands (p1 and p2) to be executed in turn. If the script is called from the command line
then p1 is expected to write its output to file x, and then p2 will also write its output to file x, starting where p1 left off.
When the shell starts p1, file x will be empty at first, so p1 will simply start writing to the file at position 0. However, when p1 is done, some mechanism is needed to ensure that p2 sees the start position not 0 (but this is exactly what happens if the position in the file is stored in the file descriptor table), but the value at which pi stopped.
How this is done is shown in Fig. 10.19. The trick is to introduce a new table - the description table open files(open file description table) - between the file descriptor table and the i-node table and store the file pointer (as well as the read/write bit) in it. In the figure, the parent process is the shell, and the child is first the pi process and then the p2 process. When the shell creates the pi process, its user structure (including the file descriptor table) is an exact copy of the same shell structure, so they both contain pointers to the same open file description table. When the pi process exits, the shell's file descriptor continues to point to the open file description table, which contains the p1 process's position in the file. When the shell now creates the p2 process, the new child process automatically inherits the position in the file, without new process, nor is the shell required to know the current value of that position.
If some extraneous process opens the file, then it will get its own entry in the open file description table with its position in the file, which is exactly what is needed. Thus, the purpose of the open file description table is to allow parent and child processes to share a single pointer in a file, but to allocate personal pointers for foreign processes.
So (returning to the problem of performing a read), we have shown how the position in the file and the i-node are determined. The I-node contains the disk addresses of the first 12 blocks of the file. If the position in the file falls within its first 12 blocks, then it reads desired block file and data are copied to the user. For files longer than 12 blocks, the i-node contains the disk address of a single indirect block (Figure 10.19). This block contains the disk addresses of additional disk blocks. For example, if the block size is 1 KB and the disk address is 4 bytes, then a single indirect block can store up to 256 disk addresses. This scheme allows you to support files up to 268 KB in size.
ext2(also referred to as ext2fs) - Second Extended File System(Second Extended File System) is a file system built on the Linux kernel. The creator and developer of ext2 is Remy Card. The ext2 file system was built by him to replace the old, previous version - ext.
In terms of such indicators as speed and performance, this file system can serve as a benchmark. This is evidenced by the results of file system performance tests. For example, in the Dell Tech Center's sequential read/write speed tests, ext2 outperforms ext3, and only outperforms the more modern ext4 in read speed.
The main disadvantage of ext2 is that it is not a journaling filesystem. However, this shortcoming was eliminated in the next file system - ext3.
ext2 is used on flash cards and solid state drives (SSDs) because the lack of journaling is an advantage when working with drives with write cycle limits.
At the time of the rapid development of the Linux system, it used the Minix OS file system. It was quite stable, but it was 16-bit. As a result, it had a hard limit of 64 Mb per partition. In addition, there was a limit on the maximum length of a file name, which was 14 characters.
These limitations combined were the reason for the development of the "extended file system" (hence the term " Extended File System»). She was given the task of solving two of Minix's key problems. The new file system was made public in April 1992. It was Ext, which extended the file size limit to 2 gigabytes and set the file name limit to 255 characters.
However, despite the success of the new file system, there were still quite a few unsolved problems. For example, there was no support for separate access, there were no data modification timestamps. The need to solve these problems was the motive for creating the next version of the extended ext2 file system (“ Second Extended File System). ext2 was developed in January 1993 and also implemented POSIX-compliant ACLs and extended file attributes.
The ext2 directory hierarchy graph is represented as a network. This is due to the fact that one file can be included in several directories at once.
All file types have symbolic names. In hierarchically organized file systems, three types of names are generally used: simple, compound, and relative. Same with ext2. In the case of a simple name, the limitation is that its length should not exceed 255 characters, in addition, the name should not contain a NULL character and a slash.
As for the NULL character, the limitations are related to the representation of strings in the C language, in the case of the slash character, everything lies in the fact that it is used as a separator character between directories.
The fully qualified name is a chain of simple symbolic names of all directories through which the path passes from the root to given file. In ext2, a file can be in multiple directories, which means it can have multiple full names (one file, multiple full names). But anyway, the full name defines the file.
ext2 attributes:
File attributes are stored in special tables, not in directories, as is common in simple file systems. As a result, the catalog has a very simple structure, consisting of two parts: an inode number and a name.
Disk partition structure
As part of ext2, the following can be distinguished:
The entire disk partition space is divided into fixed-size blocks, the blocks being a multiple of the sector size (1024, 2048, 4096 or 8192 bytes). The block size is specified when creating a file system on a disk partition. All blocks are assigned serial numbers. To reduce fragmentation and the number of hard disk head movements when reading large amounts of data, blocks are combined into groups.
The basic concept of a file system is an inode (also called inode - information node). This is a special structure containing information about the attributes and physical location of a file. The index decryptors are combined into a table contained at the beginning of each group of blocks. The superblock is the main element of the ext2 file system. It contains general information about the file system. The superblock is located 1024 bytes from the start of the section. The integrity of the superblock determines the health of the file system. The OS creates several backup copies of the superblock - in case the partition is damaged. The next block after the superblock contains a global descriptor table - a description of block groups in the form of an array with general information about all block groups.
Block group
All blocks on an ext2 partition are divided into groups. A separate entry is created for each group in the global descriptor table. This entry stores the basic parameters, such as: the block number in bitmaps and tables, the number of free blocks in the group, the number of inodes containing directories.
Block bitmap is a system in which each bit informs whether the block corresponding to it is assigned to any file. If the bit is 1, then the block is busy. An inode bitmap performs a similar function: it shows which inodes are occupied and which are not. Linux kernel tries to evenly distribute directory inodes among groups, and moves file inodes to the group with the parent directory. All remaining space, which appears in the table as data, is allocated for storing files.
Data Addressing System
The data addressing system is one of the most serious and key components of the file system. Thanks to it, the desired file is located among the many empty or busy blocks on the disk.
ext2 uses the following file block addressing scheme. To store the file address, 15 fields are allocated, each of which consists of 4 bytes. If the file fits in 12 blocks, then the numbers of the corresponding clusters are listed in the first twelve fields of the address. If the file size exceeds 12 blocks, then the next field contains the address of the cluster in which the numbers of the next blocks of the file can be located. Thus, the thirteenth field is used for indirect addressing.
With a maximum block size of 4096 bytes, the cluster corresponding to the 13th field can contain up to 1024 next block numbers in the file. If the file size exceeds 12+1024 blocks, then the 14th field is used, which contains the address of a cluster containing 1024 cluster numbers, each of which refers to 1024 blocks of the file. Here double indirect addressing is used. And if the file includes more than 12 + 1024 + 1048576 blocks, then the last 15th field is used for triple indirect addressing.
Such an addressing system allows, with a maximum block size of 4096 bytes, to have files larger than 2 TB.
File system(English file system) - an order that determines the method of organizing, storing and naming data on information carriers of IT equipment (using portable flash memory cards in portable electronic devices: digital cameras, mobile phones etc. e) and computer technology. It defines the format of the content and physical storage of information that is usually grouped in the form of files. A specific file system determines the size of a file (folder) name, the maximum possible file and partition size, and a set of file attributes. Some file systems provide service capabilities such as access control or file encryption.
File system tasks
The main functions of any file system are aimed at solving the following tasks:
file naming;
the program interface of work with files for applications;
mapping the logical model of the file system to the physical organization of the data warehouse;
organizing the stability of the file system to power failures, hardware and software errors;
In multi-user systems, another task appears: protecting the files of one user from unauthorized access by another user, as well as ensuring joint work with files, for example, when a file is opened by one of the users, for others the same file will be temporarily available in read-only mode.
The file system is the main structure used by a computer to organize information on a hard drive. When installing a new hard drive, it must be partitioned and formatted for a specific file system, after which it can store data and programs. Windows has three possible options file system: NTFS, FAT32 and the rarely used legacy FAT system (also known as FAT16).
NTFS is the preferred file system for this version of Windows. It has many advantages over the earlier FAT32 system; some of them are listed below.
The ability to automatically recover from some disk errors (FAT32 does not have this ability).
Improved support for large hard drives.
Higher degree of security. You can use permissions and encryption to prevent user access to certain files.
The FAT32 file system and the rarely used FAT system have been used in previous Windows versions, including Windows 95, Windows 98, and Windows Millennium Edition. The FAT32 file system does not provide the level of security that NTFS provides, so if a computer has a FAT32-formatted partition or volume, the files on that partition are visible to anyone with access to the computer. The FAT32 file system also has file size limits. In this version of Windows, it is not possible to create a FAT32 partition larger than 32GB. Also, a FAT32 partition cannot contain a file larger than 4GB.
The main reason for using the FAT32 system would be that the computer will be able to run both Windows 95, Windows 98, or Windows Millennium Edition, as well as this version of Windows (multi-OS configuration). To create such a configuration, you must install a previous version of the operating system on a partition formatted with FAT32 or FAT, making it the primary one (the primary partition can contain the operating system). Other sections accessed from previous versions Windows must also be formatted as FAT32. More early versions Windows can only access NTFS network partitions or volumes. NTFS partitions on the local computer will not be available.
FAT - pluses:
Requires little to be effective random access memory.
Fast work with small and medium directories.
The disk makes fewer head movements on average (compared to NTFS).
Efficient work on slow disks.
FAT - cons:
Catastrophic loss of performance with increasing fragmentation, especially for large drives (FAT32 only).
Difficulties with random access to large (say, 10% or more of the disk size) files.
Very slow work with directories containing a large number of files.
NTFS pros:
File fragmentation has little to no effect on the file system itself - the performance of a fragmented system is degraded only in terms of access to the file data itself.
The complexity of the directory structure and the number of files in one directory also does not pose any particular obstacles to performance.
Quick access to an arbitrary fragment of a file (for example, editing large .wav files).
Very fast access to small files (a few hundred bytes) - the entire file is in the same place as the system data (MFT record).
NTFS cons:
Significant system memory requirements (64 MB is the absolute minimum, more is better).
Slow disks and controllers without Bus Mastering greatly slow down NTFS performance.
Working with directories of medium size is difficult because they are almost always fragmented.
A disk that has been running for a long time in a state of 80% - 90% full will show extremely low performance.
The following file systems are considered as "native" for Linux (that is, those on which it can be installed and from which it is able to start): ext2fs, ext3fs, ext4fs, ReiserFS, XFS, JFS. It is they who are usually offered the choice when installing the vast majority of distributions. Of course, there are ways Linux installations to FAT/VFAT/FAT32 file systems, but this is only for those honeys and monsieurs who understand a lot about perversions, and I won't talk about them.
The main criteria for choosing a file system are usually reliability and speed. In some cases, you also have to take into account the compatibility factor - in this case, it means the ability of other operating systems to access a particular file system.
I'll start with ReiserFS - because the reason for writing this note was the question: what should be considered small files? After all, it is well known that it is the efficiency of working with small files that is the strength of this file system.
So, small files are files smaller than the logical block of the file system, which in Linux in most cases is equal to four kilobytes, although it can be set during formatting within certain limits (depending on the specific FS). There are countless such small files in any Unix-like OS. A typical example is the files that make up the tree of FreeBSD ports, Gentoo portages, and similar ported systems.
In most file systems, such mini files have both their own inode (an information node containing meta information about the file) and a data block, which leads to both disk space consumption and slower file operations. In particular, this is the reason for the catastrophic thoughtfulness of the FreeBSD file system (both the old, UFS, and the new, UFS2) when working with its own ports system.
In the ReiserFS file system, in such cases, separate blocks for data are not allocated - it manages to push the file data directly into the area of \u200b\u200bits own inode. Due to this and disk space is saved, and performance increases - literally several times compared to all other FS.
This handling of small ReiserFS files has given rise to a legend about its unreliability. Indeed, when the file system crashes (that is, the destruction of service areas), the data placed together with its inodes disappears along with them - and irretrievably. Whereas in those file systems where inodes and data blocks are always spatially separated, the latter can theoretically be restored. So, for ext2/ext3 there are even tools that allow you to do this.
However, like any legend, this one only gives the impression of authenticity. First, permanent data loss only applies to very small files. Among the user ones, there are practically no such ones, and all the others are easily restored from the distribution kit.
Secondly, speaking about the possibility of recovering data from blocks that have lost their binding to their inodes, I did not accidentally use the word “theoretical”. Because in practice this occupation is extremely time-consuming, not giving a guaranteed result. Everyone who has had to do this will agree that one can indulge in it only out of complete hopelessness. And this applies to all Linux file systems. So this aspect can be neglected when choosing a file system.
In terms of overall performance, ReiserFS is definitely faster than all other journaled FS, and in some respects even surpasses ext2. The results of comparing the speed of some common file file operations can be found here.
But with the compatibility of ReiserFS, the situation is somewhat worse. Access to it from the OS Windows families, as far as I know, is not possible. Some operating systems of the BSD family (DragonFlyBSD, FreeBSD) support this file system, but in read-only mode. Even the probability that an arbitrary Linux LiveCD from yesteryear does not have ReiserFS support is not zero.
And here it's time to remember about ext3fs. Its advantage is not at all in greater reliability - this is the same legend as the instability of ReiserFS. I've heard as much about ext3fs crashes as I've heard about ReiserFS crashes. I myself could not destroy either one or the other. Unless it worked with ext2 - but even that was a very long time ago, during the time of kernel 2.2 (or even 2.0).
No, the main advantage of ext3fs is its compatibility - it is guaranteed to be read by any Linux system. For example, when restoring from some ancient LiveCD assistant - a situation that is almost not so unbelievable, I had to get into it. Again, most BSD systems understand ext3fs easily (albeit without journaling). For Windows, there are also, as far as I know, all kinds of drivers and plug-ins for common file managers(type Total Commander) that provide access to ext2fs/ext3fs partitions.
In terms of performance, ext3fs leaves a mixed impression. Firstly, its performance is very dependent on the logging mode, of which there are three: with full data logging, partial data logging, and metadata-only logging. In each of the modes, it shows different performance on different types of file operations. However, in any case, the performance is not a record.
However, if the requirement for speed is put in the first place, then ext2fs is out of competition here - however, in this case you will have to put up with the lack of logging at all. And, consequently, with lengthy file system checks in case of any incorrect shutdown - and with the volume of modern disks, this can drag on for a long time ...
Concerning XFS it is possible to tell the following. In terms of compatibility, everything that is written for ReiserFS applies to it - moreover, until some time it was not supported by the standard Linux kernel. From the point of view of performance, it XFS also does not shine, speaking in total about the same level as ext3fs. And on the operation of deleting files, it generally demonstrates depressing slowness.
In my experience, using XFS pays off when working with not just large, but very large files - which are actually only DVD images and video files.
I return to the issue of reliability. A banal power off during normal user work, as a rule, painlessly transfers all journaled file systems (and none of them ensures the safety of user operations not written to disk - saving the drowning people and here remains the work of the drowning themselves). True, for any file system it is possible to simulate a situation in which turning off the power will lead to more or less serious damage to it. However, in real life, the occurrence of such situations is unlikely. And you can completely eliminate them by acquiring a source uninterruptible power supply- it will give more confidence in the safety of data than the type of file system. Well, in any case, only their regular backup can be a guarantee of restoring destroyed data ...
I think the above information is enough for an informed choice. My personal choice over the past few years has been ReiserFS. Occasionally, on systems where it is justified to move everything that is possible outside the root partition, it is advisable to use ext3fs for the root file system and ReiserFS for everything else.
If a separate partition is provided for the /boot directory (and this is recommended when using GRUB bootloader its developers) - for it, no other file system, except for ext2fs, is justified, any kind of journaling does not make sense here. Finally, if a separate partition is created for all kinds of multimedia materials, then you can think about XFS.
If we approach the explanation more methodically
ext - on initial stage Linux development was dominated by the ext2 system (extended file system, version 2). Since 2002, it has been replaced by the ext3 system, which is largely compatible with ext2, but also supports journaling functions, and, when working with kernel version 2.6 and higher, also ACL. The maximum file size is 2 TB, the maximum file system size is 8 TB. At the end of 2008, a version of ext4 was officially announced, which is backwards compatible with ext3, but many features are implemented more effectively than before. In addition, the maximum file system size is 1 EB (1,048,576 TB), and you can count on that for some time to be enough. About reiser - The system was named after its founder, Hans Reiser, and was the first logging system to access the Linux kernel for data. The SUSE version of zp was even considered the standard for some time. The main advantages of reiser over ext3 are faster speed and better placement when working with small files (and most files in a file system tend to be small). Over time, however, the development of reisefers stopped. Version 4 was announced a long time ago, which is still not ready, and support for version 3 has ended. About xfs - The xfs file system was originally developed for SGI workstations running on the IRIX operating system. Xfs is especially good for working with large files, in particular it is ideal for working with streaming video. The system supports quotas and extended attributes (ACLs).
jfs
jfs - a66peBHaTypaJFS stands for "Journaled File System". It was originally developed for IBM and then adapted for Linux. Jfs has never really enjoyed much recognition in Linux and currently ekes out a miserable existence, inferior to other file systems.
brtfs
brtfs - if it is the will of the leading kernel developers, the brtfs filesystem in Linux has a bright future. This system was developed at Oracle from the ground up. It includes device-mapper and RAID support. Brtfs is most similar to Sun's ZFS. Its most interesting features include checking the file system on the go, as well as support for SSD (Solid State Drives are hard drives based on flash memory). Unfortunately, work on brtfs will not be completed in the foreseeable future. Since version 11 of Fedora, it is possible to install brtfs, but I recommend using it only for file system developers!
There is no "fastest" or "best" file system - the estimate depends on what you intend to use the system for. Beginners Linux users working with local computer, it is recommended to work with ext3, and for server administrators - with ext4. Of course, ext4 is faster than ext3, but ext4 has significantly worse data reliability - you can easily lose information if the system suddenly shuts down.
If you installed a second UNIX-like operating system on your computer, then the following file systems will come in handy when exchanging data (from one OS to another).
sysv - used by SCO, Xenix and Coherent OS.
ufs - Used by FreeBSD, NetBSD, NextStep and SunOS. Linux can only read information from such file systems, but cannot make changes to the data. To access BSD segments, you will additionally need the BSD disklabel extension. A similar extension exists for SunOS partition tables.
ZFS is relative new system, developed by Sun for Solaris. Since the ZFS code does not comply with the GPL license, it cannot be integrated with the Linux kernel. For this reason, Linux only supports this file system indirectly through FUSE.
Windows, Mac OS X
The following file systems will be useful when exchanging information with MS DOS, Windows, OS/2, and Macintosh.
vfat - used in Windows 9x/ME. Linux can read information from such partitions and make changes to it. The vfat system drivers allow you to work with old MS DOS file systems (8 + 3 characters).
ntfs - the system is used in all modern versions of Windows: otNT and higher. Linux can read and modify its files.
hfs and hfsplus are the file systems used by Apple computers. Linux can read and modify its files.
Data CDs and DVDs usually use their own file systems.
iso9660 - The file system for CD-ROMs is described in the ISO-9660 standard, which allows only short filenames. Long names are supported in different operating systems in different ways, with the help of a variety of extensions that are incompatible with each other. Linux system capable of working with both the Rockridge extension, common in UNIX, and the Joliet extension, developed by Microsoft.
udf - this format (universal disk format) appeared and developed as the heir to ISO 9660.
Network file systems
Filesystems do not have to be on local disk- They
can connect to a computer and through the network. The Linux kernel supports various network file systems, of which the following are the most commonly used.
smbfs/cifs - help to connect network Windows directories or Samba to a directory tree.
nfs is the most important network file system in UNIX.
coda - this system is very similar to NFS. It has many additional features but it is not very common.
ncpfs - runs on the NetWare kernel protocol;oH used by Novell Netware.
Virtual file systems
There are several file systems in Linux that are not designed to store data on a hard disk (or other media), but only to exchange information between the kernel and user programs.
devpts - This filesystem provides access to pseudo-terminals (PTY for short) via /dev/pts/* in accordance with the UNIX-98 specification. (Pseudo terminals emulate a serial interface. On UNIX/Linux systems, such interfaces are used by terminal emulators such as xterm. Devices such as /dev/ttypn are typically used. In contrast, the UNIX-98 specification defines new devices. More details information is reported in the text terminal H0WT0.)
proc and sysfs - the proc file system is used to display service information related to the management of the kernel and processes. In addition to this, the sysfs filesystem builds relationships between the kernel and the hardware. Both filesystems are mounted at positions /proc and /sys.
tmpfs - This system is based on shared memory according to System V. It is usually connected at position /dev/shm and provides an efficient exchange of information between two programs. On some distributions (eg Ubuntu), the /var/run and /var/lock directories are also created using the tmpfs file system. Files in these directories are used by some network daemons to store process identification numbers and file access information. Thanks to tmpfs, this data is now reflected in RAM. The method guarantees high speed, as well as the fact that after shutting down the computer, no files will remain in the /var/run or /var/lock directories.
usbfs - the usbfs file system, starting from kernel version 2.6 and higher, gives information about connected USB devices. It is usually integrated into the proc file system. About USB device support in Linux.
Other file systems
auto - in fact, there is no file system with that name. However, the word auto can be used in /etc/fstab or with the mount command to specify a filesystem. In this case, Linux will try to recognize the file system on its own. This method works with most major file systems.
autofs, autofs4
autofs, autofs4 are also not file systems, but kernel extensions that automatically execute the mount command for the selected file systems. If a file system has not been used for some time, then the umount command is automatically executed for it. This method is useful primarily in cases where only a few of the many NFS directories are actively used at the same time.
To perform these operations, the /etc/init.d/ autofs script automatically executes the automount program at system startup. It is configured using the /etc/auto.master file. Appropriate programs are automatically installed in, for example, Red Hat and Fedora. In any case, autofs is only enabled after configuring /etc/auto.master or /etc/auto.misc.
cramfs and squashfs
cramfs and squashfs - Cram and Squash filesystems are read-only. They are used to "pack" as many zipped files into flash memory or ROM (Read Only Memory) as possible.
fuse - FUSE stands for "Filesystem in Userspace" and allows the development and use of filesystem drivers outside the kernel. Therefore, FUSE is always used with an external file system driver. FUSE works with the ntfs-3g NTFS driver in particular.
gfs and ocfs - Oracle's Global File System and Oracle Cluster File System allow you to build giant network file systems that can be accessed in parallel by many computers at the same time.
jffs and yaffs - Journaling Flash File System and Yet Another Flash File System are specially optimized for working with solid state drives and flash media. Using special algorithms, they try to evenly use all memory cells (wear leveling technology) in order to avoid premature system failure.
loop
loop - used to work with pseudo devices. A loopback device is an adapter capable of accessing a regular file as if it were a block device. Thanks to him, you can place any file system in any file, and then connect it to the directory tree using mount. The kernel function responsible for this - pseudo-device support - is implemented in the loop module.
There are various ways to use pseudo-devices. In particular, they can be used when creating disks in RAM for initial initialization (Initial RAM disk) for GRUB or LILO, when implementing encrypted file systems or testing ISO images for CD.
Storage media file systems
File systems
ISO 9660
Joliet extension of the ISO 9660 file system.
Rock Ridge (RRIP, IEEE P1282) is an ISO 9660 file system extension designed to store file attributes used in POSIX operating systems
Amiga Rock Ridge Extensions
El Torito
Apple ISO9660 Extensions
HFS, HFS+
Universal Disk Format A specification for an operating system-independent file system format for storing files on optical media. UDF is an implementation of the ISO/IEC 13346 standard
Mount Rainier
