I would like to devote an article to an overview of the APIs provided by various operating systems for tracking changes in a directory. The article appeared as a result of my work on change tracking daemons for the dklab_realsync utility (, github repository) and my own, which I do not want to announce yet.

Windows ReadDirectoryChangesW

For operating system Windows has a wonderful ReadDirectoryChangesW function that returns a set of changes for a directory, including a flag to work recursively (bWatchSubtree). Thus, the implementation of tracking changes in the directory is not difficult, and in the same dklab_realsync implementation takes 80 lines of code or 3.5 Kb. Interestingly, on Windows these events are supported even via SMB!

However, there are certain pitfalls:

the final size of the change buffer, after which the event queue will overflow and these events will be lost
according to the watchdog package documentation, the move event is sent before the changes become visible in the filesystem
buffer size limited to 64 KB for network FS

Conclusion: The ReadDirectoryChangesW function allows you to easily find out about all events in files, but the event queue may overflow and then you will need to perform a full file system scan. It is also possible to deliver events before they become relevant.

Mac OS X, FSEvents

Mac OS X also has a handy and simple API for monitoring file system changes called FSEvents . Using this API, the simplest daemon implementation is 50 lines of code or 1.8 kb. The queue cannot overflow (!), but a full scan may still be required if the fseventsd daemon crashes. It's worth noting that this API before version 10.7 does not provide changes by files, it only reports directories where something has changed. Since the events do not go anywhere and are written to the log (FSEvents service stores events in a persistent, per-volume database), detailing with directory precision saves disk space.

Conclusion: The FSEvents API for Mac OS X is the most unusual of all such APIs. The queue does not overflow and it is even possible to get events from the past. However, event granularity is given on a per-directory basis (up to version 10.7), which means the daemon is less efficient for synchronizing files.

Linux, inotify

In the linux vanilla kernel, there is one way to watch for changes in a directory, and that is inotify . There is good and detailed documentation for this API, but there is no support for recursive change tracking! Also, inotify has a limit on the maximum number of objects that can be monitored. The simplest daemon implementation takes 250 lines of code or 8 KB. A static build using dietlibc takes about 14 kb. Another unpleasant moment is that the application itself must maintain correspondences between the watch descriptor (in our case, this is always a directory) and the name. There is a function inotify_add_watch , which is passed the path to the monitored directory, but there is no reverse - inotify_get_path, which would return this very path according to the passed descriptor. Events contain only a watch descriptor and a relative path to the changed file inside the directory.

Pitfalls of recursively following a directory with inotify:

Possibility of overflowing the queue (the length of the queue is set in /proc/sys/fs/inotify/max_queued_events)
Limit on the maximum number of watch objects (set in /proc/sys/fs/inotify/max_user_watches)
Not being able to recursively follow a directory
The need to separately handle the case when a directory is created (eg mkdir -p a/b/c). You will receive an event that the directory “a” has been created, but while you are attaching a handler to this directory, another directory can already be created in it and you will not receive an event about this.
The theoretical possibility of integer overflow of watch descriptor (wd), since it is set by uint32

FreeBSD, Mac OS X, kqueue

FreeBSD and Mac OS X allow you to track changes using kqueue, which is similar in characteristics to inotify and also lacks recursive directory tracking. Also, kqueue takes descriptors as arguments. open files(directories), so when using this API, the limits on the number of monitored directories are even more strict.

Total:

As you can see, all APIs have their advantages and disadvantages. The kqueue and inotify mechanisms are the least convenient, but they are also the most efficient and reliable. Commercial operating systems provide more convenient mechanisms for tracking changes, but they also have their own characteristics. I hope now you have a better idea of how hard the fate of Dropbox and similar programs that need to get along with all this and perform reliable and efficient data synchronization :).

* Picture taken from

A small program that does not require installation, designed to monitor files in the folder you specified. It allows you to monitor both local and network folder and display alerts both on the Desktop and via the network or by E-mail! The program also has the ability to quickly respond to changes by running bat-files or certain applications.

It just so happened that today everyone, one way or another, depends on the computer. I'm not talking about computer addiction as a disease, no :). It's just that each user stores his personal files on the PC anyway ...

And since several people can work on one computer, then, naturally, you may want to make sure that no one and no one has anything to do with your data.

The most radical way is to encrypt your personal folder with a password. However, this is not always convenient, for example, if you need to access it over the network ...

In this case, you can seek help special programs to monitor file changes in the directories you need. They allow you to constantly monitor the selected folders and, if their contents change, notify the user about them. One of the few such free programs is simple observer.

Comparison with a paid analogue

A simple observer, despite its uncomplicated name, is a fairly powerful monitoring tool. file system, which allows you to monitor both a separate folder and the entire disk or even a shared network directory. You can compare it with another paid domestic development- Folder Watchdog Service:

The only downside to Simple Watcher is the inability to monitor multiple folders at once. Everything else is just a plus :).

Preparing to work with the program

Another plus of the application is that it does not require installation! To start working with the program, you only need to extract it from the archive and place it in any convenient folder(albeit in the one that should be monitored). Everything - we are ready to work :).

Oh yes! I almost forgot to warn you that libraries must be installed on your PC .NET Framework 3.5 (usually they are installed with the system, but they may not be available, so just in case, here is a download link: https://www.microsoft.com/en-us/download/details.aspx?id=21).

After running the program, we will see the following:

Don't be scared :). No one requires money from us - registration is completely free. And if you do not want to reveal your incognito, then you can not register at all! To do this, simply click the "Do not register" button.

However, in the future you will have to do this every time you start the program, so it will be easier to enter the necessary data in the appropriate fields and click "Register".

After that, the main window of the program will appear in front of us:

It consists of several function buttons, a link to the monitored folder, and a status bar that displays information about the application's activity.

Program settings

Before you start watching a folder, I advise you to familiarize yourself with the settings of the Simple Observer. To get to them, you need to click the "Settings" button:

The program settings are collected on four tabs. And the first of them is "Monitoring". Here are the settings that are responsible for the actual tracking functions. Pay attention to the following parameters:

The second tab is "Notification":

Here we can configure the type of notification we receive about changes to the folder. By default, the notification appears above the tray and is accompanied by a standard signal. However, you can turn off the notification altogether, change the signal, or set up a local network notification.

If you decide to set up network notifications, be sure to read the information on the link "Important notes on the operation of the function."

The "Actions" tab contains all the functions available for execution by the program after detecting changes in the monitored folder:

Here we can set up:

launching a specific program or batch script;
sending a report to an e-mail;
sending modified files to an FTP server;
copying the modified files to another folder.

Especially it is necessary to dwell on the point of work with E-mail notifications. If you want to receive reports (as well as modified files) on your email Mailbox, then you need to activate the "Send reports by e-mail" checkbox, and then click the activated "Settings" button:

In the settings window, we need to specify the following parameters:

"SMTP Server". If you use online mail services, then the server address usually corresponds to the address of the service itself, preceded by the prefix smtp.(For example, smtp.mail.ru, smtp.yandex.ru and so on.). If this scheme does not work, then look at the help page of your provider for the exact address.
"Mailing address". Here you specify the address from which the report will be sent. In order to avoid misunderstandings, it is better to indicate your own E-mail.
"Authentication Required" checkbox. If on your mail server authorization is required to send letters (usually on all online services), then you need to activate this option and specify the login and password from your mailbox.
Recipients field. Here, separated by a semicolon, we enter a list of addresses to which reports will be sent.

Additionally, you can pay attention to the option "Attach new files". It allows you to send files that have been changed along with the notification to your mail.

In order not to overload the Internet channel and not waste traffic in vain (if it is paid), you can limit the size of sent files using the option "Do not send files larger than ...". Below it is also possible, on the contrary, to set the files that will always be sent. Such a file can be, for example, a program operation log.

At the end of the settings, you can check its correctness by clicking the "Test" button:

If everything is set up correctly, you will receive an email with a test message.

The last tab is "Program":

Here are the options for general settings A simple watcher, such as autostart at system boot, automatic start of folder monitoring, hiding the application in the tray, etc. Here you can also check for updates and activate the display of pop-up messages (if they are not visible by default).

Watching a folder in Simple Watcher

Now that you have everything set up as you need, you can start working directly with the Simple Observer program. And first of all, you need to specify a folder for monitoring. By default, the entire Drive C: is selected, so to change the path, you need to click the "Folder" button, and specify the desired directory in the Explorer window:

I created a separate test folder on the Desktop, but the location of the folder does not matter - the program works equally well with all disks. However, it is worth mentioning once again that only a local directory can be specified through the Explorer. To monitor remote shared folder its address must be given in first settings window!

So, we are all set - you can start tracking. To do this, click the "Start" button:

After pressing it, the "Stop" button is activated, the "Folder" button is deactivated, and information about the time the folder was monitored, the number of changes in it and the time appears in the status bar. last change. In order to hide the program window, it is enough to minimize it, and it is minimized to tray. You can call the window again with a single click on the tray icon.

To test the functionality of the Simple Observer, let's put some file in our test folder:

As you can see, the program detected the changes and indicated to us that a new file with a specific name appeared in the folder. Also a report about this was sent to my e-mail. Another feature of the Simple Observer message is that if you click on the notification window, your folder will open in front of you, in which the changed file will be highlighted!

View logs

As soon as the Simple Watcher detects the first activity in the watched folder, it will automatically create the file watch.log. This is a text file that records and stores reports of all events that have occurred.

This file can be opened with normal Notepad, but it is more convenient to view it with the built-in tools of the program. To do this, just click the "Log" button:

Here we will see the data about the event (action, date, time, file name and folder that have changed) and we can quickly find the desired entry thanks to the filter system. There is also a button here that allows you to quickly clear the log file if it is too “swollen” in size :).

By the way, knowing the name and location of the log file, you can configure the Simple Observer so that it sends the log file along with each message to your email(see section on settings).

Advantages and disadvantages of the program

installation is not required;
small utility size;
the ability to track subfolders;
sending reports by e-mail;
sending notifications over the local network.

works with only one folder;
consumes quite a lot of resources on older PCs without prior configuration;
there is no way to completely hide the work of the program with "regular" means.

conclusions

To date, Simple Observer is perhaps the only completely free (even for commercial use) program of its kind!

Therefore, if you decide at all costs to install surveillance of personal data on your PC in your absence, then you simply will not find another option :). Moreover, in the program, in addition to supporting the monitoring of only one folder, there are practically no minuses.

P.S. It is allowed to freely copy and quote this article, provided that an open active link to the source is indicated and the authorship of Ruslan Tertyshny is preserved.

If you are not the only PC user, then after a long period of non-use of the machine, you may need to find out exactly what changes have occurred during this period of time. There may also be situations in which HDD shows activity, but in fact it is not clear whether it is busy processing useful processes or it is running malware. The FolderChangesView program will help you track changes in files and get up-to-date data on all changes in the file system.

FolderChangesView is a small utility that was created specifically for keeping track of all the changes that happen to files and folders. The program displays in real time all the changes that occur with the files, and the entire result is shown to the user in a convenient table. You can download the program itself from the official website of the developer at the link (it is also possible to download the crack, which is installed by simply moving it to the folder with the application): http://www.nirsoft.net/utils/folder_changes_view.html

Program settings

The utility does not require installation, because the archive contains only one .exe file. It is recommended to create a separate folder for the program, download the crack and put these two files into it. After starting, the configuration menu will open in front of you (if you close the program and open it again, the menu will appear in front of you again).

Here you need to select a drive, folders or an entire partition hard drive to be analyzed. When selecting a folder, leave the option "Scan also subfolders" checked. You also have the ability to specify folders that you don't need to monitor: you need to specify the path to the folder and check the box next to "Exclute the following folders".
Well, in conclusion, from useful settings, you can set the size of the files that you need to monitor. After selecting these basic settings, simply click "OK" and the process of monitoring the selected folders will begin.

Windows ReadDirectoryChangesW

For the Windows operating system, there is a wonderful function ReadDirectoryChangesW , which returns a set of changes for the directory, including a flag for working recursively (bWatchSubtree). Thus, the implementation of tracking changes in the directory is not difficult, and in the same dklab_realsync implementation takes 80 lines of code or 3.5 Kb. Interestingly, on Windows these events are supported even via SMB!

However, there are certain pitfalls:

the final size of the change buffer, after which the event queue will overflow and these events will be lost
according to the watchdog package documentation, the move event is sent before the changes become visible in the filesystem
buffer size limited to 64 KB for network FS

Mac OS X, FSEvents

Linux, inotify

Pitfalls of recursively following a directory with inotify:

Possibility of overflowing the queue (the length of the queue is set in /proc/sys/fs/inotify/max_queued_events)
Limit on the maximum number of watch objects (set in /proc/sys/fs/inotify/max_user_watches)
Not being able to recursively follow a directory
The need to separately handle the case when a directory is created (eg mkdir -p a/b/c). You will receive an event that the directory “a” has been created, but while you are attaching a handler to this directory, another directory can already be created in it and you will not receive an event about this.
The theoretical possibility of integer overflow of watch descriptor (wd), since it is set by uint32

FreeBSD, Mac OS X, kqueue

FreeBSD and Mac OS X allow you to monitor changes using kqueue, which is similar in characteristics to inotify and also lacks recursive directory monitoring. Also, kqueue takes descriptors of open files (directories) as arguments, so when using this API, the limits on the number of monitored directories are even more strict.

Total:

* Picture taken from

operating room Windows system is equipped with the Explorer tool, which allows you to get an idea of \u200b\u200bwhat is on the disks. In addition to the list of files, we also see information about their size and date of creation or modification. This, however, is too little information to find the data that clogs the disk the most.

The applications in this article can be divided into two categories. The first is programs that specialize in disk visualization. Such a graphical representation of the data is perceived better than a list of files. The second group of tools are applications that show the participation statistics of individual file types, among all the information stored on selected storage media.

WinDirStat

Represents the size and number of files on a computer's hard drive in graphical form, as well as on removable media of our choice.

Benefits of WinDirStat

An accessible form of graphic visualization
Also works in old Windows versions
You can define how elements are displayed
tree cards

Flaws:

Lack of color detection more file types

Distribution Type: freeware
Price: for free

SpaceSniffer Portable

SpaceSniffer scans and shows you the contents of the selected drives in an easy-to-read hierarchical color scheme.

Benefits of SpaceSniffer

Does not require installation
Extensive filtering options for data that is visualized

Distribution Type: freeware
Price: for free

Tree Size Free

TreeSize Free allows you to quickly scan the contents of any medium, and the results are displayed in a drop-down tree.

Benefits of TreeSize Free

Convenient sorting by size, number of files or folders
Handles alternate file data streams NTFS systems
Interface adapted for touch screens

Flaws:

Only in English version

Distribution Type: freeware
Price: for free

JDiskReport

JDiskReport - for lovers of statistics in the form of charts, reminiscent of charts from Excel. Instead of trees, we have bar charts and pie charts here.

Benefits of JDiskReport

Interesting graphic schemes
Many different statistics, including division by size and creation date
Available Java version that works on any platform with Java

Flaws:

The Windows version requires Java to be installed

Distribution Type: freeware
Price: for free

FolderSize

FolderSize is highlighted by the summary window. It displays charts and a list of files or folders that can be sorted according to various criteria.

Benefits of FolderSize

Displays charts and file list in one window
Allows you to pause and resume the scanning procedure, quickly takes into account changes in already checked places

Flaws: