Hi,
What do people think of the following suggestion?
After the comparison, but before the synchronization, allow a user-supplied program to process the list of files to be presented to the synchronization process. This user program would be able to change the "status" of each file (the middle column in the GUI), based on criteria more refined than simple file differences. For instance, I need to synchronize configuration files that change from time to time but are generally are identical except for paths or user names; FFS correctly flags these as different, but manual inspection indicates they should be left alone. Another use case would be to merge log files, or to reverse the direction of synchronization based on file contents.
A user would need to write a program to fit an API similar to this:
Input: A list of left/right file pairs, each with a "CompareFileResult" status (i.e., LEFT_SIDE_ONLY, RIGHT_SIDE_ONLY, LEFT_NEWER, RIGHT_NEWER, DIFFERENT_CONTENT, CONFLICT). Files that are "equal" do not need to be processed and therefore would not be part of this list.
Output: A list of left/right file pairs, each with the new, desired, "CompareFileResult" status. Either the original list, or just the list status changes (e.g. an empty list if no statuses are to be changed). As a bonus, the program could supply totally new files to replace the left and/or right files (each new file could be a merge of the left+right files, etc.).
The UI would be something like this:
1) The user sets the (optional) post-comparison program in the same window where "Run a command:" is defined.
2) The user starts a comparison.
3) The user (optionally) presses the "post-comparison filter" button.
4) The user-defined program is launched by FFS, and is passed the information defined in the API above.
5) The user-defined program responds with new, updated, statuses for the file pairs.
6) FFS would update the middle column of the GUI, reflecting the new desired statuses, possibly removing files from the list (I could see a need to add files if BOTH left+right need to change), possibly with different icons to indicate the status has changed.
7) The user can then continue with the usual workflow, and allow FFS to perform the synchronization.
I'm pretty sure the above cannot be achieved with the existing filters.
I wrote a program that runs on completion to adjust paths and usernames after synchronization, but it's very specialized (the program needs to scan the same directories as FFS) and wasteful (it undoes what FFS did).
Am I the only one with this need?
Can I help?
Thanks,
Martin
Suggestion: Post-comparison filter
- Posts: 3
- Joined: 25 May 2023
-
- Posts: 3582
- Joined: 11 Jun 2019
Sounds incredibly complicated and niche. If you are undoing what FFS does, why not filter them out anyway?
- Posts: 3
- Joined: 25 May 2023
I want to filter some files based on contents of the files, which FFS doesn't support. Let me give you two actual examples:
• FFS stores in GlobalSettings.xml some settings that are semi-permanent, and timestamps of the last synchronizations (which change often). When I compare this file to its backup, FFS always says it's different, and while the difference isn't significant, it looks like the backup should be done again and again. I suggest to have a means to ignore timestamp changes to determine if the file has changed.
• Photoshop stores several complex options in XML files, some of which are user/path dependent. When I replicate these XML files to my secondary computer to propagate the changes to the options, I need to fix the paths after FFS does its job (this is what I mean by undoing what FFS does). I suggest to have a means to ignore path differences inside the XML file to determine if the file has changed, and even better to fix the paths while the file is being replicated by FFS.
I did suspect it was a niche use case, as you mentioned. I'm not sure it's that complicated; I browsed the sources and I think the program structure would allow it (send a list to a program, remove the items sent back by the program), but I haven't programmed in C++ in years, otherwise I'd have hacked a proof of concept.
My goal was to have comments, so thanks for taking the time to add yours!
• FFS stores in GlobalSettings.xml some settings that are semi-permanent, and timestamps of the last synchronizations (which change often). When I compare this file to its backup, FFS always says it's different, and while the difference isn't significant, it looks like the backup should be done again and again. I suggest to have a means to ignore timestamp changes to determine if the file has changed.
• Photoshop stores several complex options in XML files, some of which are user/path dependent. When I replicate these XML files to my secondary computer to propagate the changes to the options, I need to fix the paths after FFS does its job (this is what I mean by undoing what FFS does). I suggest to have a means to ignore path differences inside the XML file to determine if the file has changed, and even better to fix the paths while the file is being replicated by FFS.
I did suspect it was a niche use case, as you mentioned. I'm not sure it's that complicated; I browsed the sources and I think the program structure would allow it (send a list to a program, remove the items sent back by the program), but I haven't programmed in C++ in years, otherwise I'd have hacked a proof of concept.
My goal was to have comments, so thanks for taking the time to add yours!
-
- Posts: 3582
- Joined: 11 Jun 2019
FFS does actually allow comparing by content, it just takes significantly longer. The time wasted syncing the GlobalSettings.xml file is less than how long it would take to compare the contents of the source and destination to determine it doesn't need to be synced.
The Photoshop example is semi-legitimate, as I ran into something similar. Here was my situation: Two computers, desktop at home and laptop for school. I used a program for school work and would save project files that depended on consistent absolute paths instead of relative locations. The problem for me was that my two computers had different user folder paths. When trying to open these projects on the computer that they weren't created on, I got errors because my user folder names weren't consistent. The way I fixed it was by creating a hard link, I think, on each computer in the Users folder. The link names were set to the user folder name of the other computer's account and pointed to the local user folder. My laptop user was just "Owner" and my desktop was "jorda" (MS account), vice versa on the desktop. So on the desktop, I created a link named "Owner" that pointed to "jorda" and a link named "jorda" pointing to "Owner" on the laptop.
You may look into this possibility. It will prevent having to constantly make changes to your program every time you use a program that uses absolute paths instead of a variable, like %APPDATA%. The best solution, which I did as soon as the semester ended, is to make the two computers have the same user paths. I did this by buying a new laptop and setting it up the correct way lol. I was due for an upgrade anyway. You could also just create another user account with the correct user folder name, then move the data over.
The Photoshop example is semi-legitimate, as I ran into something similar. Here was my situation: Two computers, desktop at home and laptop for school. I used a program for school work and would save project files that depended on consistent absolute paths instead of relative locations. The problem for me was that my two computers had different user folder paths. When trying to open these projects on the computer that they weren't created on, I got errors because my user folder names weren't consistent. The way I fixed it was by creating a hard link, I think, on each computer in the Users folder. The link names were set to the user folder name of the other computer's account and pointed to the local user folder. My laptop user was just "Owner" and my desktop was "jorda" (MS account), vice versa on the desktop. So on the desktop, I created a link named "Owner" that pointed to "jorda" and a link named "jorda" pointing to "Owner" on the laptop.
You may look into this possibility. It will prevent having to constantly make changes to your program every time you use a program that uses absolute paths instead of a variable, like %APPDATA%. The best solution, which I did as soon as the semester ended, is to make the two computers have the same user paths. I did this by buying a new laptop and setting it up the correct way lol. I was due for an upgrade anyway. You could also just create another user account with the correct user folder name, then move the data over.
- Posts: 3
- Joined: 25 May 2023
Hi,
As far as I know, when FFS compares by contents, it's all or nothing (the whole file has to match); for some files, I want to ignore part of their contents.
As for Photoshop, I am using multiple disks on my main computer (for parallel performance reasons), but a single disk for a subset of the files on my secondary computer (a laptop). I am actually using the same username on both computers, but %AppData% doesn't cover the fact that some files are on C:\...\AppData and some files are on D:\...\AppData (even some Microsoft programs disregard the value of %AppData% and go straight to C:\...\AppData). Your suggestion of using hard links would be perfect on Linux, but it's trickier on Windows. Technically speaking, a Junction is what is used for directories, and I'm not sure I can create a junction from D: to C: on a computer where the D: drive doesn't exist. I'll look into it, but it doesn't address the other use cases I have (and it's beyond the scope of a FFS discussion...).
Thanks
As far as I know, when FFS compares by contents, it's all or nothing (the whole file has to match); for some files, I want to ignore part of their contents.
As for Photoshop, I am using multiple disks on my main computer (for parallel performance reasons), but a single disk for a subset of the files on my secondary computer (a laptop). I am actually using the same username on both computers, but %AppData% doesn't cover the fact that some files are on C:\...\AppData and some files are on D:\...\AppData (even some Microsoft programs disregard the value of %AppData% and go straight to C:\...\AppData). Your suggestion of using hard links would be perfect on Linux, but it's trickier on Windows. Technically speaking, a Junction is what is used for directories, and I'm not sure I can create a junction from D: to C: on a computer where the D: drive doesn't exist. I'll look into it, but it doesn't address the other use cases I have (and it's beyond the scope of a FFS discussion...).
Thanks
