environment variables for filter settings

[micha--]

Dear Zenju,

first: Thank you very much for this good program, and your support :) :) :) :)

Edit (23.02.2015 23:55 <== Thanks to Plerry, who shows me the Syntax: %variable:StrToFind=NewStr%):

I need environment variables for filter settings in *.ffs_batch, *.ffs_gui, they will not be expanded within Filter-Items.

My use case is:
- A disc, which contains a WIN System and user data.
- I want to save the user written data, but not the windows system.

For that I would like to have evaluation of environment variables in Filter-Items (Include | Exclude --> Item).

%HomePath% is expanded to --> \Users\<myName>

It has no DriveLetter, hence i try to use it in Filter-Items.

%WinDir% is expanded to --> C:\Windows

The DriveLetter can removed by:

%WinDir:*:=% is expanded to --> \Windows

I have tested %HomePath% and %WinDir:*:=% both in *.ffs_gui and in *.ffs_batch with a filter-configuration like the following. But the environment variables will not be expanded within the filter-items.

        <GlobalFilter>
            <Include>
                <Item>*</Item>
            </Include>
            <Exclude>
                <Item>\hiberfil.sys</Item>
                <Item>\pagefile.sys</Item>
                <Item>\System Volume Information\</Item>
                <Item>\$Recycle.Bin\</Item>
                <Item>\RECYCLER\</Item>
                <Item>\RECYCLED\</Item>
                <Item>%WinDir:*:=%\</Item>
                <Item>%ProgramFiles:*:=%\</Item>
                <Item>%ProgramFiles(x86):*:=%\</Item>
                <Item>%ProgramData:*:=%\</Item>
                <Item>%HOMEPATH%\AppData\Local\Microsoft\Windows\</Item>
                <Item>%HOMEPATH%\AppData\Local\Temp\</Item>
                <Item>%HOMEPATH%\NTUSER.DAT</Item>
                <Item>%HOMEPATH%\ntuser.dat.LOG1</Item>
                <Item>%HOMEPATH%\ntuser.dat.LOG2</Item>
            </Exclude>
            ...
        </GlobalFilter>


E.g. if the left/right folder pair's are "C:\" and "D:\testFFS",
then e.g. Filter-Item "%WinDir:*:=%\" (expanded to --> "\Windows\") leads to the left/right pathes:

"C:\" + "\Windows\" = "C:\Windows\" = "C:\Windows\"
and
"D:\testFFS" + "\Windows\" = "D:\testFFS\Windows\"

--------

Edit (23.02.2015 23:55):
The following please look as deleted. I'll leave it only for documentation here.
Edit (20.02.2015 20:45):

%HOMEPATH% is expanded --> \Users\<myName>

It has no DriveLetter, hence it can be used in filter Items (in <Include> ... </Include>, <Exclude> ... </Exclude>).

Many environment variables in WIN are expanded to "DriveLetter + Path", e.g.:

    %SystemRoot% or %WinDir% --> C:\Windows
    %ProgramFiles% --> C:\Program Files
    %ProgramFiles(x86)% --> C:\Program Files (x86)
    %ProgramData% --> C:\ProgramData


http://en.wikipedia.org/wiki/Environment_variable#Windows


These environment variables can be used in FolderPairs.
If they are used in *filter Items*, the DriveLetter is damaging, because a filter item (e.g. "c:\Windows") is appended to the left/right pair's folder (e.g. "C:\" and "D:\testFFS"):

Then left side: "C:\" + "c:\Windows" = "C:\c:\Windows"
and right side: "D:\testFFS" + "c:\Windows" = "D:\testFFSc:\Windows"

Hence the DriveLetter must be cropped: With "\Windows" (without DriveLetter)

the left side: "C:\" + "\Windows" = "C:\\Windows" = "C:\Windows"
and right side: "D:\testFFS" + "\Windows" = "D:\testFFS\Windows"

works fine.

Hence i ask you for the following treatment of environment variable values used in filter Items:

* If it not begins with "\\" and the 2nd char is not a ":", then do nothing.
* If it not begins with "\\" and the 2nd char is a ":", then use the substring without 1st and 2nd char.
* If it begins with "\\" it is a Universal/Uniform Naming Convention - location.
http://en.wikipedia.org/wiki/Path_%28computing%29#Uniform_Naming_Convention
I think, this can be implemented later, if someone needed it.


Thanks, micha

[Plerry]

The help-file suggests this is presently already supported.
(under the indicated limitations).
Look at the page bottom of the topic Macros.

[micha--]

Sorry, you are right. I have testet with an old Version. I will test with the new Version, now, and give feedback ...

And thanks for the hint.

[micha--]

OK, i have tested now. It doesn't work on my WIN 7, with FFS 6.14. It doesn't matter if i doubleclick test.ffs_gui in the filemanager, or if i start
"C:\Program Files\FreeFileSync\FreeFileSync.exe" "test.ffs_gui"
at cmd-line.

I have also tested with

<Item>%SystemRoot%\</Item>

or with

<Item>%WinDir%\</Item>


Can someone else test it too? To test whether it is just a bug in my system or an error in FFS.

For testing you can use the following test.ffs_gui. First make a subdirectory on e.g. D:\testFFS, it is used at bottom of file.

<?xml version="1.0" encoding="UTF-8"?>
<FreeFileSync XmlFormat="4" XmlType="GUI">
    <MainConfig>
        <Comparison>
            <Variant>TimeAndSize</Variant>
            <TimeShift>0</TimeShift>
            <Symlinks>Exclude</Symlinks>
        </Comparison>
        <SyncConfig>
            <Variant>Mirror</Variant>
            <CustomDirections>
                <LeftOnly>right</LeftOnly>
                <RightOnly>left</RightOnly>
                <LeftNewer>right</LeftNewer>
                <RightNewer>left</RightNewer>
                <Different>none</Different>
                <Conflict>none</Conflict>
            </CustomDirections>
            <DetectMovedFiles>false</DetectMovedFiles>
            <DeletionPolicy>RecycleBin</DeletionPolicy>
            <VersioningFolder Style="Replace"/>
        </SyncConfig>
        <GlobalFilter>
            <Include>
                <Item>\Windows\</Item>
            </Include>
            <Exclude>
                <Item>%WinDir:*:=%\</Item>
                <!-- Item>\Windows\</Item -->
            </Exclude>
            <TimeSpan Type="None">0</TimeSpan>
            <SizeMin Unit="None">0</SizeMin>
            <SizeMax Unit="None">0</SizeMax>
        </GlobalFilter>
        <FolderPairs>
            <Pair>
                <Left>C:\</Left>
                <Right>D:\testFFS</Right>
            </Pair>
        </FolderPairs>
        <OnCompletion/>
    </MainConfig>
    <GuiConfig>
        <HandleError>Popup</HandleError>
        <MiddleGridView>Action</MiddleGridView>
    </GuiConfig>
</FreeFileSync>


[Plerry]

No time to test right now; however ...
I'm not sure if this works for *.ffs_gui-files.
The Macro-section of the help-file refers to *.ffs_batch-files.
That is obviously where the use of environment variables makes most sense.

If a batch-approach works but is no option for you, perhaps you can use
a command-line option (see the help-file "Command Line Usage" section,
particularly item 3)

[Plerry]

Just noticed:
It seems you are using %SystemRoot% and %WinDir% in the include or exclude list.
However, if you check the value of theses variables you will notice they contain
the absolute location (like C:\Windows).
These absolute addresses are perfect for defining the folder pairs.
However, include/exclude definitions are/must be relative to the left/right pair's
root, and can not be absolute.

[micha--]

Thanks for answering your 2 points :)

1. I will test with a *.ffs_batch file and give feedback.

2. Yes, "C:\Windows" can not be applied to relative paths. I hope Zenju strip off the "C:" from begining, for fitting the condition "relative to the left/right pair's root".
Then e.g. "\Windows" appends to the left side is "C:\Windows" and appended it to the right side is "D:\testFFS\Windows"
<-- left side = "C:\" and right side = "D:\testFFS\".
I edit this in my first post.

> ... *.ffs_batch-files. That is obviously where the use of environment variables makes most sense.

No, i think GUI and Batch are only two modi of running FFS. Like driving a car by navi or by map, you want to travel from A to B, and both navi-mode or map-mode yield you from A to B. You can use environment variables in GUI-mode same as in Batch-mode (in my use case i want to do so). Therefore i will ask Zenju to treat GUI- and Batch-mode equal.

[micha--]

1. I have tested it in BATCH-mode ( *.ffs_batch) and in GUI-mode ( *.ffs_gui), with

        <GlobalFilter>
            <Include>
                <Item>\Windows\</Item>
            </Include>
            <Exclude>
                <Item>%WinDir:*:=%\</Item>
            </Exclude>
            <TimeSpan Type="None">0</TimeSpan>
            <SizeMin Unit="None">0</SizeMin>
            <SizeMax Unit="None">0</SizeMax>
        </GlobalFilter>


and with

            <Include>
                <Item>\Users\myName\</Item>
            </Include>
            <Exclude>
                <Item>%HomePath%\</Item>
            </Exclude>


Exclude did not work in all 4 cases. And all 4 cases works, if i write
\Windows instead of %WinDir%
respectively
\Users\myName instead of %HomePath%

[Plerry]

1. I have tested it in BATCH-mode ( *.ffs_batch) and in GUI-mode ( *.ffs_gui), with

        <GlobalFilter>
            <Include>
                <Item>\Windows\</Item>
            </Include>
            <Exclude>
                <Item>%WinDir:*:=%\</Item>
            </Exclude>
            <TimeSpan Type="None">0</TimeSpan>
            <SizeMin Unit="None">0</SizeMin>
            <SizeMax Unit="None">0</SizeMax>
        </GlobalFilter>


and with

            <Include>
                <Item>\Users\myName\</Item>
            </Include>
            <Exclude>
                <Item>%HomePath%\</Item>
            </Exclude>


Exclude did not work in all 4 cases. And all 4 cases works, if i write
\Windows instead of %WinDir%
respectively
\Users\myName instead of %HomePath%

This result is as expected, due to the absolute path included in
the %WinDir% and %HomePath% variables ...

In your earlier reaction you suggested Zenju to make FFS evaluate
the variables and convert absolute location into relative, where applicable.
I feel this is not realistic, as it would need to get converted to the left/right
pair root. But ... to the root of left or the root of right?
And how to deal with multiple left/right pairs and global includes/excludes?
Additionally, it would be quite specific.
As you know exactly what you want, it is much simpler to define/calculate
yourself a custom variable e.g. by removing the %SystemDrive% part
from %WinDir% in your batch/cmd-file.

As an aside:
In your example you seem to try to define one and the same folder
(e.g. \Windows\) in both the include and exclude filter specification.
That is obviously very strange.
Normally the include filter specifies all files (*), or specific file-types
(e.g. *.pdf) to be included,
and the exclude specifies filter specific files (e.g. *\desktop.ini), file-types
(e.g. *.tmp) and/or specific folders (e.g. *\#recycler\) to be excluded.

[micha--]

> This result is as expected, due to the absolute path included in
the %WinDir% and %HomePath% variables ...

No. Not "absolute" or "relative path", but "with" or "without Driveletter". Thats all (nothing must to be converted accordingly to the left/right folder pair root). And you are right: my asking for 'Pleasse strip off the "C:" from begining (e.g. "\Windows" <-- "C:\Windows"), ...' is not precise enough. I have edited this at bottom of my first post. I hope it is exact now. Please read there and give me your feedback. And Thank you for your critic.

> As you know exactly what you want, it is much simpler to define/calculate
yourself a custom variable e.g. by removing the %SystemDrive% part
from %WinDir% ...

I have tried to use the ~P operator on environment variables, but ~P only works on FOR-LOOP-Parameters and on batch/cmd-file-Parameters (%0, %1, ...) :(


> As an aside:
In your example you seem to try to define one and the same folder
(e.g. \Windows) in both the include and exclude filter specification.

Yes, and so you can test easily, if in

            <Include>
                <Item>\Users\myName\</Item>
            </Include>
            <Exclude>
                <Item>%HomePath%\</Item>
            </Exclude>


%HomePath% is resolved (to \Users\myName), then we have

            <Include>
                <Item>\Users\myName\</Item>
            </Include>
            <Exclude>
                <Item>\Users\myName\</Item>
            </Exclude>


and FFS's file list is empty (==> nothing to synchronize).

If %HomePath% is not resolved, then

            <Include>
                <Item>\Users\myName\</Item>
            </Include>
            <Exclude>
                <Item>%HomePath%\</Item>
            </Exclude>


excludes all folders named "%HomePath%\" (then % is only a sign like x) and FFS's file list consists of all files from "\Users\myName\" (since within is no folder named "%HomePath%\") (==> something to synchronize).


> That is obviously very strange.
Normally the include filter specifies all files (*), or specific file-types
(e.g. *.pdf) to be included,
and the exclude specifies filter specific files (e.g. *\desktop.ini), file-types
(e.g. *.tmp) and/or specific folders (e.g. *\#recycler\) to be excluded.

Yes, you are right, but this use case was not "normal use", but "test, if %HomePath% is resolved" :)

----------

And (i wrote it above) in none of the 4 cases:
- %HomePath% in *.fss_gui
- %HomePath% in *.fss_batch
- %WinDir% in *.fss_gui
- %WinDir% in *.fss_batch
was the environment variable resolved.

[Plerry]

> And (i wrote it above) in none of the 4 cases:
- ...
was the environment variable resolved.

We keep running in circles ...
As long as you keep using these variables in this form in the include/exclude
definitions, it will never work as (again) includes/excludes are/must be relative
to the applicable lef/right pair.

You can try if variable substitution does work
by replacing <Left>C:\</Left> by <Left>%SystemDrive%</Left>
in your folder-pair definition, as folder-pairs an absolute location
must be specified.

If, as per your suggestion, variables used in the include/exclude definition were to be
stripped of just their drive-letter it might suit your specific case, as your left-part
of the pair is simply C:\.
However, both the left and right location can be specified to be essentially
anywhere/anything. If Zenju were to change anything in FFS, it would obviously
need to fit any arbitrary left and/or right location.
Hence my previous remarks about left-right pairs and not finding it realistic to expect
that FFS would be changed to strip variables used in the include/exclude definition.

> I have tried to use the ~P operator on environment variables, but ~P only works
on FOR-LOOP-Parameters and on batch/cmd-file-Parameters (%0, %1, ...) :(

Try using the variable edit/replace command.
See e.g. http://ss64.com/nt/syntax-replace.html
As you need to refer to another variable in the "find"-part, you need to use the
indirect method via a call-statement:
call set MyVar=%%WinDir:%SystemDrive%=%%\
If %WinDir% equals C:\Windows, and %SystemDrive% equals C:\, %MyVar% will become \Windows\.
(The backslash at the end is due to the addition of "\" behind the "call set"-statement.

You can now refer to your custom variable MyVar as described at the bottom
of the Macro-section in the FFS-help.
This should solve the problem for you specific case.

[micha--]

>As long as you keep using these variables in this form in the include/exclude
definitions, it will never work as (again) includes/excludes are/must be relative
to the applicable lef/right pair.

Sorry, i don't understand. I need an example of absolute lef/right folder pairs, in which my example of Filter-Items break, in my first post.


> If Zenju were to change anything in FFS, it would obviously
need to fit any arbitrary left and/or right location.

No, in "FolderPairs --> Pair --> Left | Right" is nothing to change. But i needed evaluation of environment variables in "Include | Exclude --> Item".


> Hence my previous remarks about left-right pairs and not finding it realistic to expect
that FFS would be changed to strip variables used in the include/exclude definition.

No "strip" necessary anymore, due to your following hint:


>Try using the variable edit/replace command.
See e.g. http://ss64.com/nt/syntax-replace.html

Thank you very much for the link --> Variable Edit/Replace - Syntax :) :) :)

The following are working in a cmd-Window:

echo %WinDir:c:=%

echo %WinDir:*:=%

call set x=%WinDir:%SystemDrive%=%
echo %x%

>You can now refer to your custom variable MyVar as described at the bottom
of the Macro-section in the FFS-help.
This should solve the problem for you specific case.

No: I need environment variables in Filter-Items.

Thank you for the "%variable:StrToFind=NewStr%" - Syntax :)

------------

I have edited my first post.

[Plerry]

@micha
You forgot to escape some of the backslashes in your updated code:
e.g. <Item>%WinDir:*:=%\</Item>
should read <Item>%WinDir:*\:=%\</Item>
Note the added backslash (\) behind the asteriks (*)

By the way:
It is not really clear from your last reply and
updated first post if the use of (modified) variables
now works in include/exclude definitions or not.
If not, this could be an aspect Zenju could consider to add to FFS.

[micha--]

>You forgot to escape some of the backslashes in your updated code: ...

No, i don't want a "\" beetwen "*" and ":"

>By the way:
Nice to see that even variable operators work in FFS.

Nice it would be, but environment variables will not be expanded within filter-items.

[Plerry]

>You forgot to escape some of the backslashes in your updated code: ...

No, i don't want a "\" beetwen "*" and ":"

>By the way:
Nice to see that even variable operators work in FFS.

Nice it would be, but environment variables will not be expanded within filter-items.

> No, i don't want a "\" beetwen "*" and ":"

Sorry, I missed your ":" before the "=" ...
I was thought you were doing
<Item>%WinDir:*\=%\</Item>
which seems would also work in your specific case.
But, your solution is more elegant.

[micha--]

Now the Problem seems to me to be well formulated (in the first post) and is waiting for solving.

[micha--]

> No, i don't want a "\" beetwen "*" and ":"

Sorry, I missed your ":" before the "=" ...
I was thought you were doing
<Item>%WinDir:*\=%\</Item>
which seems would also work in your specific case.
But, your solution is more elegant.

echo %WinDir:*\=%

echo %WinDir:*:=%

You can test it in a cmd-Window :)