Hi!
I would like to sync a zip file that comes from the internet, and that is dynamically generated each time I access an address (address resembling http://arandomnas.myqnapcloud.com:8080/share.cgi?ssid=Tw3g9O1&ep=Qm2Yx8Ys8F3cMW792ko==), to a local folder on my computer.
Is there a way to do what I want only with FFsync and then export as ffs_batch file? Whether there is or not, how would you sync the zip file to your computer?
Every answer is welcome, even if you just give me ideas on how to achieve this.
Notes :
- The zip file contains an arborescence of files that is hosted on a NAS.
I am not the admin and I think I can't just use FTP.
- Files are removed and added regularly from the NAS, so I can't just overwrite the zip file every time it changes.
- Here is how I want the files to be synced :
- Keep locally files removed remotely
- Update locally files updated remotely
- Add locally files added remotely
Sync online zip file with local folder
-
- Posts: 6
- Joined: 18 May 2020
Last edited by Maxime on 14 Aug 2020, 10:29, edited 1 time in total.
-
- Posts: 2272
- Joined: 22 Aug 2012
With FFS, you can essentially only sync files one-on-one.
So, you can sync a ZIP-file to another ZIP-file.
But FFS (at least presently) does not do any zipping or unzipping.
So, you can sync a ZIP-file to another ZIP-file.
But FFS (at least presently) does not do any zipping or unzipping.
-
- Posts: 6
- Joined: 18 May 2020
This ZIP-file to ZIP-file syncing is not interesting here, since if some files were updated in the remote ZIP-file and some deleted, the only possibility I have is to overwrite one file or the other, which would lose updates or files deleted remotely.
Since I can't do what I want using FFS only, how would you approach it?
Update : By "ZIP-file to ZIP-file syncing", I mean syncing the 2 ZIP-files, each inside a folder, seeing them as files and not compressed folders.
Since I can't do what I want using FFS only, how would you approach it?
Update : By "ZIP-file to ZIP-file syncing", I mean syncing the 2 ZIP-files, each inside a folder, seeing them as files and not compressed folders.
Last edited by Maxime on 14 Aug 2020, 06:31, edited 1 time in total.
- Posts: 12
- Joined: 12 Aug 2020
Hello Maxime,
let me see if I have well understood what you want.
For the last 3 requisites, you can use the "Update" type of sync of FFS because it does exactly it, doesn't remove from destination deleted files on source, updates updated files and adds new ones.
So the only 2 problem you need to resolve is to download the zip file and unzip it to a temp folder that will be the "source" for FFS.
To download it from command line you could use WGET or PowerShell:
https://superuser.com/questions/25538/how-to-download-files-from-command-line-in-windows-like-wget-or-curl
To unzip it, the first simple solution that comes to my mind is to use the command line version of 7-Zip.
You can install it from the official site:
https://www.7-zip.org/
Than, add the path of the installation directory to the environment variable like stated here:
https://stackoverflow.com/questions/14122732/unzip-files-7-zip-via-cmd-command
Than, with a simple command you can extract the files recursively to a specified folder.
Have a look at this page: https://sevenzip.osdn.jp/chm/cmdline/commands/extract_full.htm
Finally, if everything works manually as expected, you can create a FFS batch for automating the sync.
Than, you can create a batch file Update.bat that do all the tasks sequentially calling:
wget
7z
FreeFileSync
You can also schedule the execution of batch file at regular interval by Task Scheduler
I hope this could help.
Bye
Daniele
EDIT: I forgot to say: at the end of the batch, clean (remove) the temp directory so you can start with a fresh one at next execution.
let me see if I have well understood what you want.
For the last 3 requisites, you can use the "Update" type of sync of FFS because it does exactly it, doesn't remove from destination deleted files on source, updates updated files and adds new ones.
So the only 2 problem you need to resolve is to download the zip file and unzip it to a temp folder that will be the "source" for FFS.
To download it from command line you could use WGET or PowerShell:
https://superuser.com/questions/25538/how-to-download-files-from-command-line-in-windows-like-wget-or-curl
To unzip it, the first simple solution that comes to my mind is to use the command line version of 7-Zip.
You can install it from the official site:
https://www.7-zip.org/
Than, add the path of the installation directory to the environment variable like stated here:
https://stackoverflow.com/questions/14122732/unzip-files-7-zip-via-cmd-command
Than, with a simple command you can extract the files recursively to a specified folder.
Have a look at this page: https://sevenzip.osdn.jp/chm/cmdline/commands/extract_full.htm
Finally, if everything works manually as expected, you can create a FFS batch for automating the sync.
Than, you can create a batch file Update.bat that do all the tasks sequentially calling:
wget
7z
FreeFileSync
You can also schedule the execution of batch file at regular interval by Task Scheduler
I hope this could help.
Bye
Daniele
EDIT: I forgot to say: at the end of the batch, clean (remove) the temp directory so you can start with a fresh one at next execution.
-
- Posts: 6
- Joined: 18 May 2020
Thank you very much!
I've followed your tips, modifying some things, and I've come up with this batch file:
It works perfectly when launched from cmd.exe, so thanks a lot to you, jdaniele, but I still have many problems.
- I can't launch the batch file just by clicking on it. simple commands as "cd" do not work. I have made a batch file containing only the following code, and it doesn't even work.
But I've asked for a solution on another forum (https://stackoverflow.com/questions/63410392/cd-doesnt-work-in-batch-file-in-very-simple-script) since it isn't related to FFS.
- I couldn't find a way to delete the folder after FFS finished the execution: I have the feeling that when I execute the following line, it just launches the process and doesn't start for it to end, so i couldn't just delete the folder on the next line.
- Even after adding FFS to Path, I can't use FFS from command line by typing "freefilesync" instead of the ugly "C:\Program Files\FreeFileSync\FreeFileSync.exe".
Here's how I added FFS to Path:
- I opened the "Edit the system environment variables" settings menu by searching it via the start menu
- I clicked "Environment variables"
- Then "Edit..."
- Then "New"
- And I finally typed "C:\Program Files\FreeFileSync\" and saved.
But since I have the same problem with 7-zip, maybe I should ask for help on another forum.
I hope someone will be able to help with my problems, especially the 2nd one. I didn't use the Task Scheduler yet since I can't execute the batch file directly: I have to run it by typing the path to it in cmd.exe. But it seems perfect in my case.
I've followed your tips, modifying some things, and I've come up with this batch file:
@echo off
for /f "tokens=2 delims==" %%a in ('wmic OS Get localdatetime /value') do set "dt=%%a"
set "YYYY=%dt:~0,4%" & set "MM=%dt:~4,2%" & set "DD=%dt:~6,2%"
set "HH=%dt:~8,2%" & set "Min=%dt:~10,2%" & set "Sec=%dt:~12,2%"
set "fullstamp=%YYYY%-%MM%-%DD%_%HH%-%Min%-%Sec%"
mkdir "%TEMP%\%fullstamp%"
cd "%TEMP%\%fullstamp%"
"C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -Command "& {iwr -outf Optinfo.zip 'http://arandomnas.myqnapcloud.com:8080/share.cgi?ssid=Tw3g9O1&ep=Qm2Yx8Ys8F3cMW792ko=='}"
"C:\Program Files\7-Zip\7z.exe" x Optinfo.zip -oOptinfo
"C:\Program Files\FreeFileSync\FreeFileSync.exe" "D:\Documents\Poinca\MP\4 OPT\FFS\SyncSettings.ffs_batch"
- I can't launch the batch file just by clicking on it. simple commands as "cd" do not work. I have made a batch file containing only the following code, and it doesn't even work.
cd D:
PAUSE
- I couldn't find a way to delete the folder after FFS finished the execution: I have the feeling that when I execute the following line, it just launches the process and doesn't start for it to end, so i couldn't just delete the folder on the next line.
"C:\Program Files\FreeFileSync\FreeFileSync.exe" "D:\Documents\Poinca\MP\4 OPT\FFS\SyncSettings.ffs_batch"
Here's how I added FFS to Path:
- I opened the "Edit the system environment variables" settings menu by searching it via the start menu
- I clicked "Environment variables"
- Then "Edit..."
- Then "New"
- And I finally typed "C:\Program Files\FreeFileSync\" and saved.
But since I have the same problem with 7-zip, maybe I should ask for help on another forum.
I hope someone will be able to help with my problems, especially the 2nd one. I didn't use the Task Scheduler yet since I can't execute the batch file directly: I have to run it by typing the path to it in cmd.exe. But it seems perfect in my case.
Last edited by Maxime on 14 Aug 2020, 14:46, edited 1 time in total.
- Posts: 12
- Joined: 12 Aug 2020
Hi Maxime, you are welcome.
About the cd D:, as you can see, they answered to you. So that should be OK now.
About deleting the folder, after executing the batch, can you open the task manager and check if FFS is still running in the process list?
When you saved the FFS batch, did you check the "Auto-close" option?
If the FFS process is closed but you still have trouble to delete it, try to download the handle command:
https://docs.microsoft.com/en-us/sysinternals/downloads/handle
Run it like that: handle > list.txt
Than open the list.txt and search for the directory you are going to delete.
Than, look at the process that own the handle on it. That should be killed (if possible).
About the steps to add the FFS to the path, here are the correct steps:
- open the "Edit the system environment variables" settings menu by searching it via the start menu
- click on "Environment variables"
- in the "System variables" select the "path" and click edit:
- In the popup that appears, click new and append at the end of the list the FFS full path (ONLY THE PATH, not the EXE) and click OK
- Click OK again to close the Environment window
That should work.
Let me know.
Bye
Daniele
About the cd D:, as you can see, they answered to you. So that should be OK now.
About deleting the folder, after executing the batch, can you open the task manager and check if FFS is still running in the process list?
When you saved the FFS batch, did you check the "Auto-close" option?
- FFS_auto_close.PNG (13.98 KiB) Viewed 3191 times
https://docs.microsoft.com/en-us/sysinternals/downloads/handle
Run it like that: handle > list.txt
Than open the list.txt and search for the directory you are going to delete.
Than, look at the process that own the handle on it. That should be killed (if possible).
About the steps to add the FFS to the path, here are the correct steps:
- open the "Edit the system environment variables" settings menu by searching it via the start menu
- click on "Environment variables"
- in the "System variables" select the "path" and click edit:
- Env_1.PNG (21.91 KiB) Viewed 3191 times
- Env_2.PNG (14.2 KiB) Viewed 3191 times
That should work.
Let me know.
Bye
Daniele
-
- Posts: 6
- Joined: 18 May 2020
- Sorry about what I said about how I added FreeFileSync to the PATH, I made an unfortunate typo: I did put the parent directory in the PATH (typo corrected). I did exactly the steps you just sent me, and cmd.exe still outputs the too familiar error:
- I can't remember if I had checked the Auto-close box, so I exported the file again.
- My problem with the command that launches the synchronisation is not that FreeFileSync doesn't stop, but rather that cmd.exe does not wait for the process to stop before trying to delete the folder. This comes from the fact that (correct me if I'm wrong) the command to launch the synchronisation launches another process rather than executing the sync entirely before executing the next command (here, delete the folder).
- Should I add a few lines of code (using the handle command) to find the process of the sync, and constantly check if it's still active, to finally delete the folder when it closes?
- Otherwise, I could find and delete the folder remaining from the last run each time I run the script, and replace it. That is really not ideal, but it would allow me to avoid having a lot of heavy folders that do not disappear before one week: I would have just one. Not ideal, but a semi-solution nevertheless. I think it would be far easier to code than if I have to find a process in a text file (but maybe PowerShell could help me there, given that it can output objects).
- Another solution would be to wait a safe time period, like 30 minutes, before deleting the folder.
- Or I could just try and delete it every minute (not by force) and count on the protection of Windows that doesn't allow deleting active folders (although I'm not sure that this protection is implemented with cmd.exe)
- But if you have a solution to just be able to launch the process and wait until it's closed to execute the rest of the commands, it would be really cool.
- I'll update you about my problem with the cd command.
'freefilesync' is not recognized as an internal or external command, operable program or batch file.
- My problem with the command that launches the synchronisation is not that FreeFileSync doesn't stop, but rather that cmd.exe does not wait for the process to stop before trying to delete the folder. This comes from the fact that (correct me if I'm wrong) the command to launch the synchronisation launches another process rather than executing the sync entirely before executing the next command (here, delete the folder).
- Should I add a few lines of code (using the handle command) to find the process of the sync, and constantly check if it's still active, to finally delete the folder when it closes?
- Otherwise, I could find and delete the folder remaining from the last run each time I run the script, and replace it. That is really not ideal, but it would allow me to avoid having a lot of heavy folders that do not disappear before one week: I would have just one. Not ideal, but a semi-solution nevertheless. I think it would be far easier to code than if I have to find a process in a text file (but maybe PowerShell could help me there, given that it can output objects).
- Another solution would be to wait a safe time period, like 30 minutes, before deleting the folder.
- Or I could just try and delete it every minute (not by force) and count on the protection of Windows that doesn't allow deleting active folders (although I'm not sure that this protection is implemented with cmd.exe)
- But if you have a solution to just be able to launch the process and wait until it's closed to execute the rest of the commands, it would be really cool.
- I'll update you about my problem with the cd command.
- Posts: 12
- Joined: 12 Aug 2020
Hi Maxime,
if you open a command prompt, can you just type "SET" and enter to see the list of all environment variables and copy and paste here the path value?
If you can't execute FFS, there should be an explanation.
I followed my own steps and they works.
About waiting for a process before going ahead, have a look at this page:
https://stackoverflow.com/questions/8177695/how-to-wait-for-a-process-to-terminate-to-execute-another-process-in-batch-file
There are many solutions as you can see.
Anyway, the important thing is that FFS closes after sync, and that is the first thing to check in the task manager.
I mentioned "handle" just in case you cannot find WHO is locking the directory.
If after the entire batch you are able to delete it manually, for sure the process ended correctly.
As you said, is probably a problem to WAIT FFS to finish before try to delete the temp folder.
Ok, let me know.
Bye
Daniele
if you open a command prompt, can you just type "SET" and enter to see the list of all environment variables and copy and paste here the path value?
If you can't execute FFS, there should be an explanation.
I followed my own steps and they works.
About waiting for a process before going ahead, have a look at this page:
https://stackoverflow.com/questions/8177695/how-to-wait-for-a-process-to-terminate-to-execute-another-process-in-batch-file
There are many solutions as you can see.
Anyway, the important thing is that FFS closes after sync, and that is the first thing to check in the task manager.
I mentioned "handle" just in case you cannot find WHO is locking the directory.
If after the entire batch you are able to delete it manually, for sure the process ended correctly.
As you said, is probably a problem to WAIT FFS to finish before try to delete the temp folder.
Ok, let me know.
Bye
Daniele
-
- Posts: 6
- Joined: 18 May 2020
Hi jdaniele,
I've done 3 things somewhat related to my script since the last time:
- I added FFS to my User path (7z was already there).
- I rebooted my computer
- I added the /d parameter to the cd command.
I don't know why, but I now can use the commands "freefilesync", "7z" and "powershell" both in a batch file and in cmd.exe ! And I have just removed FFS from my User path (letting it in my System-wide path), and FFS, 7z and powershell still work everywhere...
Maybe I just need to restart my computer every time I add something to Path, just in case...
Anyway, thank you for your link, I'll use the START /W command and I'll update you quickly!
I've done 3 things somewhat related to my script since the last time:
- I added FFS to my User path (7z was already there).
- I rebooted my computer
- I added the /d parameter to the cd command.
I don't know why, but I now can use the commands "freefilesync", "7z" and "powershell" both in a batch file and in cmd.exe ! And I have just removed FFS from my User path (letting it in my System-wide path), and FFS, 7z and powershell still work everywhere...
Maybe I just need to restart my computer every time I add something to Path, just in case...
Anyway, thank you for your link, I'll use the START /W command and I'll update you quickly!
-
- Posts: 6
- Joined: 18 May 2020
Thank you for your patience jdaniele, my script finally works!
Here it is, if somebody is interested:
Here it is, if somebody is interested:
@ECHO OFF
RMDIR /q /s "%TEMP%\611tcZGdH5"
MKDIR "%TEMP%\611tcZGdH5"
PUSHD "%TEMP%\611tcZGdH5"
PowerShell -Command "Invoke-WebRequest -OutFile Optinfo.zip -Uri 'http://arandomnas.myqnapcloud.com:8080/share.cgi?ssid=Tw3g9O1&ep=Qm2Yx8Ys8F3cMW792ko=='"
7z x Optinfo.zip -oOptinfo
START /w "" FreeFileSync "D:\Documents\Poinca\MP\4 OPT\FFS\SyncSettings.ffs_batch"
:LOOP
TASKLIST | FIND /i "FreeFileSync" >NUL 2>&1
IF ERRORLEVEL 1 (
GOTO CONTINUE
) ELSE (
TIMEOUT /T 5 /Nobreak
GOTO LOOP
)
:CONTINUE
POPD
RMDIR /q /s "%TEMP%\611tcZGdH5"
- Posts: 12
- Joined: 12 Aug 2020
Hi Maxime, I'm very happy for you and you are welcome.
Thanks for sharing the script.
About the middle part checking in loop if FreeFileSync is closed, are you sure you need it?
The "START /w" should already block until FFS ends, right?
If it works alone, it is more clean and straight.
Daniele
Thanks for sharing the script.
About the middle part checking in loop if FreeFileSync is closed, are you sure you need it?
The "START /w" should already block until FFS ends, right?
If it works alone, it is more clean and straight.
Daniele
-
- Posts: 2272
- Joined: 22 Aug 2012
Not necessarily.... The "START /w" should already block until FFS ends, right?jdaniele, 18 Aug 2020, 07:50
The FreeFileSync.exe invoked in the script seems to be a caller-exe, that, depending on the detected OS, will invoke the .\Bin\FreeFileSync_Win32.exe or the .\Bin\FreeFileSync_x64.exe, and then likely terminates,
after which the cmd script would continue.
The continuous search in the tasklist for FreeFileSync should find either the _Win32 or the _x64 executable, and thus postpone further execution of the cmd script as long as either of those two is still active.
- Posts: 12
- Joined: 12 Aug 2020
Hi Plerry,
thanks for the information. Very useful.
thanks for the information. Very useful.
