FFS Consistently hangs on Synchronize

[n4af]

(Moving from Open Discussion)
Have tried maybe 20 times. Always hangs. Usually the screen clears indicating
heavy i/o, but my event monitor does not show heavy i/o or memory usage--maybe
tight loop ??
Update--just tried on ext3 file and ran ok.
EXT4 solid failure. adding NODELALLOC to mount d/n make a difference.
Ubuntu10.04, ffs
[404, Invalid URL: http://blountscreek.org/hang3.png]

[Zenju]

Hi n4af,

the copying routine is pretty much standard, so there is not that much that
can go wrong. So let's narrow this issue down:

- Have you tried older versions of the tool and if so, do they exhibit the same problem?

- Could you test whether it's related to recycle bin usage?

- The file access routine is located in "shared/fileIO.cpp":
Could you replace
::fopen(filename.c_str(), "rb,type=record,noseek");
by
::fopen(filename.c_str(), "rb");
compile again and see if it makes a difference? Maybe the "type=record,noseek"
makes problems on some FS.

Regards, Zenju

[n4af]

Thank you for the quck response Zenju.

I have recompiled, removing type=record,noseek, so the line reads
::fopen(filename.c_str(), "rb"; ----------------- by commenting out the type
ending.
Because I left the semicolon off, the first compile failed- so I know it is
reading fileIO.cpp

The failure remained the same:
[404, Invalid URL: http://blountscreek.org/hang5.png]

I have not tried earlier versions but did try writing the output to my home
directory instead of USB hard drive- with the same
results.

Thanks, Howard

[Zenju]

That was a rough shot. In order to fix this problem I need to know exactly
what statement causes this hang. The copying routine is located in file
"shared/fileHandling.cpp" in function "FreeFileSync::copyFile()" right below
"#elif defined FFS_LINUX" approximately at row 1720.
Can you add some logging information just to see how far the program
progresses until it stops?
Just insert a statement like

std::cout << __FILE__ << " -> " << __LINE__ << std::endl;

every couple of rows. When the tool starts to copy a file it will emit the row
in the sourcecode it processes.

Thanks, Zenju

[n4af]

Sorry, I am probably doing this wrong:


std::cout << __FILE__ << " -> " << __LINE__ << std::endl;
produces
shared/fileHandling.cpp:1474: error: ���cout��� is not a member of ���std���
shared/fileHandling.cpp:1480: error: ���cout��� is not a member of ���std���
shared/fileHandling.cpp:1492: error: ���cout��� is not a member of ���std���
shared/fileHandling.cpp:1501: error: ���cout��� is not a member of ���std���
make: *** Error 1

[Zenju]

Okay, seems a include is missing. I've tested this in debug build, where
#include <iostream>
is already present. Adding this line at the top of the file should do.

[n4af]

Clean compile. Will test shortly.

Tks, Howard

[n4af]

Was not totally sure where to put the trace points, but here is the source:

void FreeFileSync::copyFile(const Zstring& sourceFile,
const Zstring& targetFile,
bool copyFileSymLinks,
CopyFileCallback* callback)
{
using FreeFileSync::CopyFileCallback;
std::cout << __FILE__ << " -> " << __LINE__ << std::endl; <<< 1475

//symbolic link handling
if ( copyFileSymLinks &&
symlinkExists(sourceFile))
return copySymlinkInternal(sourceFile, targetFile); //throw (FileError)
std::cout << __FILE__ << " -> " << __LINE__ << std::endl;

//begin of regular file copy
struct stat fileInfo;
if (::stat(sourceFile.c_str(), &fileInfo) != 0) //read file attributes from
source file (resolving symlinks)
{
const wxString errorMessage = wxString(_("Error reading file attributes:")) +
wxT("\n\"") + zToWx(sourceFile) + wxT("\"");
throw FileError(errorMessage + wxT("\n\n") +
FreeFileSync::getLastErrorFormatted());
}

//open sourceFile for reading
FileInput fileIn(sourceFile); //throw FileError()
std::cout << __FILE__ << " -> " << __LINE__ << std::endl;

//create targetFile and open it for writing
const Zstring temporary = createTempName(targetFile); //use temporary file
until a correct date has been set

//ensure cleanup (e.g. network drop): call BEFORE creating fileOut object!
Loki::ScopeGuard guardTempFile = Loki::MakeGuard(::unlink, temporary);

FileOutput fileOut(temporary); //throw FileError()
std::cout << __FILE__ << " -> " << __LINE__ << std::endl; <<<#1502

---------------------
what i get is rippling thru the added line #. it ends with the last
(unfortunate I d/n catch line #):

shared/fileHandling.cpp -> 1475
shared/fileHandling.cpp -> 1481
shared/fileHandling.cpp -> 1493
shared/fileHandling.cpp -> 1502

This ripples thru the entire output, as you would expect. It ends with the
last point, i chose:
shared/fileHandling.cpp -> 1475
shared/fileHandling.cpp -> 1481
shared/fileHandling.cpp -> 1493
shared/fileHandling.cpp -> 1502
Terminated

Perhaps you would like me to refine this ?

Tks, Howard

[Zenju]

> Was not totally sure where to put the trace points
Just litter the sourcecode with these statements. The important thing is to
detect the line where it hangs. Hopefully this is always the same line and I
get some clue about what may cause issues.

>Perhaps you would like me to refine this ?
Yes

Just to be sure this issue is related to FreeFileSync and not a general
problem: Copying manually via "cp" or your file browser works correctly,
right?

[n4af]

Correct. I have done this same backup with Clonezilla, FSarchiver, Lucky
Backup, and Rdiff-Backup.

Using FreeFileSync, the sync works ok --USING EXT3.on Mepis 8.5 partition.
Consistently fails trying to sync ext4 partion-- Ubuntu Lucid or Linux Mint 9.

Tks, Howard

[n4af]

Hi Zenju -

CORRECTION. The problem is not as global as Ext4;
I just backed up my home directory on Ubuntu 10.04 w/ no problems (Ext4).

Normally, the procedure here is to back up the entire partition that is not
booted, but just mounted r/o.
Most of my testing has been against another ext4 partition and I am wondering
if the problem is not specific files.

Will do some more checking from this end.

Thanks, Howard

[Zenju]

Okay, thanks a lot for your efforts! Just keep me updated when you find the
row/part where it hangs. Currently I can't think of parts of the copy routine
that depend on something other than on what usual copy routines depend. But
you're always smarter in the end.

[n4af]

Something is pretty flakey. My problem seems to be with items in /lib/udev-
like Iphone-set-info and Console.
I took /lib/udev out of the mix (exclude) and the sync ran (literally) to
99.99% w/ 5sec to go. The file it failed on was legit. I even cp it over to
make sure. So still scratching my head.

I may open a separate incident to ask if you have come across a compile error
I hit on mepis. using Compile.txt-
gtk seems to compile ok.
but I fail on return to make within the ffs directory. Something about config
not found in /usr/bin/lib/wx/config or somesuch.

Thanks, Howard

[Zenju]

Perhaps you can distill a single file that causes this hang. Then we can
further analyze the boundary conditions, e.g. test whether it hangs if the
file is located on another drive, etc. Perhaps it's even possible to reproduce
this error on my Ubuntu Linux. In this case I could debug this, which is of
couse more efficient than emitting log texts to the console via std::cout.

[n4af]

Hi Zenju -

I am having problems with the hang destroying my USB drive MFT. I do not think
it has anything to do with FFS, but is keeping me from doing anything to
recreate right now.

Tks, Howard

[Zenju]

I understand. Nevertheless if you have something I can work with at some
point, don't hesitate to inform me.

Best regards