[unison-users] unison refuses to sync a certain single file

Discussion:

'joerg van den hoff' j.van_den_hoff@hzdr.de [unison-users]

2017-07-06 21:04:01 UTC

hi everybody,

I'm a happy unison user for many years using it for sync between a desktop
and a labtop, both running macos.

today I have changed the setup to a 'star topology' with a linux server at
the "hub" in order to sync desktop mac to linux server at work and linux
server to labtop mac at home using essentially the same profile as before
for syncing the mac machines directly.

I have encountered so far the following unusual and unexpected hickups:

* quite a number of spurious/apparent content or props changes when
syncing between desktop and server and then immediately between server and
labtop (i.e. changes in files that where not modified by me in
between...). it's only a couple but still... this I have to investigate a
bit further, though.

* more irritating: there is a single ancient file that refused to sync
from desktop to the pristine new root on the linux server. I tried it
repeatedly but always got the same error message (when executing unison on
the desktop mac):

Looking for changes
Waiting for changes from server
Reconciling changes

local remoteHost
new file ====> new file path/to/file [] .

Proceed with propagating updates? [] y
Propagating updates

UNISON 2.48.4 started propagating changes at 18:07:31.90 on 06 Jul 2017
[BGN] Updating file path/to/file from /path/to/local/root to
//remoteHost//path/to/remote/root
Failed: Error in renaming
/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp
to
/path/to/remote/root/.unison.fileName-bad.101a027f249072fb2349bf25e6616da4.unison.tmp:
No such file or directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]
100% 00:00 ETAFailed [path/to/file]: Error in renaming
/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp
to
/path/to/remote/root/.unison.fileName-bad.101a027f249072fb2349bf25e6616da4.unison.tmp:
No such file or directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]
UNISON 2.48.4 finished propagating changes at 18:07:32.10 on 06 Jul 2017

on the linux (ubuntu) server 2.48.3 is running.

the problem only went away after copying the file manually to the new
root, deleting it locally and than syncing again (i.e. the copy from
remoteHost to desktop than worked). question: any pointer/idea what might
have caused this? the proximal cause obviously is "No such file or
directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]"
but why??

thank you
joerg

ps: thanks a lot for unison!

------------------------------------

------------------------------------

------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/unison-users/join
(Yahoo! ID required)

<*> To change settings via email:
unison-users-***@yahoogroups.com
unison-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
unison-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Adrian Klaver adrian.klaver@aklaver.com [unison-users]

2017-07-06 21:51:41 UTC

Permalink

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
hi everybody,
I'm a happy unison user for many years using it for sync between a desktop
and a labtop, both running macos.
today I have changed the setup to a 'star topology' with a linux server at
the "hub" in order to sync desktop mac to linux server at work and linux
server to labtop mac at home using essentially the same profile as before
for syncing the mac machines directly.
* quite a number of spurious/apparent content or props changes when
syncing between desktop and server and then immediately between server and
labtop (i.e. changes in files that where not modified by me in
between...). it's only a couple but still... this I have to investigate a
bit further, though.

I see those also occasionally when doing something similar. Generally
it's down to differences in file permissions and/or file
ownership(user.group) between OS'es or distros.

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
* more irritating: there is a single ancient file that refused to sync
from desktop to the pristine new root on the linux server. I tried it
repeatedly but always got the same error message (when executing unison on
Looking for changes
Waiting for changes from server
Reconciling changes
local remoteHost
new file ====> new file path/to/file [] .
Proceed with propagating updates? [] y
Propagating updates

Best guess is the below is due to permissions issues on the directory.

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
UNISON 2.48.4 started propagating changes at 18:07:31.90 on 06 Jul 2017
[BGN] Updating file path/to/file from /path/to/local/root to
//remoteHost//path/to/remote/root
Failed: Error in renaming
/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp
to
No such file or directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]
100% 00:00 ETAFailed [path/to/file]: Error in renaming
/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp
to
No such file or directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]
UNISON 2.48.4 finished propagating changes at 18:07:32.10 on 06 Jul 2017
on the linux (ubuntu) server 2.48.3 is running.
the problem only went away after copying the file manually to the new
root, deleting it locally and than syncing again (i.e. the copy from
remoteHost to desktop than worked). question: any pointer/idea what might
have caused this? the proximal cause obviously is "No such file or
directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]"
but why??
thank you
joerg
ps: thanks a lot for unison!

--
Adrian Klaver
***@aklaver.com

------------------------------------

------------------------------------

------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/unison-users/join
(Yahoo! ID required)

<*> To change settings via email:
unison-users-***@yahoogroups.com
unison-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
unison-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

'joerg van den hoff' j.van_den_hoff@hzdr.de [unison-users]

2017-07-06 22:08:28 UTC

Permalink

On Thu, 06 Jul 2017 23:51:41 +0200, Adrian Klaver

Post by Adrian Klaver ***@aklaver.com [unison-users]

thanks for the quick response...

Post by Adrian Klaver ***@aklaver.com [unison-users]
I see those also occasionally when doing something similar. Generally

that's somewhat reassuring...

Post by Adrian Klaver ***@aklaver.com [unison-users]
it's down to differences in file permissions and/or file
ownership(user.group) between OS'es or distros.

but this, then, should show up only as "props" changes rather than content
changes, no?

Post by Adrian Klaver ***@aklaver.com [unison-users]

Best guess is the below is due to permissions issues on the directory.

you mean on the remote root? that's hardly possible since I've created it
on the the remote machine from within my user account and the bulk of the
sync (the other 10^5 files ;-)) just synced fine? what sort of permission
issues could this be that affect only that single file (which is readable
locally of course as well...)?

Post by Adrian Klaver ***@aklaver.com [unison-users]

------------------------------------

------------------------------------

------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/unison-users/join
(Yahoo! ID required)

<*> To change settings via email:
unison-users-***@yahoogroups.com
unison-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
unison-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Adrian Klaver adrian.klaver@aklaver.com [unison-users]

2017-07-06 23:11:13 UTC

Permalink

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
On Thu, 06 Jul 2017 23:51:41 +0200, Adrian Klaver

Post by Adrian Klaver ***@aklaver.com [unison-users]

thanks for the quick response...

Post by Adrian Klaver ***@aklaver.com [unison-users]
I see those also occasionally when doing something similar. Generally

that's somewhat reassuring...

Post by Adrian Klaver ***@aklaver.com [unison-users]
it's down to differences in file permissions and/or file
ownership(user.group) between OS'es or distros.

but this, then, should show up only as "props" changes rather than
content changes, no?

Missed the part about content changes.

Are they text files where you could use the diff in the GUI?

What type of files are they?

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]

Post by Adrian Klaver ***@aklaver.com [unison-users]

Best guess is the below is due to permissions issues on the directory.

you mean on the remote root? that's hardly possible since I've created
it on the the remote machine from within my user account and the bulk of
the sync (the other 10^5 files ;-)) just synced fine? what sort of
permission issues could this be that affect only that single file (which
is readable locally of course as well...)?

But did the file have the permissions to be in that directory on the
remote machine?

What is the file name?
--
Adrian Klaver
***@aklaver.com

------------------------------------

------------------------------------

------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/unison-users/join
(Yahoo! ID required)

<*> To change settings via email:
unison-users-***@yahoogroups.com
unison-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
unison-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

'joerg van den hoff' j.van_den_hoff@hzdr.de [unison-users]

2017-07-07 06:40:27 UTC

Permalink

On Fri, 07 Jul 2017 01:11:13 +0200, Adrian Klaver

Post by Adrian Klaver ***@aklaver.com [unison-users]

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
On Thu, 06 Jul 2017 23:51:41 +0200, Adrian Klaver

Post by Adrian Klaver ***@aklaver.com [unison-users]

thanks for the quick response...

Post by Adrian Klaver ***@aklaver.com [unison-users]
I see those also occasionally when doing something similar. Generally

that's somewhat reassuring...

Post by Adrian Klaver ***@aklaver.com [unison-users]
it's down to differences in file permissions and/or file
ownership(user.group) between OS'es or distros.

but this, then, should show up only as "props" changes rather than
content changes, no?

Missed the part about content changes.
Are they text files where you could use the diff in the GUI?

right now I'm at the labtop and have executed a sync which did _not_ show
up the spurious changed/new files. as I recall from yesterday evening it
was stuff like

.dbus-keyrings/org_freedesktop_general

which was reported as "new" on both sides and

.rnd

(a binary file whose origin I don't know -- it's binary, names sounds like
it being a seed of some random number?) which was reported as changed
IIRC... there were also a few (untouched) text files where the diff, then,
did not show any printable characters (maybe presumed white space
changes?). sorry for not being able to be more specific. I realize that no
sensible advice can be given in such a situation. I will keep an eye on
this if it pops up again.

Post by Adrian Klaver ***@aklaver.com [unison-users]
What type of files are they?

see above. a couple of system or application-side generated files, a
couple of my own files.

Post by Adrian Klaver ***@aklaver.com [unison-users]

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]

Post by Adrian Klaver ***@aklaver.com [unison-users]

Best guess is the below is due to permissions issues on the directory.

you mean on the remote root? that's hardly possible since I've created
it on the the remote machine from within my user account and the bulk
of the sync (the other 10^5 files ;-)) just synced fine? what sort of
permission issues could this be that affect only that single file
(which is readable locally of course as well...)?

But did the file have the permissions to be in that directory on the
remote machine?

yes. the procedure was:

1. generate pristine root dir on server within my user-account's home dir
tree on the server (same user name, sane umask etc)
2. run `unison -force {root on desktopmap (my home dir)} to
{new_empty_root_on_server}

the run went through with some 40 GB and complaint about that single file
and did so repeatedly when re-executing the `unison' run. with or without
`-force'.

Post by Adrian Klaver ***@aklaver.com [unison-users]
What is the file name?

the basename is `3frames.mov' right beside a file `3frames.png' (stuff
within a directory containing an ancient unzipped keynote presentation)
which was transfered just fine...

thank you

joerg
------------------------------------

------------------------------------

------------------------------------

Yahoo Groups Links

<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
Individual Email | Traditional

<*> To change settings online go to:
http://groups.yahoo.com/group/unison-users/join
(Yahoo! ID required)

<*> To change settings via email:
unison-users-***@yahoogroups.com
unison-users-***@yahoogroups.com

<*> To unsubscribe from this group, send an email to:
unison-users-***@yahoogroups.com

<*> Your use of Yahoo Groups is subject to:
https://info.yahoo.com/legal/us/yahoo/utos/terms/

Adrian Klaver adrian.klaver@aklaver.com [unison-users]

2017-07-07 14:39:45 UTC

Permalink

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
On Fri, 07 Jul 2017 01:11:13 +0200, Adrian Klaver

Post by Adrian Klaver ***@aklaver.com [unison-users]
Missed the part about content changes.
Are they text files where you could use the diff in the GUI?

right now I'm at the labtop and have executed a sync which did _not_
show up the spurious changed/new files. as I recall from yesterday
evening it was stuff like
.dbus-keyrings/org_freedesktop_general
which was reported as "new" on both sides and
.rnd
(a binary file whose origin I don't know -- it's binary, names sounds
like it being a seed of some random number?) which was reported as
changed IIRC... there were also a few (untouched) text files where the
diff, then, did not show any printable characters (maybe presumed white
space changes?). sorry for not being able to be more specific. I realize
that no sensible advice can be given in such a situation. I will keep an
eye on this if it pops up again.

Yeah, that is not all that unusual. I see that with applications that
are chatty with their support files. You do different things on
different machines and the support files diverge.

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]

Post by Adrian Klaver ***@aklaver.com [unison-users]
But did the file have the permissions to be in that directory on the
remote machine?

1. generate pristine root dir on server within my user-account's home
dir tree on the server (same user name, sane umask etc)
2. run `unison -force {root on desktopmap (my home dir)} to
{new_empty_root_on_server}
the run went through with some 40 GB and complaint about that single
file and did so repeatedly when re-executing the `unison' run. with or
without `-force'.

Post by Adrian Klaver ***@aklaver.com [unison-users]
What is the file name?

the basename is `3frames.mov' right beside a file `3frames.png' (stuff
within a directory containing an ancient unzipped keynote presentation)
which was transfered just fine...

Hmm. In your original post there was:

Failed: Error in renaming
/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp
to
/path/to/remote/root/.unison.fileName-bad.101a027f249072fb2349bf25e6616da4.unison.tmp:

A quick search of the source seems this has do with with encoding
issues. Might want to try with:

http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-manual.html#prefs

-unicode false

Or try renaming the file.

Post by 'joerg van den hoff' ***@hzdr.de [unison-users]
thank you
joerg

Nick Demou ndemou@enlogic.gr [unison-users]

2017-07-07 11:18:41 UTC

Permalink

[...] question: any pointer/idea what might
have caused this? the proximal cause obviously is "No such file or
directory
[rename(/path/to/remote/root/.unison.fileName.101a027f249072fb2349bf25e6616da4.unison.tmp)]"
but why??

I'm replying from vague memory and this may not be the case for you but
I'm pretty sure I was getting this error when I was accidentally trying
to sync the same directory twice in the same profile. E.g. something
like this:

# only sync these sub-directories
path = sub/path
path = sub/path/deeper

If a file got changed in /some/path/sub/path/deeper I was getting the
above error but I spotted my silly configuration eventually.

'joerg van den hoff' j.van_den_hoff@hzdr.de [unison-users]

2017-07-11 16:19:58 UTC

Permalink

Post by Nick Demou ***@enlogic.gr [unison-users]

I'm replying from vague memory and this may not be the case for you but
I'm pretty sure I was getting this error when I was accidentally trying
to sync the same directory twice in the same profile. E.g. something
# only sync these sub-directories
path = sub/path
path = sub/path/deeper
If a file got changed in /some/path/sub/path/deeper I was getting the
above error but I spotted my silly configuration eventually.

well, that's probably not what's happening here: my profile is for a full
sync of the roots except a list of explicit `ignore = ' and `ignorenot ='
directives....

I am still trying to get this sorted out, though.