[RndTbl] rsync with multi-direction delete.

Sean Cody sean at tinfoilhat.ca
Mon Sep 17 15:51:06 CDT 2007


Multi-master with rsync is well... there be dragons.
At least this is a very easy way to drive yourself insane.

I speak from experience here; for another PoV ask SW the pains this  
can cause, especially once the data set gets huge and the file system  
comes really busy.
There is an 'methodology' hack which can work quite well but it isn't  
a true multi-master... think of it as a segregated master and it  
won't work in this situation.

I would really give up trying to make rsync do multi-master...  
seriously that way leads to madness and lost data (without decent  
locking there will be race conditions and difference/delta timing/ 
transaction problems).  I would suggest re-thinking your work flow to  
move towards a single master solution and just doing very periodic  
backups of the master (think revision control systems).  You could  
also try rsnapshot (check google) but that's shoe horning and still  
requires single master.

On 17-Sep-07, at 3:17 PM, Adam Thompson wrote:

> Montana Quiring wrote:
>> What I want: when I delete a file or directory on either place it  
>> will
>> remove it from the other.
>> What it does now: If I delete a directory or file from the server it
>> removes it from the USB device (ya!) but if I remove a file/directory
>> from the USB device and perform an rsync it replaces the deleted
>> file/directory.
>>
>> Here is the rsync command I'm using:
>> rsync --delete --delete-excluded --recursive -uv
>> quiringm at 192.168.1.102:/mnt/hdd/music/podcasts/ /Volumes/MEK/ 
>> Podcasts/
>>
> As Bill already suggested, rsync might not be ideal for this  
> situation, as it assumes a single-master scenario; you're looking  
> for a multi-master architecture.
>
> There is one easy way to use rsync - but you have to know in  
> advance which location has the deleted files, and name that as the  
> source.  So when sync'ing from server to USB, use the command you  
> mention above, but when syncing from USB to server, reverse the  
> last two arguments.  You see immediately that you can only make  
> changes to one location OR the other in between syncs, as any  
> changes made in one location will always be undone by the other.
>
> In fact, true two-way synchronization in the manner you're talking  
> about is still a fairly hard problem in the generic case.  It's  
> fairly easy if you have some way of keeping a log of all activity  
> to both trees, you can compare timestamps to see whose actions need  
> to be replicated where, but without any a priori knowledge, how is  
> any tool supposed to know the difference between these two scenarios:
>
>    A.  You've added a directory (and some files) to system #1, that  
> don't appear on system #2 yet.
>
>    B.  You've deleted a directory (and some files) on system #2,  
> but they remain on system #1.
>
> To a sync tool comparing systems #1 and #2, those two scenarios are  
> indistinguishable.  And, so far as I know, there's no DWIM  
> extensions to rsync yet :-)
>
> One package that *might* be able to solve your problem with some  
> extra work on your part is XFile (http://www.idiom.com/~zilla/ 
> xfiles.html).
>
> Good luck with this...
>
> -Adam
>
>
> _______________________________________________
> Roundtable mailing list
> Roundtable at muug.mb.ca
> http://www.muug.mb.ca/mailman/listinfo/roundtable

-- 
Sean




More information about the Roundtable mailing list