Restoring file times

Do you remember this? Well, now I have a similar but quite different case: I want to restore the files' times. Why? Read me out.

Since a long time I'm doing backups to an external drive. For that I simply use rsync. When I bought my second laptop, I took the chance to test restoring from that backup, so I just did a last backup in the old machine and a restore in the new one. This allowed me to add the last few things that were missing. After a few more weeks finding missing things, doing more backup/restore cycles between the old and new laptops, I was happy.

But there was something that I didn't realize till a few months later. The file times were not backed up, and clearly not restored. This didn't affect me in any way, at least until I tried to a new post to this glob.

See, this glob is generated with ikiwiki. ikiwiki uses the files' mtime to generate the posts' times, and uses that to sort them and to assign them as belonging to some year or month. With the mtimes lost, ikiwiki was assigning all the older posts to some day in Oct, 2012, which was the date of the last backup/restore cycle.

Frustrated (yes, sometimes I have a low threshold), I left it like that: no planet was flooded because in any case the rss feeds didn't really change, and nobody would notice and complain about it. In fact, nobody did.¹ Have in mid that as I kept my old laptop as a server, and that I didn't clean up all the files I keep in my backup, I still had the original mtimes in the original files.

Yesterday, playing around with unrelated things, I came across the post I mentioned at the beginning of this one, and had the urge to really fix that mess. Today I started looking at it, and found that this command was some part of it:

find src/projects/glob/work/ -name *.mdwn | xargs stat --format '%X %Y %Z %n'

but processing its output in bash seemed like a burden. Luckily, I have the solution for that, a project that I started for exactly that same reason: bash sucks at manipulating data.

So I grabbed my editor, punched keys for some 7 minutes, run 3 or 4 tests, making sure that everything would go smooth, run once more on the live system, rebuilt my glob² and voilà! It's fixed now.

How long is the code? Not much really:

from os import utime

with remote ('192.168.0.42', allow_agent=False, password='XXXXXXXX') as fds:
    # find al files (not only posts), get its times and path
    with cd ('src/projects/glob/work'):
        find ('.', '-name', '*.mdwn') | xargs ('stat', format='%X %Y %Z %n')

(i, o, e)= fds

for line in e.readlines ():
    print (line.decode ('utf-8'), end='')

for line in o.readlines ():
    data= line.split ()
    access, modif, change= [ int (x) for x in data[:3] ]
    file_path= data[3].decode ('utf-8')
    if _e (file_path):
        utime (file_path, times= (access, modif))
    else:
        print ("%s not found!" % file_path)

Notice that this example is even more complex than initially thought: it ssh's into the server to fetch the original times.

This makes one thing clear: I have to improve a lot the remote() API. For instance, so far I have no way to know if the remote commands executed fine or not, and the way to read the lines is still ugly and not homogeneous with how other ayrton functions/executables work.

Also note that this does not restore the ctimes, as I hadn't found a way to do it, but didn't really did a lot of effort. I don't really need them.

Of course, I also fixed my backup and restore scripts :)