Feed
 

Time Machine not behaving itself?

Avatar Rick Churchill
Not Panicking Yet!

All the recent photo files and folders on my external hard drive called “LIBRARY“ have disappeared. (Everything after 23-01-2018). Lightroom copies my camera files into folders in date order and after processing I move them into a year folder i.e “2018”. Lightroom alerted me yesterday that it couldn’t find some files. Not to panic, I have them backed up with Time Machine I thought.

When I “Enter Time Machine” I am presented with a stack of Finder windows. At first, even after several attempts, I was not able to flip back through these so I gave up and examined the files manually (see below) however on returning to Time Machine it suddenly started working correctly and confirmed that these files are now missing in all backups.

I manually opened backup folders on my back-up drive named “2018-04-03…”“2018-06-05…” and the latest, “2018-07-25…”. They all have folders after 23-01-2018 missing and the folder modification date information is the same in each backup. Folder called “LIBRARY” Created 10-10-2017, Modified 23-01-2018; Folder called “2018” Created 29-12-2017, Modified 22-01-2018.

Any ideas why folders can disappear from backups made several months ago (or that for some reason they have never been backed up) and more importantly have I lost these files forever?

(I should add that my LIBRARY external drive failed and was replaced in July but I cannot see how this would affect the backup drive.)

Re: Time Machine not behaving itself?

Avatar Mick Burrell
Staring at the end, you say the drive called LIBRARY failed and was replaced in July. I assume that it's contents went with it but you copied the relevant folders from the computer to the new drive. Is that correct?

If so, did you by any chance check that the folders had contents at that time? If they did, whatever has happened has happened since. However, if they did not then you probably copied empty folders over to the new drive.

Have you checked the size of the folders to see if they have contents?

Nearer the beginning - after processing I move them into a year folder i.e '2018'. - I assume this is on your internal hard drive and hence gets copied to Time Machine. Correct?

Incidentally, looking at folders in TM should do no harm but manually moving stuff around can have unpredictable results.

Re: Time Machine not behaving itself?

Avatar Rick Churchill
I assume that it's contents went with it but you copied the relevant folders from the computer to the new drive. Is that correct?
Yes I copied the relevant folders from the backup drive to the new drive. I did it outside of Time Machine, copying Pictures then Video then Music. I can’t be sure that the missing files & folders were copied back and at that stage were included in the backup.

Size of folders- I am not sure what you mean. Those folders that exist have the relevant photo files inside, those that are missing Lightroom keeps a small jpeg in its catalogue and puts a query over the folder to say that it cannot determine where the original is and it is right because these folders are missing.

I don’t think it’s relevant but since the new external drive was attached in early July Lightroom has been copying photo images from my camera to Pictures on my internal HD. It does this when I forget to attach the external drive (or I have plugged it in and the drive is in the process of mounting) and I don’t notice the warning message. (It then continues to use this destination until changed)

Moving files and folders. No the year folder called “2018” resides on the external hard drive and I move the “day” folders into it using Lightroom. Lightroom makes the new folder which can be accessed either using the Lightroom catalogue structure or using Finder. I have used search (Spotlight?) to look for several missing files and looked in adjacent folders in case I inadvertently moved the folders but without success. I’ve never moved folders around on the backup drive.

Time Machine is set to backup both internal HD and the external drive.

Things are looking bad aren’t they?

Re: Time Machine not behaving itself?

Avatar Mick Burrell
Size of folders - either in the finder display if you show size there or use a right click on one and choose Get Info. You'll be able to tell if it's really empty or has some photos you can't see.

How did you copy from the backup drive if it died?

Re: Time Machine not behaving itself?

Avatar Rick Churchill
Thanks Mike. Taking the last point first. I have not made my set-up clear.

I have an external 2 TByte drive called LIBRARY on which I keep all my photos, video and music, the on-board SSD storage of 256GBytes being insufficient. (When my delaminating screen is sorted I intend to replace the SSD with a 1 TByte sized module).

Separately I have a 2 TByte Back-up external drive onto which are copied using Time Machine the files from both my internal HD and from the LIBRARY drive.

So when my LIBRARY external drive failed I was able to connect a replacement drive and name it LIBRARY and then copy, using Finder, the files back from the latest backup. This drive has just over 1 Tbyte used (with the reduced number of files)

I cannot look at the size of the missing folders as they no longer are showing in Finder. Only my Lightroom catalogue points to where they use to be. Looking at the size of the 2018 folder in LIBRARY it is showing only 1.5 GBytes and it should be much more (2017 is 127 GBytes)

(Possibly my old drive has the missing files on its platter but it will no longer mount.)

Re: Time Machine not behaving itself?

Avatar Mick Burrell
I think I've found the issue as long as what I say about TM below is accurate. I think it is but can't confirm. When you say

So when my LIBRARY external drive failed I was able to connect a replacement drive and name it LIBRARY and then copy, using Finder, the files back from the latest backup.

I have to assume you mean you copied them back from the latest backup i.e. Time Machine, as the latest backup to the original LIBRARY drive would be unobtainable as the drive is dead.

Time Machine is a complex database with internal links to files from the past. It does not make loads of duplicates of unaltered files. Therefore if you look (using Finder) at Folder A from the latest backup, it may only actually include files which have altered since the previous backup. If none have changed it could even be empty! If you look at it in TM itself, it should show you, using its internal links, all the files that currently reside in the "live" version.

If this is what you did, as your photos don't alter every day, you may well have copied over empty or nearly empty folders. Too late now of course but if you'd used TM to restore the folders rather than Finder, you may have been able to restore them complete to the new drive.

As I said, I can't confirm this is exactly how TM works but I think it does and it may offer a solution as to why the folders on LIBRARY are now empty. I've no idea why those folders are empty on TM though.

Re: Time Machine not behaving itself?

Avatar Rick Churchill
Interesting. Yes Time Machine ought to be an incremental backup process as it would fill up my 2 TByte drive quickly but if that were true then surely only new files or ones that I have modified would have been included in the copying process but ALL files including those I made in in 2016 and 2017 appear on my new drive. Only those after 23rd January 2018 are missing.

Using Finder on files that are in Lightroom I find that a folder 2018-01-04 in Lightroom contains 9 photos and using Finder in the latest backup 9 photos and in the 1st May backup 9 again but in the 1st March 11 photos. (This was probably because between March and May I threw 2 away). i.e. the photo files appear to be in every backup! (At least finder is showing them to be, they are not shown as aliases)

The back up drive still contains early backups and I have looked at the one for the 1st March 2018. This backup ought to have included my skiing holiday photos at the end of February but does not.

Curiouser and curiouser.

(I wonder if there's a chance that a computer shop can extract data from my (crashed?) old LIBRARY drive.)

Re: Time Machine not behaving itself?

Avatar Mick Burrell
You need a data specialist to extract data. I think these guys are in Portsmouth:

Data Recovery

They may be the company who came to talk to the Fareham group some time ago. John Hooper would know.

Re: Time Machine not behaving itself?

Avatar Derek Wright
See Meeting log of the 27th April 2013

Meeting log of the 13 April 2013
when we had a speaker from DataQuest International.

Re: Time Machine not behaving itself?

Avatar Rick Churchill
Thanks Mike and Derek. I am in a quandary.

Laptop Fixers in Broadstone could attempt a data recovery for £60 but they would probably prize the case open and remove the drive and plug it into an external drive enclosure which is something I can do if it has a SATA interface. If it is the interface that is faulty then the data is recoverable but if the disc itself is faulty they and I will recover nothing.

PC World will do the same for £150. They may be doing more and £150 is less than the £300 DataQuest do but I am reluctant to spend that money.

Otherwise PC World would send the drive back to the manufacturer for a £75 refund but the case has to be intact else the warranty is void even though I can prove to them that the drive is broken. (It's a manufacturers warranty not their own)

I think I might, when I have plucked up courage go in with a screwdriver!

Incidentally I have been exploring the video and music files and there is nothing after January 2018 on the new LIBRARY disc or the Backup Disc. It looks as if Time Machine may have stopped copying files after this date. When did we change to High Sierra?

Re: Time Machine not behaving itself?

Avatar Mick Burrell
It's unlikely to be an error on the platter as some would still be readable - it's most likely the mechanism which means if there is data on the platter (and I doubt you can be sure of that) then it will be recoverable but a clean room is advisable. I don't mean to imply your study is dirty but it needs to be totally dust free for best results.

Footnote to anyone who's read this far:
How often should you check your backup system is working as you expect it to? Answers on a post card please 😉 (Sorry Rick - couldn't resist!)

Re: Time Machine not behaving itself?

Avatar Rick Churchill
AWTEW - Alls Well That Ends Well!

Thanks Mike.

I bit the bullet and removed the drive from it's case. See this excellent Youtube video on how to disassemble and repair an external drive.

Interestingly the drive proper has a sticker covering all the screw heads with a warning that the warranty is void if the label is removed. I wonder if that just removing it from it's case I haven't invalidated the warranty! I had to cut through some plastic film in order to separate the drive from it's interface that plugs into one end but that's all.
Anyway with the SATA pins exposed I plugged this into a Probox External enclosure Similar to this one except mine is only a USB2.0 type. and connected it to the computer and it mounted!

I have just copied 69 GBytes of missing Picture files and 10 GBytes of missing Video files across and am contemplating how to do the music which is scattered across many folders. I might delete the whole music folder and copy everything across.

It will be interesting to see if Time Machine backs up everything.

Your comment on clean room although valid the comment made by the guy in the link above is that the disc spins at 5700(?) and flings off dust particles into the built-in filter so unless the dust is sticky it will be OK in a normal clean room.

Incidentally is there a Mac programme that will inspect every disc segment and check there is no damage? I suppose I could reformat and I think the software then detects and avoids writing to damaged segments. Reduced capacity would show that there is damage.

Re: Time Machine not behaving itself?

Avatar Derek Wright
The need for a clean rooom when messing with the internals of disk drives is to reduce the chance of contaminating the platters and heads with domestic waste eg skin particles, dust etc. Anything that could interrupt the flight of the heads over the spinning platters can cause head crashes which dammage (destroy) the heads and gouge out the surface of the platter.

Re a Mac program to check your drive - have a look at what the"First Aid" function in the Disk Utility app can do - if this is not sufficient then ask Google for Mac based disk health check programs. Techtools is a program that comes to mind.

Re: Time Machine not behaving itself?

Avatar Rick Churchill
Yes I understand that particles on the disc platter can cause the head to destroy the disc surface and that a clean room is best but most of us don't have the facility and making a box with dust extraction, glove access and a plate to view the work is a lot of bother although it can be done. However the YouTube article points out that the disc spins so fast that specs of dust will be flung clear and will be trapped by the small filter at the edge of the platform long before the head moves from it's rest position onto the platter. (It goes without saying that the disc surface should never be touched)

I would also add the need to earth oneself to avoid static build-up. My very crude method (and I do not recommend this) is to place one hand on an earthed radiator frequently. It would be better to buy an anti-static wrist strap.

Thanks for your suggestions as to a disc checking app.

Re: Time Machine not behaving itself?

Avatar Rick Churchill
TIME MACHINE MISBEHAVIOUR SOLVED!!!! (“Rabbit of much euphoria” or “Happy bunny”)

I copied about 100Gbytes of files back on to my external drive LIBRARY from the previous faulty one so I was surprised when the latest backup was 17bytes! The reason was that it was not backing up the external drive at all. When I checked the exclusion list in Options I found it there. The only reason I can think of that no files after 23rd January have been backed up is that on that date the external drive appeared on the exclusion list because the drive was possibly renamed and the OS did not update TM as to the new name.

Anyway, now that I have it included and possibly because after 23rd January I renamed the Pictures folder to PICTURES it has taken over 3 hours to “clean up old backups” and has made 633 Gbytes of free space. Unfortunately, TM has decided that my 1.45Tbytes of external drive and 190Gbytes of on-board SSD storage is too much for the 2Tbyte drive and has given up with the messsge “Backup Failed. There is not enough space on the xxx drive”.

I’ll open a new thread to discuss storage solutions.
 
Feed