7

It was brought up today about the customers right to have their data deleted and went on to an interesting talk about backed up data (we have a rolling 93 day window for backups on AWS s3)

I was wondering if/how anyone out there goes about deleting customer data within backups? It would seem that this data protection act covers backed up data too?

How do you go about this in situations such as mine where we have a 73GB nightly backup file created everyday (expanding to 589GB data and 117GB log files) so in theory if this is fully enforceable and includes updates then we'd need to restore 93 backups it'd take:

Restore backup - 3 hours
Delete customer data - 1 minute - 2 hours (dependant on usage)
Backup 50 minutes

(I appreciate that even although this is a large database to me, working in a small company, this is still small in comparison to enterprises!)

So if we made an application automatically do this, it'd take a minimum of [4 hours per backup] * 93 = 372 hours (15 and a half days!) of processing (on a separate server, so we don't affect our live system)

Luckily we haven't had a request like this yet, but my other concern over this is, if the person writing the script to delete the data accidentally deleted part of someone elses data, we'd now have no backup to fall back on! Surely this would go against your SLA of backups?

I look forward to hearing people's views and any evidence of law on this?

| improve this question | |
  • 1
    By "Data Protection Act 1998", presumably you are referring to the UK DPA? – MrWhite Mar 3 '17 at 10:00
  • I don't know the law, but as for how to do it, even if you can, it would depend on your backup method. Backup (and restoring etc) using rsync is different to r1soft is different to everything else. – Steve Mar 3 '17 at 10:30
  • 1
    @w3dk yes I meant the UK DPA – Liam Wheldon Mar 3 '17 at 10:50
  • @Steve We create nightly MSSQL backups, I've not even included transaction logs into this as they're only held for 7 days anyway so would fit within a reasonable window. – Liam Wheldon Mar 3 '17 at 10:51
4

Morgan Lewis is a law firm with offices in the UK and they have published information (as of 2012 which is the most recent authoritative publication I can find on the subject) on the ICO's guidance on deleting personal data under the DPA 1998.

According to their legal assessment of the guidance the ICO recognized the difficulty in deleting electronic data under the act as it can still exist in the organisations systems in one form or another (backup records would seem to apply here) and as such they have adopted what they refer to as a "realistic approach" towards the deletion of electronic data on the basis that it is possible to put the data "beyond use" without actually deleting every last trace of the data. The article states that the key findings of the ICO are...

  • Where information has been deleted, but where it still exists in the "electronic ether", such data will not be "live data", and therefore data protection compliance issues will not apply to the data, as long as the data controller does not intend to use or access the data again. The ICO draws an analogy with a bag of shredded paper files-it would be possible to reconstitute the information from the shredded paper, but it would be extremely difficult, and it is unlikely that the organisation would have any intention of doing so.

  • It is possible for a data controller to put undeleted data "beyond use" if the data controller is not able, or will not attempt, to use the personal data to inform any decision in respect of any individual or in a manner that affects the individual in any way, does not give any other organisation access to the personal data, puts appropriate security measures in place in relation to the data, and commits to permanent deletion of the information if and when it becomes possible.

Based on the above one option that would not require the extraction, decompression, then recompression and storage of a large number of backup archives would be to add some form of data source separate to the existing backup and restoration systems where an index of people who have opted to have their data deleted are recorded. Then if a restoration of backed up data needs to be done after the restoration has been completed the index can be loaded and the records gone through to see if anyone listed in the index has had their personal information restored through backups, and then have it deleted as needed. Given the fact that you state that you have not had to deal with this to date an easier option (given the low chance of restoring data where someone has opted to be deleted) would be to maintain hardcopy records of deletion requests with the minimal amount of data needed to identify the record needing to be deleted and establish a business policy where one of the steps taken after data restoration is to compare these records to the restored database and see if a record which is meant to have been deleted has been restored.

This would work based on my reading of the Act and my reading of the supporting article by the Morgan Lewis law firm and would be unlikely to cause a major hassle being even a manual process as the number of records that would be requested to be deleted prior to the recommended 6 year auto destruction of data timeline would be low to begin with, and when taken with the rarity of needing to restore complete copies of databases from backup archives would wind up reducing it to an exceedingly small level whereby the process could be achieved in a very small amount of time by a pre-defined user going through and searching for the requested data based on the deletion index list to ensure that if it has been restored it is then deleted manually again. Based on what you have stated this would mean that only data deleted in the last 93 days would likely need to be manually deleted again (even less chance as it is a higher chance of restoring a more recent backup) which would present minimal manpower requirements and ensure compliance with the act.

| improve this answer | |
  • Hi Chris, thank you so much for your in depth response to my query! It's way beyond the sort of response I expected and coincides with my thoughts about backups never being altered. I've passed this on to the rest of the management team as it's such a vague area. – Liam Wheldon Mar 3 '17 at 13:59
  • I've actually just come across the article while looking things up on this "sods law" Here's the article for everyone else's reference lexology.com/library/… Thank you again Chris for your response! – Liam Wheldon Mar 3 '17 at 14:07
1

You shouldn't really receive requests to delete personal data if you are following good practices in safeguarding people's data and not storing data longer than you need to.

There is no law within the UK how long you can store personal data for but it does say you should only ever store it as long as you need to.

Source

Personal data processed for any purpose or purposes shall not be kept for longer than is necessary for that purpose or those purposes.

This is the fifth data protection principle. In practice, it means that you will need to:

  • review the length of time you keep personal data;
  • consider the purpose or purposes you hold the information for in deciding whether (and for how long) to retain it;
  • securely delete information that is no longer needed for this purpose or these purposes;
  • and update, archive or securely delete information if it goes out of date.

A good practice would be to allow your customers to select how long that retention is, or allow them to delete the data themselves, you could also have a undelete protocol that holds the data for 30days after deletion at which point unless cancelled purges the data into the abyss.

This way you have no concern how long you store data for since customers are given the option to delete the data themselves.

| improve this answer | |
  • Simon, that is good in principal, but when doing server wide backups it isn't realistic to have a customer option for how long or for them to delete – Steve Mar 3 '17 at 10:45
  • Which service is it you are providing? – Simon Hayter♦ Mar 3 '17 at 10:46
  • @SimonHayter This is more along the lines of customers such as Prisons/schools that we have who, if they ended a contract have the right as any other company to request that we delete all data belonging to them. Like I say, so far we've only ever had a request for data to be removed from the platform and not to the extent of backup. We provide assessments. All customer data is within 1 database (having multiple has other drawbacks, especially with over 2,000 of them as our old system had) – Liam Wheldon Mar 3 '17 at 10:56
  • I think the real question is... does the data held in (secure) backup for a period of 93 days count towards "holding data for longer than is necessary"? Otherwise, I didn't think (by my v.limited understanding of the UK DPA 1998) that a simple "request" for data deletion would be legally sufficient. Doesn't this require a court order under the grounds that the data is incorrect or something? My concern would be in the event of a data recovery and recovering previously deleted records (so some kind of transaction history would need to be kept in order to prevent this). – MrWhite Mar 3 '17 at 12:32
  • @LiamWheldon sounds like you need to update your backing up system. It's possible to backup SQL into multiple files, for example main.sql, customer-a.sql and so forth. Then when a customer becomes no longer a customer you do a delete customer-a.sql within all the compressed archive files, then when you do a rollback, should you need to... that previous customer is not within the 93 days retention. – Simon Hayter♦ Mar 3 '17 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.