Our world is governed by laws. Natural laws – what goes up must come down. Laws of thermo-dynamics. Laws of unintended consequences. Murphy’s law – If things could go wrong, they will go wrong at some point (Usually when you can least afford it to go wrong). These laws will play out in your organisation and in your Data Vault.
Ever had the wrong file/data arrive, invariably only discovered some time later? Data arrive out of sequence, invariably only discovered some time later? Require to load history pre-dating the current already time variant data in your DV satellite? In other words, do you operate in the real world?
We know and understand that the Data Vault standard processes are quite forgiving in nature. For example – late arriving keys, broken referential integrity and the like are ingested with no ill effect to the Data Vault. However the above scenarios traditionally could perhaps force you to roll back to a certain point (if you can) and replay the files or data streams in the correct order (if possible). If neither of these options are available – a hand crafted custom jobs would be required to go and “fix” the issues. This means that you will require special permission and sign-off to correct any incident (Murphy’s law) , compromise the audit trial of the data within your DV and delay the availability of the data while you fix and rigorously test the fix to ensure nothing else is compromised (Law of unintended consequences)!
This paper will explore an extended use of the Record Tracking satellite construct to allow the above scenarios to be remedied without the need to replay data into the DV. In doing so the audit trial of information will remain in-tact and provide little to no delay to information delivery to the business.