This week we launched an archiving process for our Document Management System (DMS). Underlying the process we used a product from Interwoven called WorkSpace Archive Manager (WAM). The WAM tool basically moves an entire electronic file (the documents, emails, folders and the meta-data) from one library to another, in our case from the Matters library to an Archive library.
I intend to blog about our general process and what we discovered working with our lawyers at a later date. The purpose of this post is to point out a bug or should that be a “feature” with the WAM tool. One that will probably cause you problems in the future if you don’t address it before you start.
We found in our analysis that keeping document numbers when moving documents between libraries was essential for the lawyers, especially as these numbers are often used in references on the documents themselves. We did look at using the WAM tools ability to store the document number in a separate “old document number” custom field, but lawyers didn’t like this. They wanted the document number to be the document number, end of!
Basically what this meant was, if you have ready to archive:
- matter 1/1 with document numbers 100 and 101
- matter 1/2 with document numbers 102 and 103
We would want them to appear in the archive with exactly the same matter numbers AND the same document numbers.
If you intend to archive everything in one big batch job and then seal up the archive and never write anything else to it then you’ll probably never hit the bug. But in reality you’ll probably want to archive in phases or even more likely on an ongoing basis. If that’s the case then you’ll likely hit the problem.
So in the example above, say we archived 1/1 one week and 1/2 a few weeks later. We would see the problem occur because we’d need to use two WAM jobs to do this. What happens is that the WAM run will create a webdoc entry in the document table (docmaster) of the archive library for the workspaces/folders using the next available document number (in the example of 1/1 it would use 102 – the next available number).
And this is where the problem occurs, when you come to archive 1/2 and it can’t maintain document numbers as 102 and probably 103 are already in use!
The trick to resolving this is to “seed” your document number in the archive libraries high (we chose 75000000), thus the next available document number for the webdoc entries is 75000001+. This will ensure all the actual document numbers can be maintained as they will unlikely “meet” a duplicate number. To me this is a flaw in the product as when you archive, you’re archiving the folders etc as well and thus surely these should maintain their numbers from the source library too.
It certainly should not go creating new entries in the archive with new numbers!
Yup this is not good, good job there is a workaround though. For me obviously document numbers are sacrisanct because its effectively part of the data and an archivings product entire job is to take your data and move it maintaining integrity which if the numbers are changed it is not doing.
Products like WAM always fill me a bit with dread, not because they are bad but mainly because what they do is so important and they tend to operate on vast swathes of data at a time thus when things go wrong they tend to go wrong big time.
Good job with the workaround though.
I take it the WAM issue has been recognised as a bug and will be fixed in a future release ?
It’s been confirmed as a bug. But I’m not sure it’s scheduled into a new release.
My guess is, as with other tools (rather than the core product) any fix will be in an 8.5 release rather than a patch.
DocAuto’s WorkSpace Manager product does archiving like WAM, but since there is no three-tier call open to developers to set document numbers, we have been forced to store the original document identification (server-database-number-version, since all of these together are the only way to guarantee a unique document) in a custom field.
We take a lot of heat about not preserving document numbers, but there are so many situations where it is really impossible to preserve them (like combining the content from multiple libraries). Offsets are fine, but why is it so onerous to just have an “original document number” field? This is very common in systems that were converted from other platforms anyway…
David, when we originally piloted the WAM product that is exactly how we had it set up. We got an awful lot of feedback from our business about the document number and their wish to have it preserved.
It may be to do with every document being stamped with a reference. In a firm of our size it is difficult to explain clearly to everyone the concept of an “original document field”, people liked knowing that the number in the reference is the number of the document.
I wouldn’t say this is a bug but rather a limitation in the way that the Autonomy database works. Each library has a table called MHGROUP.DOCNUMDB which specifies the next available document number. The API does not allow setting the document number either, if you wanted to you would have to update it directly in SQL and then you would get duplicate key errors when your database did finally get to that number on it’s own.