I am not sure that when I was invited to give this second Memorial Lecture in honour of Maurice Bond the Council appreciated the appropriateness of such a privilege in this particular venue. It was for an occasion here, exactly twenty years ago, that I produced the first volume of what was intended to be an ambitious project to publish in facsimile the archives of the Royal Institution. It was a glittering occasion - as you might expect - but somehow the glitter obscured the small focal exhibition which sought to demonstrate the stages in producing the facsimile of the first volume of the Managers.' Minutes. Unlike those members of the landed aristocracy who had helped Count Rumford form his remarkable Institution, the landed aristocrats of 1971 were here strictly for the food and the wine. And yet the occasion then seemed so appropriate as a celebration of an institution devoted to harnessing technology for the good of science, knowledge and the betterment of mankind. The process, developed at Scolar Press. in Menston, Yorkshire, involved the use of a computer to control the entire process of making the master negatives from exposure to development. 1 named the process gammatype and it was used only twice: for the Trinity manuscript of Milton's poems and the Managers.' Minutes. Like later processes which I developed at janus Press in Ilkley which enabled continuous tone negatives to be made without the use of a camera and then printed from conventional positive lithographic plates, laser separations for colour printing combined with alcohol damping, no patents were applied for and they have never, to my knowledge, been used again. More recently, 1 have been examining ways in which computer technology might assist in the overwhelming problems which archivists face in preserving, and providing access to the huge quantities of historical documents in their care, and this has led to the development of a project at the UCL Library School which could, 1 believe, yield promising results. More of that later.
In his 1966 address to the Society of Archivists in 1966 Roger Ellis drew attention to the fact that the growth in the number and variety of records being accumulated in the archives and record offices of every kind of institution required the profession to re-think the sort of education appropriate for those responsible for preserving and controlling the records created as part of what we may call the administrative process. Although the subject of my talk this evening is Preserving the Record it is obvious that there is little point in preserving materials for subsequent use without providing convenient access to them. We are all familiar with accumulations of historical debris which have been allowed to deteriorate into disorder - like the records of the Punjab Irrigation Company in Lahore which possesses, in a state of such perfect confusion as to be useless, a unique resource for the history of irrigation in India since 1730.
One of the principles most dear to archivists is that of order: it is, in a sense, the foundation of everything they do. But I wonder how many of you know, for English public records at least, the origins of that administrative tidiness which characterises the various processes in a record office? I hope to show, on some other occasion, how the genius of a subordinate officer in the Treasury, one Arthur Symonds, laid down the principles for the administration of government departments in a work which was not intended for public circulation, has never been reprinted, and survives in just a handful of copies: Papers relative to the Obstruction of Public Business and the Organization of the Civil Service. The preliminary letter, addressed to Gladstone as Chancellor of the Exchequer, is dated 23 February, 1853, and from the copyright receipt stamp in the British Library copy we may say with some certainty that it was published in the summer of 1853. The work had its origins in the 1840s when there was much talk of providing for the nation a Public Record Office. Construction began, as we know, in 1851 and by 1853 departmental records were being transferred to the new building in Chancery Lane. What concerned Symonds was the evident disorder into which government departments had fallen:
The ultimate object is to bring back the condition of Offices to the state of subordination to one Scheme of Organization, which at one time characterized our Institutions, so that each part may have its own proper function, and subserve the common purpose of the whole, without delay or obstruction to any other part, and so that the Prime Minister may truly preside over the whole, while each associated Minister being charged only with those duties which are truly special, may be able fully and promptly to discharge them; and the total result may be Unity of Principle and Unity of Action, with, as far as may be, Unity of Practice in the execution of detail.
It is clear that Symonds saw his proposed Department of Registration, Record and Publication as coming under the Privy Council. His picture of the Department is worth repeating:
As this Department consists to so large an extent of details, and is dependent for success on extreme accuracy in details, some of its officers must be habituated to mechanical services, which involve more or less intelligence, while all require entire reference to the occasions when the Records and Papers will be needed. I refer to those Officers whom I have designated the Collector, the Sorter, the Receiver, the Filer, and the Depositor. Some of these Officers should be well accustomed to books and bookbinding, to cataloguing books, and making indexes, and other subsidiary duties, which persons of these callings are in the habit of discharging.
For the purposes of this audience the concluding section of Symonds' book is by far the most interesting for it sets out clearly the proper basis for distinguishing the procedures of Registration and those of Record. Symonds divides his Department of Registration, Record, and Publication into two classes, Intellectual and Mechanical. The Intellectual includes the Enumerator, the Describer, the Definer, the Classifier, and the Designaton the Mechanical the Collector, the Sorter, the Receiver, the Filer, and the Depositor.
The function of the former will be to index, and in some instances to examine by indexing; and of the latter to keep in proper places and in a proper manner the instruments of Registration and Record. Presiding over these staff is the Keeper of Books and Papers who should know not only the contents of the things under his charge, but the proper means of keeping them for facility of reference, and also for their preservation. The use of improper bindings, of improper materials such as even bad ink, bad paper, bad paste, and other things accounted trivial, but necessary for the salvation of the Record, should be objects within his cognizance. It is, quite possibly, the earliest work on management as seen from a Victorian perspective, but its great virtue would seem to lie in the clarity of its expository prose, unlike so many of the empty and confused manuals on management we are exposed to today.
If there were very real difficulties facing the profession in 1966 these have increased enormously in the past twenty-five years and it is a sobering thought that there are probably more files and documents of one sort or another in the world's archives than there have been human beings on this planet since the Age of Pericles. Controlling such prodigious quantities of historical evidence presents the archivist with challenges not too dissimilar from those facing librarians, except that where the librarian (and I include here the manuscript librarian) must deal with both discrete items requiring individual description and classification as well as administrative publications which are usually catalogued according to provenance, archivists and records managers have historically depended almost exclusively upon administrative provenance (respect des fonds) as a guiding principle in the arrangement of documents in a meaningful series. This will simplify greatly the process of selection for any substantive preservation program and reduce the necessity for more than synoptic indexing because record groups tend to have a limiting and self-defining coherence.
If control of vast quantities of the various articles to be encountered in an archive is to be achieved it is obvious that some of the traditional manual methods will have to be replaced by automated procedures, and in this it seems to me that archivists and librarians have much in common more, certainly, than they appeared to have in 1966 and there is a view widely held in America that since for the historian all evidence (printed, manuscript or audiovisual) is undifferentiated there is much to be gained from consolidating access in unified databases such as those available on OCLC or RLIN. For what we term manuscripts the arguments in favour of this approach seem persuasive enough, but 1 very much doubt that the principle of unit entry, the basis of MARC as a format, can without modification be applied to archives. This is because of the manifest impossibility of describing the minimal archival unit in collections above a certain size. This is a problem which librarians face with the huge increase in published and semi~published materials issued by central and local government, corporate institutions and associations of every kind. The independence to create print provided by the combination of desktop microcomputer, laser printer and short-run photocopying machines has resulted in a new type of publication which, because it does not fall within the terms of legal deposit, is seldom systematically collected. In London alone the aggregations of individuals forming what we would call a society or association, whether linked by profession or interest, now number in the thousands and no doubt they all issue semi~published newsletters, accounts of meetings and information for members.
Long before the principle of administrative provenance was codified for archivists the librarians in the British Museum had fastened on the "'dump heading" as a device to satisfy the minimal requirement of getting collections of printed documents (not all of which are ephemeral) into the catalogue without having to describe them individually. That, as it has turned out, proved to be a very effective preservation measure since many of these "dumps" were unlikely to be found, except by the most astute and determined consulter of the catalogue. In America the debate about how best to provide access to manuscript and archival materials opinion seems to have polarised. Supporters of the MARC format, like Hope Mayo at the Pierpont Morgan Library, point to its flexibility and adaptability to describing everything from a charter to a Chippendale chair; opponents, like Thomas Amos at Saint John-s University and formerly associated with the Hill Monastic Manuscript Library, point to the irrelevance of cataloguing rules like AACR2 and the constraints imposed by MARC field structures on the "free text" approach so dear to cataloguers of manuscripts. They also point to the inadequacy of many library systems to handle records over a certain length: in OCLC, for example, a record cannot exceed 4096 characters (including coding), a field cannot exceed 1230 characters, and the maximum number of variable fields is 50. What neither side seems to have grasped is the manifest impossibility of describing the manuscript materials for the history of Western Civilisation in any practicable way other than with the use of what I shall call a re&,ister record. A synonym increasingly found in American discussions is the census record In this respect archivists and manuscript librarians might profit from the experience of the Incunable Short Title Catalogue (ISTC) which provides a comprehensive register of the output of the printing press before 1501 and is more like a finding aid than a catalogue. The most important feature of the register record should be its reference function pointing to other printed sources (such as catalogues or handlists) which describe an item. or a collection, in greater detail. The separate functions of maintaining a widely available machine-readable register and providing more detailed descriptions in catalogues, handlists and summaries must, it seems to me, be maintained. And this is where the technology currently available must be used to best effect both to provide access and to assist in preservation.
There are, I suggest, three stages to be accomplished: (1) international agreement on a subset of the MARC format for a basic register record; (2) a commercially funded publishing program to convert to digitised form the corpus of secondary sources which describe or list manuscripts and archives; and (3) a program to convert existing preservation-standard microfilm of such materials to digitised form. While in no way wishing to underestimate the magnitude of each stage there does not seem to be any alternative: unless, of course, despair has finally reduced us to paralysis. The technology for the first stage is in place and functioning well, and our own National Register of Archives could well form the basis for wider distribution outside the Commission. I am not at all sure that CD-ROM is an appropriate medium for such a Register, and re-writable optical disks (which can be updated from floppies) seem to me an altogether preferable approach. It is currently possible to procure a re-writable optical disk cartridge holding a gigabyte of data for less than £200, and a gigabyte could easily handle 100,000 register-type records. Japanese manufacturers will shortly be releasing disks capable of storing 10 gigabytes and controlled by a standard 386 PC. Price, where hardware is concerned, is already showing signs of diminishing. For the second stage we need ambitious publishers, and I see no reason why institutions holding materials in demand should not prudently explore their own publishing initiatives. For the third stage we need principally the intellectual courage to decide priorities. The technology for converting film to digitised images has arrived.
In its latest report, published this summer, the Commission on Preservation and Access addresses "the feasibility of a project to study the means, costs and benefits of converting large quantities of preserved library materials from microfilm to digital images". This report, by Donald Waters at Yale, follows on from an earlier report (July 1990) in which Michael Lesk observed that image digitisation would soon be both relatively cheap and available. Lesk and I have been talking about ways of digitising film since 1987, and I am pleased to report that it has taken only four years to get this idea accepted as a possibility. Since then, the Mekel 400M and 40OF cameras for converting film and fiche to compressed digitised form have appeared and four 400M cameras are being used in the Virginia State Archives to convert existing negative stock as well as new film. In all. there are now fifty machines in operation in America, and each machine can process a 100 foot roll of film in one hour. There is not a single installation in Europe outside the world of commerce.
The Yale report examines, in considerable detail, all aspects of the planning process for a project to digitise 400 books from microfilm copy - that is about 25% of stage 2 of the plan 1 have already referred to. What worried me about the Yale report is that it envisages a total of 36 months to accomplish the various stages, and with just 400 digitised and indexed books as a practical result. This seems to me a classic example of bureaucratic overkill, because in three years time many of the conclusions drawn from the early stages of the project will be obsolete.
The point of departure, quite properly understood at Yale, is concern for access and two major investigations of the methods used by scholars in the humanities and sciences (the Research Library Group's Program for Research Information Management and the Faxon Institute's report presented at its 1991 conference at Reston, Virginia in April this year) confirm that scholars highly value and tend to favor information that is readily at hand and ... a critical measure of success for libraries charting a course into the future is how readily they steer information into the hands of their clients. ... The mission of the access-oriented library is to generate, preserve, and improve access to collections of recorded knowledge. This mission guides the fundamental relationship between the access services and the library collections.
That seems to me just a little more intellectually appealing than the conclusion reached by Maurice Line at the end of a week-long workshop on the future of the academic research library held at Birmingham in 1989:
The NAL [New Academic Library] is above all concerned with people: the people it serves, and the people who serve them. It is a customer-based library. ... In order to serve customers, they must be understood; not only must their demands be met, so long as they are within the terms of reference of the library, but their needs must be met as far as possible. A library can readily satisfy demands by keeping them to a minimum by giving a limited range of services in a limited way. ... Needs are not always expressed; the better the library, the more they are likely to be articulated, but it is easy to appear to give a satisfactory service because no-one complains.
When flatbed scanners were first introduced four years ago they aroused considerable interest in America, and they have been used to convert large quantities of unbound paper materials (they are currently being used in the White House). At the Microcomputer Symposium which I organised at the British Library in 1987 the capability of scanners to handle materials as disparate as maps, manuscripts and palm leaves was demonstrated. But they are quite impractical for use in most archives for two reasons: (1) they are unsuitable for bound volumes; and (2) they do not permit sophisticated techniques for image enhancement and optical filtration necessary with faded or discoloured documents or documents with annotations in pencil. They have the added disadvantage of failing to provide a backup in the event of some unforeseen electronic disaster. For these reasons it is now clear that digitisation is best achieved using microfilm as a first stage. With vast quantities of archive-standard microfilm to hand (presumably filmed because of its historical importance) it means that the transformation from analogue to digital format can proceed without subjecting fragile materials to further destructive handling. Such a cautious approach should appeal to a brotherhood distinguished for its conservatism in all things!
If I am right - more important if Yale University and the Commission on Preservation and Access are right then it would seem that before very long we shall have whole libraries of historical source material available for consultation in digitised form. The interesting question we must ask is - on what sources will these libraries be based? The answer to that question will have less to do, 1 suspect, with what archivists imagine to be important and more to do with evolving trends in historical studies. In this respect archivists share a problem with librarians, for the electronic distribution of research materials is going to be as important in the future as the electronic distribution of catalogues and finding aids with which we are now concerned. On the other hand it is my impression that archivists, protected as they have been during the past ten years from the management theories of the Harvard Business School, are in closer touch with the scholarly community using their documents than are some librarians.
For centuries students have had little option but to come to the materials, but the expense of foreign travel, combined with the ever-rising cost of metropolitan accommodation, is already changing the way in which research is conducted, and you have only to look at the sort of research projects listed in the British Library's Current Research in Britain to see the kind of use which is now being made of local records situated outside the large metropolitan centres. While this seems to me an altogether healthy development there will nevertheless continue to be a determined core of researchers for whom sources only to be found in London, Paris, New York or Washington are indispensable, and just as inexpensive printing in the second half of the nineteenth century encouraged the creation of enormous projects to publish primary sources so, 1 think, the new technology will soon be put to similar tasks. The agents of this will, in the first instance. be publishers of traditional microform collections, many of whom are already investing substantial sums in CD-ROM technology. Personally, I think they are misguided because that technology was never designed for the storage of text and images. The more recent optical disk technology, by contrast., was developed specifically to handle vast amounts of either ASCII text or compressed images, and functions at speeds comparable with conventional magnetic disks. Within two years or so the technology will be available to repositories to create their own libraries of primary material, and though there may not be a vast market for what they produce it is a technology designed to replicate huge quantities of data on a demand-cost basis.
But even the extraordinary developments in the mass storage of electronic data we are now witnessing will not get us very far when we consider the sheer bulk of the paper archives which are deteriorating every year, and it is the records of our own civilisation which are at the greatest risk. Which presents archivists with a particularly difficult dilemma, though one to which they have become accustomed: for appraisal, the corner-stone of any archival preservation policy is an inevitably crude and inadequate attempt to second-guess the future. And if the appraisal of eye-readable records is difficult, what of electronic records which will increasingly figure in the archivist's responsibilities? What is to be done with disks or tapes which require obsolescent hardware and operating systems to enable them to be read? Is the solution to continually transform such records into the format of the day? or dump them to paper, thereby adding significantly to an already near-hopeless situation? This is a problem about which those interested in the history of texts have a considerable interest since writers today, with few exceptions, are migrating from the typewriter to the microcomputer and that remarkable tool effectively obscures the stages through which a writer-'s work progresses - unless, of course, every stage is backed up on paper which is seldom the case. So the kinds of study which great collections of modern literary manuscripts - like the Humanities Research Center at Texas - have made possible are unlikely to exist for writers in the next century.
I may have given you the impression that the way ahead is a relatively straightforward one: of course it is not, and for a variety of reasons. We have witnessed mistakes - made on a grand scale -before, and you have only to look at the kilometres of quite useless microfilm acquired by research libraries between 1946 and 1980 to understand the caution with which many of you doubtless approach the new technology. One example will suffice. In 1960 World Microfilms produced a microfilm of Sir Frederic Madden's journal. The positive copies which are still being marketed at £700 proved quite unsuitable for either digitising or copyflo so 1 requested a specimen duplicate of the master negative for testing. That, too, proved useless and so the entire manuscript must be re-filmed. That sad story can be corroborated by other examples in almost every major research library. The single most important feature of the project (to which I referred in my opening paragraph) on which I am currently engaged is that given an adequate original film (16mm or 35mm) or fiche every stage of the transformation to digital form can, indeed must be monitored. Because the storage medium is a re-writable optical disk cartridge mistakes can be rectified and images added at any time and the disk can easily accommodate whatever retrieval software is being used. A variety of hardware platforms can be catered for and the proposed system will support whatever operating system (MSDOS or UNIX for example) proves the most satisfactory for handling large amounts of data. Software or data updates do not necessitate a re-mastering process. Electronic replication is simple and rapid and the cartridge disks are, with reasonable handling, impervious to damage and can be kept secure. Substrate tests on a regular basis can establish reliability. There is, however, one fairly formidable penalty for such open access to archival data -ownership control. But 1 can see no practicable way of effectively exercising control over electronically distributed information; and it seems, on balance, a small enough price to pay for the benefits to scholarship and the future of that knowledge and understanding of the present and the past wherein lies our best hope for the future.