Friday, April 13, 2012

Genealogy In The Digital Age


Nowadays we’re all getting spoiled by the proliferation of genealogical information online, especially of images of original documents located in archives halfway across the country—or halfway around the world. But unfortunately not everything is online yet; what’s online is often over-processed and not very legible, and it’s not always free, either. Genealogy has become big business.

When I started out, online genealogy was in its infancy, so I ordered microfilms through my local Family History Center—lots and lots of microfilms. (They were only $2 a pop back then, so I renewed many of them to “Indefinite Loan” status and they are still at the Center.) And at that time, the Center had an excellent microfilm printer, so I printed out the records I found. That meant I bought lots of steel filing cabinets, too, which even then weren’t exactly cheap.

I still have a vast pile of paper records that never made it into the cabinets. (Fortunately, filing cabinets are useful for other family members, too.) In my defense, let me say that I had no idea back then how many ancestors I would discover and how many documents I would want to save. Nowadays my genealogical records are mostly in digital format, filed in digital filing cabinets on my computer, and I am working on getting all the paper documents and photos into digital format so that if the original papers don’t survive, the images—and the information on them—will.

The Center’s microfilm printer eventually faded and died, but technology evolved. I evolved too. Not long before the printer died, I bought a small but (for its time) very advanced digital camera, a Canon A95, to be used on a vacation. When we came back, I used it not only to photograph images displayed on a microfilm reader, but also to take photos directly from the microfilm, with the aid of a small desktop camera stand, a lightbox, and two add-on lenses, a Hoya 10X and a Canon 4X. (I was able to do this because the camera had a center-point zoom feature which enabled me to judge whether the focus was clear or not. Newer, more “advanced” cameras, don’t play well with add-on lenses.)

The whole photography package—including editing software—saved me vast sums of money, since so many of my ancestors were concentrated in the same areas for generations, and there were many church registers where relatives turned up in one capacity or another on virtually every page.

Alas, after about 5 years of faithful service, the A95 that was the linchpin of the whole operation lost the center-point-zoom feature and couldn’t be fully restored. I still have the camera, though, because even if I can’t use it to photograph legible microfilm images with my old setup, it still takes excellent photos (including photos of microfilm or fiche images as displayed on a reader), and is very compact and lightweight—a major consideration when traveling.

Fortunately, many of the original records I wanted were now coming online. I was delighted when Ancestry.com began to put online the US Indian censuses and the Drouin collection of Quebec church records, along with some of the surviving Canada censuses and Quebec notarial records. I cheered when FamilySearch.org began digitizing its vast collection of genealogical record and putting Canadian records online. This gateway page includes links to the Drouin collection of Quebec original records, including notary records and Catholic parish records, plus links to original records of other provinces and to the Canadian censuses, and indeed original records from all over the world—and it’s all available for free. If the records you want aren’t online yet, they will be; massive amounts of material are added every week.

The records online at commercial websites are all too often over-processed, to make the paper as white as possible even if doing so fades the writing on the pages to the point of illegibility. FamilySearch online records are straight from their microfilms, not “improved” by any ham-handed editing, so I can download the images and manipulate them so that they are often more legible if less “pretty” than the same images on Ancestry.

Unfortunately, not everything I want or need is online yet and may not be available soon enough to do this senior citizen any good. So I often still turn to microfilm for the records I want (which now may be ordered online only and are sent to your nearest Family History Center for viewing).

Technology marches on. The microfilm printer at my Family HIstory Center is long since deceased, replaced by a microfilm scanner.  It’s convenient and free to use (at least at my local center), and you save your digitized images to a flash drive, external hard drive, or directly to your notebook or laptop computer. Sounds perfect, yes?

Well, not exactly. It works fine with images that are clear and with good contrast, but the one at my local Family History Center (like my scanner at home) has a maximum quality of only 600 dpi, so scanning just won’t do justice to a record in very poor condition. Larger Family History Centers, other repositories with microfilm and microfiche records, and of course the Family History Library in Salt Lake City, may have more scanners, but those scanners are likely to have long lines of people waiting to use them, so you have to limit your time at the scanner, then come back as many times as necessary to collect all the records you want. The software can be a challenge as well, although most places do provide instructions at each scanner, and presumably the people in charge of the place can help you if necessary.

The first thing to realize about a microfilm/microfiche scanner is that if the facility is busy, you can’t hog the unit; you have to share. This means that you have to nail down the locations of the images you want before taking the film or fiche to the scanner, so you can work quickly and efficiently. Even if you’re the only patron in the place, you should use a reader to locate the records to avoid wear and tear on the scanner and to minimize the likelihood of the computer connected to the scanner having a nervous breakdown and being out of action for weeks or months.

Locating the records is no help if you drop the film or can’t find the document once the film loaded onto the scanner, so while I’m at the reader, I “bookmark” the images I want, without damaging the film. I use transparent 1/4-inch removable plastic colored marking dots. Specifically, I use Avery See Through Color Dots, #05796, and they’re available at many large office supply stores, or you can order them online. Every package of 860 dots has equal amounts of 4 colors (red, yellow, blue, and green) and a dot leaves no sticky residue on the microfilm. Just to be doubly safe, I mark the beginning of the target record with a colored dot placed in the space just before the first image (not on the actual image), in the upper or lower margin if there’s not enough space between images. I mark the end of the document with a dot of a different color just after the last image, again without overlap onto the actual image. (I usually use green or yellow at the beginning and red or blue at the end.) Once I’m on the scanner, I remove the dots as I go, put them back onto the sheet they were originally attached to, and re-use them. If good manners means that I have to yield the scanner to another patron in mid-session, I take one of the color dots from the sheet and mark the new starting image as I did before.
On some microfilms, page or image numbers are given; If that’s the case, you can skip the removable dots and just write down the numbers (beginning AND end) before going to the scanner, then check them off as you scan them. Write a note of where you had to break off the session if you can’t finish everything at one go. There are few things so maddening as discovering after you’ve sent the microfilm back (or after you’ve taken the plane to go home) that you missed a crucial page or three.

Once at the scanner, locate the first image you want to scan and get it centered and focused as clearly as possible. The scanner’s screen may have guidelines on the monitor for that purpose. Be aware that with scanners at Family History Centers the associated computers will add “For Personal Use Only” in large letters across the bottom of the document, so allow room for that; you don’t want vital information to be covered. Use the scanner computer’s software to determine the best settings to use for this document. If the film is negative (white text on black background), the software should be set to convert the image to positive. Use the Prescan feature if it’s available; that way you should be able to adjust the light level after the initial scan is displayed on the computer, to maximize legibility.

When it comes to the resolution setting, you have to make a tradeoff between image quality and the time it takes complete the scan. For some very clear microfilms or fiche with good contrast, a 200 dpi scan may suffice, but I usually use 300 or 400 dpi, especially if the film is blue rather than black (meaning the contrast is poor). If the displayed image is very dark or very pale (or if part is very dark and part is very pale), I use the maximum 600 dpi for the entire document. Scans of 200 or 300 dpi are the fastest (just one pass of the scanner with the machine I use locally), 400 dpi takes 2 or 3 passes, and 600 dpi takes 5 passes. If you go with 600 dpi and there are other people waiting to use the scanner, limit the number of images you scan to 5 or less (if the document is 6 pages, the person at the head of the line will probably not complain if you ask him/her nicely to let you scan that 6th page).

Once scanned, the computer monitor will display the result; you should be able to adjust the light level. The last step is to click on whatever button will tell the computer you want to scan another image. The image you just scanned will go to a sort of parking place until you’re done. When done, you save the images you have scanned to a folder you create on the scanner’s computer, then download them to your own storage device at your convenience. Save the images in TIFF format rather than JPEGs if permitted; JPEGs will degrade every time you make any change to the image. Be sure to give the folder a useful name, such as the film number or name and a brief description of the contents. If this folder is one of several from the same film, you may want to add a batch number, especially if it’s all one long document. (I scanned a 107-page probate not long ago, and if I hadn’t put those batch numbers in I’d never have been able to reassemble it in order.) If you’re not going to download the folder to your own storage media yet, add your name or initials; you don’t want to download someone else’s files—or worse, find that someone else has downloaded your files by mistake and then deleted them. Once you’ve downloaded your files, delete them from the computer.
Although I use the microfilm scanner at my Family History Center for some documents, I still prefer photography. I get better results by photographing documents (and photos too) with a digital camera than by scanning; most scanners’ highest quality setting is 600 dpi, while the camera can capture 1200 dpi—especially useful with documents faded or blackened by age and/or poor storage conditions. Moreover, I never have to wait in line for a short stint of scanning, then come back several times to get the other documents I want to collect. If the microfilm reader displays the images on a vertical screen, great; if it’s one which projects the image onto a slanted surface, I don’t mind that the image will come out in a trapezoid shape (like the opening credits of “Star Wars”) rather than a square or rectangle, because—unlikely as you might think—that’s a quick fix.

Once I get the images, scanned or photographed, onto my computer at home, I always make a working copy to edit the images, and park it in a different location on the computer, while the original unedited images remain untouched in my Pictures folder and will be backed up automatically onto my external hard drive that night. If I’m traveling, I park the new files on my laptop and back them up onto a portable hard drive or a data DVD until I can transfer them to my desktop computer at home. (One of these days I’m going to back up all my files to the Cloud as well.) Either way, once backed up, the files can be deleted from the flash drive so I can use it again.

Whether I’m working with scanned images, images downloaded from the web, or image I have photographed, I use Adobe Photoshop Elements as my editing software. There are other good photo-editing programs out there—one may be already on your computer. But Photoshop Elements works very well for me, and isn’t overly expensive, so I have no incentive to change. (I started with version 4 and upgrade only occasionally. I’m now using 6 or 9, depending on which icon my cursor happens to land on.)

Photos of documents can be edited, one by one, as soon as they are on my computer and duplicated. (My cameras only take JPEGs, but I can convert the photos to TIFFs on my computer so they don’t degrade during editing.) However, scans from microfilm or fiche saved as a batch are now married and have to be separated in order for me to edit them for maximum clarity. Therefore, I do not begin the editing process by opening up Photoshop Elements, because Photoshop Elements will display only the first image of a multi-image Tiff file and not the rest—not without a little further effort on my part.

Since I use a Mac, I begin by opening the working copy of the unedited scanned TIFFs in Preview and saving it as a PDF file. How do I do this? I can’t tell you for sure how to do it on a Windows machine, but on a Mac you open the TIFF document, then click on File->Print, which brings up the print menu. In the lower left corner of the menu is a button labeled PDF. Pull that PDF menu down and choose Save As PDF. (I save it to the Desktop.) This part of the process takes about 20 seconds.

Now I open Photoshop Elements, choose “Select”, and open the new working PDF. The screen will display thumbnails of all of the images in the sidebar. I select the image(s) of the first document, or if it’s all one document, select All. Next I choose “Save As” (usually I use the same name as on the original TIFF folder), but before I hit that Save button, I change the file format in the dialog box back to TIFF. Photoshop Elements will immediately warn me that changing the file format can only be done as a copy. That’s exactly what I want it to do, so I click “Okay”, and voila! The individual images of the file are now divorced. That’s because these TIFFs were created by Photoshop Elements rather than by the scanner’s computer. I repeat the process as necessary to separate (and name) all of the individual documents. (To avoid confusion, to which I am prone, I then park the original working copy and the first PDF elsewhere off the Desktop. When I’m finished processing all the images, I can Trash both sets. I’ll still have the original download.)

I am now in business. I can open up any of the new TIFF files in Photoshop Elements, and thumbnails of all the separate pages will appear. I have turned an “inseparable” document into a folder of separable images that I can process one at a time. Now I can crop and/or edit any of the images that need it. I can turn a negative image to positive in Photoshop Elements simply by opening it and pressing Command+i. I can massage all of an image as a whole, or I can select and massage only an area that needs extra work—such as adjusting the light levels only in a washed-out or too-dark area—to be completely legible. If it’s all one long document, I can add page numbers to the name of each image. This means that I can separate a long document spread through several files—say, the probate for Davey Jones—into single pages with individual names showing the order of the pages (DJ24, DJ25, etc) and put them all into one folder called Davey Jones Probate, and file that with the rest of Davey’s records in my digital filing cabinet.
If I am editing a “Star Wars” photographed image made at a microfilm reader, I start by opening it and saving it as a TIFF. I can change the trapezoidal image to a rectangular one. Here’s the recipe: First I level the small “top”and the longer “bottom” using the Image->Rotate and adjust them to completely horizontal (you can do this in tiny fractions of a degree to get it right). Next, I crop it, leaving a small margin all around the page. If it’s a negative image, I hit Command-i to turn it positive. Next, I choose Image->Distort; little handles will appear at the corners. I use the handles at the narrow-edge corners to pull each side outward until it forms a right angle at the wide edge and the image is now a rectangle. Next I choose “File->Save As” option; the software will ask if I want to apply the transformation before merging the two layers into one. I click on “Yes”, then cancel the save. Now I can re-crop if needed, then click on Levels (Command+L; the L can be lower case) and edit the general light level for optimum contrast. (I can use the circle or square selection tool to select a particular area that is still too pale or too dark and adjust that area individually.) When I’m satisfied, I choose “Save As” and give the image a more meaningful name than “IMG” plus a number, which is what the camera named it.

Last but by no means least, once any single image has been optimized and saved, I can resize it and/or crop it to focus on a particular area and resize that area. I can save just that area under a new name with “Save As” and change the format to ,jpg or .tif, or I can choose “Save for web” (.gif) if the image is small already. (That’s how I was able to post signatures from notary documents in my last post.) I can turn a document or part of a document into a small PDF e-mail attachment that any computer can open. And I can do it all without losing the whole of the original optimized image simply by closing the parent file without saving the changes.

All of this sounds time-consuming, but once I got the hang of it—which was fairly quickly—generally it has always taken me less than 3 minutes to crop and edit an image, save it, then generate a resized version to share. Even from the start 10 years ago, with no prior photo-editing experience, it usually took me less than a minute or two to edit an image of a badly degraded old document to make it legible. (Damaged or degraded heirloom photos may take considerably longer to repair; you don’t want your great-grandparents’ wedding picture to look like a zombie jamboree from a horror movie!)

Is it worth the trouble? Decide for yourself: below are two copies of a page in an 18th-century Danish churchbook, one as photographed on a microfilm reader with a slanted screen, and the other inverted and edited (and both resized to fit on this blog):






























I for one believe it is worthwhile, because I’m interested not just in collecting the record but in milking the document for every scrap of information that may lead me to more records and more information about my ancestors. Therefore, I don’t want any of that information to be illegible because it’s gone black with age or faded into faint flyspecks. That “illegible” information may hold important clues to find more records and/or to reconstruct part of my ancestor’s life that no one else has found. (I can’t replace words that have been torn away from the page a century or two ago, though. Pity.)

You see, even on a black-with-age page, the original information is still there, buried under the mold or whatever, and it’s not hard to get at it simply by adjusting the light levels. Images which are very badly faded may be too far gone to work with, but I have successfully changed the light levels even online over-processed images that appeared at first to have no surviving information, enough to tell whether that image has one of my family members on it—and if it does, I can order the microfilm (or download it from FamilySearch.org if it’s available) and get a better image to work with, and the information on it.

And in genealogy, it’s the information that counts.


POSTSCRIPT: Now that the 1940 Federal Census has been released, we’re all agog to find our families in it, but hampered by the fact that very few of the states have been indexed yet. I found many of my family members in one afternoon, including some in a large city, because Steve Morse and his colleague Joel D. Weintraub have developed a LINK One-Step Enumeration District finder for the 1940 census, with links to the online images of the actual census records at either the National Archives, Ancestry, or My Heritage. If your relatives didn’t move between 1930 and 1940 and you have their Enumeration District number from 1930, the site can convert the 1930 ED into its 1940 equivalent. If you have an address for about 1940, the site can tell you which Enumeration Districts might have that address and link you to a map of each possible district. Bravo and many thanks to these genealogical heroes!