Microfiche was ideal and I'm not sure why they stopped using it. Maybe the death of analog photography and no one making film any more? All you need is a light source and some magnification and you're good.
Any digital format is going to be lacking. Imagine if they had digitized all these paper records to the state of the art in 1983: Commodore64 cassette tapes. Whatever whiz-bang format they're hyping today will be a joke 25 years from now guaranteed.
I can answer this.
Even the best archival quality microfilm and microfiche was only guaranteed to last about 50 years, then it started fading like the old celluloid films. If the film records were used often, they broke down faster than that. They got scratched or the film broke, losing documents. If they weren't duplicated and replaced often enough, they became illegible or were destroyed.
Much like any other library, a misfiling could mean those documents were gone for good. As a result, best practice was to keep master copies of the films and duplicate them when the working copies wore out, were broken, or misfiled. But often employees who needed a document right then would grab the master copy and damage it or fail to put it back. Then it was gone for good. Just like music or video, each copy-of-a-copy is lower quality than the master film, until the entire thing is illegible.
In addition, the filming process required a horde of semi-skilled to skilled technicians and a variety of expensive equipment to produce. The equipment broke often, required highly skilled repairman to fix, and the development process was very toxic.
Read-back and reproduction required even more machines that were frequently broken and required skilled repairman to fix; in addition, the reader-printers generated a great deal of heat that required top-notch ventilation and air conditioning to compensate and keep the room cool enough to store the film.
Since making copies was a low-pay-grade task, people who relied on microfilm would often hire a horde of people to make copies, while the higher paid employees would submit orders for the documents they needed. That's two people looking at one document. Add in the person who filmed it, and that's three. Add in a quality control person on the original film, and that's four. Some of these records, they didn't WANT four people looking at it. But give the high-pay-grade employees a scanner at their desk and a computer, and three out of four of those people could be eliminated from the supply chain. Digital is more secretive.
Even though the film reduced the footprint of the files down to 1/3 of the area that paper files took up, that was still a huge footprint compared to a few server racks for a digital system. Also, a digital system allowed for more eyes-off processing than microfilm did. In the long run digital was smaller, cheaper, and more secret than microfilm could ever hope to be. As long as the copies still exist in high quality digital form, they can be converted to whatever future type of medium is required. So as long as the original PDF still exists, it doesn't matter if the disk is broken, missing, or outdated, because you can download another one from the server, or convert the whole collection to the new storage format en masse..
Of course, the mother of all power outages could spell disaster for a digital archive, while microfilm copies would sit there, silently decaying, waiting for the power to come back on, at least for a while. And paper copies, when created from the proper archival materials and carefully stored, can last a thousand years or more.
I worked in and later ran a microfilm factory in the late 80s/early 90s, so that's how I know.
I got testosterone checked as part of getting medical insurance outside of the US. Complain to them. I got a clean bill of health. The only thing they flagged is my low resting heartrate, which I had to do cardio tests to clear. I passed.
