*

Offline Tom Bishop

  • Zetetic Council Member
  • **
  • Posts: 10633
  • Flat Earth Believer
    • View Profile
Preserving Wiki Article Links & Videos
« on: October 29, 2018, 06:19:07 PM »
Pete had recently installed a video upload tool for preserving videos on the Wiki, which seems great. I am planning on making archives of the most frequently used videos.

I would like to discuss the best way to preserve article quotes. For example, one page that I have in progress on my personal 'Works In Progress' section on my Wiki user page is Astronomical Prediction Based on Patterns. I am collecting quotes and pieces of evidence at the moment, and am planning to add more summary detail to the article once I feel that the evidence is sufficient. My original idea was to request a .pdf attachment option for the Wiki, to preserve the context of the quotes so that the page doesn't end up as a collection of missing 404s. However, I did a test run of saving all web page links as pdf and saw this:



I noticed the following:

- All of these PDFs make the total file size for archiving the sources over 71 MB. Do we really want each article to be over 71 MB?

- Some of these PDFs are entire books, which I saved with Google Book's Download as PDF option. The Gravitation Vs. Relativity will be useful for making quotes on other subjects and should be in a central library location, but the Story of the Heavens book is only useful to me at this time for this one quote.

What would be the best way to handle this? Rely on archive.org and website robots.txt to preserve content?

Maybe I should just upload the smaller PDFs to the Wiki, and add the larger PDFs that are books to the library, regardless of whether we are only using them for one quote? Make a small PDF of just the chapter?
« Last Edit: October 29, 2018, 07:16:59 PM by Tom Bishop »

*

Offline Tom Bishop

  • Zetetic Council Member
  • **
  • Posts: 10633
  • Flat Earth Believer
    • View Profile
Re: Preserving Wiki Article Links & Videos
« Reply #1 on: October 30, 2018, 09:56:47 PM »
Saving in .html rather than pdf makes much smaller (but a bit uglier) file sources:



Another option is to rely on something like http://archive.is, but I am unsure of its long term viability.
« Last Edit: October 30, 2018, 10:14:47 PM by Tom Bishop »

*

Offline Tom Bishop

  • Zetetic Council Member
  • **
  • Posts: 10633
  • Flat Earth Believer
    • View Profile
Re: Preserving Wiki Article Links & Videos
« Reply #2 on: November 03, 2018, 05:28:03 PM »
Unfortunately, we are losing links. The link in the Coriolis section at https://wiki.tfes.org/Coriolis_Effect_(Weather)#Twin_Cyclones which shows a Kelvin Wave in development at the equator with counter-rotating vortexes that evolve into twin cyclones now goes to a forbidden page and it is not available at archive.org or any other source I could find.
« Last Edit: November 03, 2018, 10:12:31 PM by Tom Bishop »

*

Offline Tom Bishop

  • Zetetic Council Member
  • **
  • Posts: 10633
  • Flat Earth Believer
    • View Profile
Re: Preserving Wiki Article Links & Videos
« Reply #3 on: November 06, 2018, 11:56:06 PM »
I found a solution, which I feel is manageable:

Archive.org allows you to submit links to archive, which will be processed instantly. Unlike archive.is, archive.org will backup pdfs, powerpoint files, and a variety of other filetypes. Archive.org is part of the Internet Memory Foundation, and has a budget of 17M annually. Archive.org no longer follows robots.txt requests, and will archive all content. Chrome and Firefox extensions are available for easy archiving of the active webpage.

For now  will be archiving all content on both archive.is and archive.org. At some time in the future I will see about an automated process to turn all links on a page into pdf, and keep pdf copies on hand as well, as a personal backup.