Metalink Puns

Kalisto was drafted to Smackdown on Friday

An ongoing problem for new KDE Neon users is the download speed and reliability of getting the nearly 2GB ISO image from neon.kde.org. A very common scenario is to see slow transfer speeds like 200KB/sec or to have the download fail after an hour or more of waiting. While many of us already have KDE Neon this hurts adoption and makes it more difficult for new users to download.

About 2 years ago /u/shvchk created a system that generates torrents and it's a very useful tool. The problem again is that his torrents aren't seeded much and they don't use the official mirror list.

So I created metalink.xyz which will eventually become a web frontend for converting metalink files to web seeded torrent files. I couldn't find an existing tool that does it. The KDE mirrors generate a metalink file for every file, including the ISO files. It includes a list of all the pieces, their hashes, and a list of HTTP mirror URLs.

Direct link to the torrent generated every hour: https://metalink.xyz/kdeneon.torrent

The growth of two of the most-often denigrated subs on Reddit, /r/shitredditsays and /r/subredditdrama, has been facilitated by bots which advertise their presence in the threads to which these subreddits are linked.

These bots are generally hated, and their posts are usually removed manually.

For example, say, in /r/boats, Joe Blogs has a fight with James Biggs about swaging. Subredditdrama posts a link to the thread, and a bot posts a link in /r/boats pointing out that subredditdrama has linked to the thread. Joe and James are alerted to the presence of the subredditdrama thread, and subredditdrama's subscriber base is grown by the addition of two angry people.

It would be nice to disable this kind of link in subreddits, such as /r/boats, which don't want to encourage such links.

It would not discourage thread invasions from places such as SRS, SD and SRSS, but it would prevent such thread invasions resulting in more subscribers for these subreddits.

This change I think would be easy to implement:

• Add a check box in "community settings" to allow moderators to disallow meta links, i.e. links to within reddit itself, inside that subreddit

• If such a checkbox is checked, then any such metalinks in that subreddit would go directly into the spam filter, as already occurs with tumblr links.

• Moderators can manually allow harmless metalinks, such as links to articles in friendly subreddits.

I get the following error when I try to use yum update:

Cannot retrieve metalink for repository: updates/19/x86_64. Please verify its path and try again

Loaded plugins: langpacks fedora | 4.2 kB 00:00
Error: Cannot retrieve metalink for repository: updates/19/x86_64. Please verify its path and try again

I tried URLGRABBER_DEBUG=1 yum update and I get a lot of text. The final result is still the above error message. The final few lines look like so:

• NSS error -8172 (SEC_ERROR_UNTRUSTED_ISSUER)
• Peer's certificate issuer has been marked as not trusted by the user.
• Closing connection 10 2013-10-22 01:03:37,868 exception: [Errno 14] curl#60 - "Peer's certificate issuer has been marked as not trusted by the user." 2013-10-22 01:03:37,868 retrycode (14) not in list [-1, 2, 4, 5, 6, 7], re-raising Error: Cannot retrieve metalink for repository: updates/19/x86_64. Please verify its path and try again

What should I do?

Additional information I think is relevant: I'm accessing the internet using my university's wifi, and starting a few months ago, everybody kept getting security warnings for all websites. The IT people gave us a .pem file to install onto our browsers and it seemed to do away with the security warnings.

I'm trying to download a huge archive in the format:

http://xx.com/cgi-bin/imageserver/imageserver.pl?oid=19181119-01&getpdf=true

and then the link returns a PDF that aria downloads.

The problem is I'm trying to get aria2c download approx 45,000 urls in a text file, and I'm running into a few problems:

1. when I try aria2c -i listofURLS.txt and the list contains 45,000 entries, the process seems to get autokilled. Do I need more RAM or something or is it just unable to handle so many entries?

2. To get around problem #1, I've split up the files into 5,000 line separate files and then running the same command on each file, periodically checking in to see if the 5,000 files have finished and I can start on the next 5,000 line file.

3. I'm using a VPS to download these files, but I notice that sometimes half way through, aria2c just hangs and crashes. I can't do anything besides kill the process and restart.

4. Is there a way to check if a file has been downloaded successfully and redownload if not (or redownload if a newer version)? These are just PDFs, not bittorrents and no metalink files and no file information available - I'm just downloading remotely.

5. Oh and if it helps, I'm actually running aria2c on an rclone mount in a VPS since I'm going straight from the download into Google Drive instead of storing it locally on the VPS. The file size of the files I'm attemping to download is an estimated 5TB so didn't know of a way to just download it all locally as hard to find VPS with so much HD space

tl;dr: aria2c crashes with 45,000 entries; random periodic crashes; how to verify regular files without metalink/bittorrent/no file information? Also using aria2c directly into rclone mount

thanks!

https://snapcraft.io/curl-metalink

This snap ships a build of cURL with [Metalink][1] support to provide a *workaround* for Debianish GNU/Linux distribution where the feature is not enabled due to the missing packaging of libmetalink([Debian][2], [Ubuntu][3]).

This snap does not guarantee to be up-to-date with upstream and free of security exploits, and is only recommended for downloading metalinks only.

[1]: https://en.wikipedia.org/wiki/Metalink (Metalink - Wikipedia)

[2]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=683969 (#683969 - curl: Build using --with-libmetalink - Debian Bug report logs)

[3]: https://bugs.launchpad.net/ubuntu/+source/curl/+bug/1391779 (Bug #1391779 “curl not build with metalink support” : Bugs : curl package : Ubuntu)

I am trying to update the version of OpenSSH in use from 5.8 (eek) to 7.3. When I first started this work I ran "yum update openssh" which YUM reported nothing had been done, OpenSSH is up to date (5.8). Knowing this is wrong but not knowing what next step to take I issued "yum clean all" because it fixed a few things in the past.

After I issued yum clean all I then ran yum update openssh again, but now I am getting this error:

>Loaded plugins: langpacks, refresh-packagekit Error: Cannot retrieve metalink for repository: fedora. Please verify its path and try again

I have verified I have internet connectivity, the firewall is not blocking anything from this host.

I have tried a few different steps to resolve this, but all the commands I issue result in yum returning the error above.

What are my next steps in troubleshooting this?

Edit: Fedora 16 (also ancient, but one step at a time...)

To anyone observant enough to notice that I just recently posted a Plymouth boot freeze fix for 21 XFCE: this is not a question about that PC, so that fix has nothing to do with my next question.

I'm running a fairly new XFCE Fedora 21 install, and I am suddenly having weird yum problems. I basically can't do anything with yum except clean all and view history. I can't undo or redo anything. Here's what happens when I try any update via yum (e.g., in this case, yum update):

Loaded plugins: langpacks epel/i386/metalink | 33 kB 00:00

Could not parse metalink https://mirrors.fedoraproject.org

/metalink?repo=epel-7&arch=i386 error was

No repomd file

One of the configured repositories failed (Unknown), and yum doesn't have enough cached data to continue. At this point the only safe thing yum can do is fail. There are a few ways to work "fix" [sic] this:

1. Contact the upstream for the repository and get them to fix the problem.

2. Reconfigure the baseurl/etc. for the repository, to point to a working upstream. This is most often useful if you are using a newer distribution release than is supported by the repository (and the packages for the previous distribution release still work).

3. Disable the repository, so yum won't use it by default. Yum will then just ignore the repository until you permanently enable it again or use --enablerepo for temporary usage:

yum-config-manager --disable <repoid>

4. Configure the failing repository to be skipped, if it is unavailable.

Note that yum will try to contact the repo. when it runs most commands, so will have to try and fail each time (and thus. yum will be be much slower). If it is a very temporary problem though, this is often a nice compromise:

yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

File /var/cache/yum/i386/21/epel/metalink.xml does not exist

I'm not sure what "no repomd file" means, what the repomd file is, or where it's supposed to be.

Also, I checked, and the error is correct that no "metalink.xml" file exists at the described location... but I don't know what to put there or what it's supposed to be doing.

I am obviously attempting to employ Google-fu to solve this on my own, but it's not working out great so far and I have obligations depending (hopefully) on the function of this computer, so I was hoping someone here might

Hey, so after a recent post on another subreddit i finally decided that i want to try and finish installing Linux on my machine. However im having a problem installing Ubuntu, whenever i try to install it it stops halfway through and tells me "cannot download the metalink and therefore the iso" what can i do to fix this?