These are all the torrents currently managed and released by Anna’s Archive. For more information, see “Our projects” on the Datasets page. For Library Genesis and Sci-Hub torrents, the Libgen.li torrents page maintains an overview.

These torrents are not meant for downloading individual books. They are meant for long-term preservation.

Torrents with “aac” in the filename use the Anna’s Archive Containers format. Torrents that are crossed out have been superseded by newer torrents, for example because newer metadata has become available. Some torrents that have messages in their filename are “adopted torrents”, which is a perk of our top tier “Amazing Archivist” membership.

You can help out enormously by seeding torrents that are low on seeders. If everyone who reads this chips in, we can preserve these collections forever. This is the current breakdown:

Status Torrents Size Seeders
🔴 54 154.0TB <4
🟡 183 92.5TB 4–10
🟢 111 17.2TB >10

IMPORTANT: If you seed large amounts of our collection (50TB or more), please contact us at AnnaArchivist@proton.me so we can let you know when we deprecate any large torrents.

  • Adam@doomscroll.n8e.dev
    link
    fedilink
    English
    arrow-up
    76
    ·
    8 months ago

    It seems the majority of the torrents with poor seeder count are in the 1.5TB+ range. I just simply don’t have the storage for that. Most everything in the 0-300GB range is pretty well covered.

    • Appoxo@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      33
      arrow-down
      2
      ·
      8 months ago

      Agreed. I’d like to share a bit of disk storage but I only have 2 TB and I need that for my own consumption.
      Give us smaller torrents (e.g. 50GB parts) instead.

        • interdimensionalmeme@lemmy.ml
          link
          fedilink
          English
          arrow-up
          9
          arrow-down
          1
          ·
          8 months ago

          How do you make the torrent software automatically rotate which content it preserves based on the files with the fewest copies in the Swarm ? I don’t want to manage this manually or have to select files by hand.

          • MachineFab812@discuss.tchncs.de
            link
            fedilink
            English
            arrow-up
            1
            ·
            7 months ago

            That’s an answer I don’t have. I would just focus on grabbing what has the fewest copies for the snap-shot in time when I am downloading the files. Other archivers will grab what needs more copies at the time they download, just as you and I are basing our downloads of what has the fewest existing copies weighed against what others have seeded before us.

            Its a churn of archivers “rotating” the content we each choose to preserve(almost entirely without regard for WHAT the specific content is).

            If enough people were too active about deleting what they’ve downloaded that has “enough” copies and replacing it with content that doesn’t … okay, that’s unlikely, but we still need more seeders in general, and those of us who keep on seeding that which we’ve already downloaded make the decision of what needs more copies seeded in the future easier to make for new archivists and/or those who have invested in more capacity.

        • Crotaro@beehaw.org
          link
          fedilink
          English
          arrow-up
          1
          ·
          edit-2
          7 months ago

          How do I do this? When I open up the magnet link, it only seems to give me one checkbox for the entire 10 TB thing. Sure, I can check “download first and last parts before anything else” but it would still try to download all 10 TB, no?

          Edit: Nevermind, I didn’t grab one of the files that has aac in its name.

        • Appoxo@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          2
          ·
          8 months ago

          That’s my rented seedbox I sync my downloads from.
          At home I have an intel nuc 11th gen with 8tb attached via USB.

      • MachineFab812@discuss.tchncs.de
        link
        fedilink
        English
        arrow-up
        20
        ·
        edit-2
        4 months ago

        Yes. When downloading the torrent, go to the files list for that torrent and un-check boxes. Resist the urge to leave only the first files in the list(as most people will do this leaving few-if-anyone seeding the rest), **and try instead to grab files from the middle or end, or just be random about it.

        When the torrent finishes downloading the files you’ve selected, it will automatically seed those portions of the torrent which you have downloaded.

        EDIT: I just remembered, some torrent programs will actually show you the seed ratio per file in the torrent. There are reasons hardly anyone is (sincerely) trying to reinvent this wheel.

        • spencer@lemmy.ca
          link
          fedilink
          English
          arrow-up
          2
          ·
          8 months ago

          This is true, but I don’t know if you’d be counted as a seeder on that list though if you don’t have the full torrent.

          • khannie@lemmy.world
            link
            fedilink
            English
            arrow-up
            9
            ·
            8 months ago

            I don’t think you would but you would count towards the “availability” that’s listed, so ten leechers each with a unique 10% of the torrent would give a 1.0 availability for a new lecher who wanted to become a seed.

        • interdimensionalmeme@lemmy.ml
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          2
          ·
          edit-2
          8 months ago

          This is too much work, someone automate this with the smartest algorithm and I will preserve with the power of 10x 2tb 10gbe seedbox.

          Automatically auto seed on a rotation the 10% least available file of each torrent Percentage should be adjustable globally and per torrent.

  • onlinepersona@programming.dev
    link
    fedilink
    English
    arrow-up
    59
    arrow-down
    8
    ·
    8 months ago

    Bruh, this is a terrible way to share this. Why not torrents of the raw material in predefined categories that won’t change? Like “1984 - Sci-Fi - English - A-N”, “1984 - Sci-Fi - English - O-Z”, “1990 - Biology”, “2012 - Physics”. Then people would actually even download this to use it themselves, instead of some archive that has to be extracted and will take a multitude of the space again.

    The hell am I going to do with a 300GB archive file that I cannot even look into? I might as well be storing an encrypted blob 300GB large or just reducing the size of my partition by 300GB.

    It’s great that people want to preserve human knowledge, but there surely are better ways to do this.

      • WallEx@feddit.de
        link
        fedilink
        English
        arrow-up
        54
        arrow-down
        1
        ·
        7 months ago

        This is just bad communication, beating down on people that are delivering constructive criticism.

      • onlinepersona@programming.dev
        link
        fedilink
        English
        arrow-up
        16
        arrow-down
        1
        ·
        7 months ago

        Way to gatekeep. Don’t you think it would be better if more people could contribute bandwidth and storage with what they have instead of buy a new hardrive? Wouldn’t you want more redundancy, instead of less?

        • fatboy93@lemm.ee
          link
          fedilink
          English
          arrow-up
          9
          ·
          7 months ago

          I don’t think they use an indexable compression as well, right? That essentially kills stuff for me.

          The easiest way to host is not TB/PB sized archives but indices and slices for those.

          It easier for a lot of us to download a few gigs and share that, rather than download TB/PB sized archives.

  • DengueDucky@lemmy.ml
    link
    fedilink
    English
    arrow-up
    45
    ·
    8 months ago

    It’s unclear to me how these torrents are used. If individual books are not downloaded from them, is this only to make it possible to create similar sites in the future, in case this one is taken down?

    • empireOfLove@lemmy.one
      link
      fedilink
      English
      arrow-up
      58
      ·
      edit-2
      8 months ago

      They are meant for long-term preservation.

      This is basically a “distributed backup” of the entire database. The torrents are not actively serving files- they’re there to store multiple copies of the main database across the globe so that the entire database can be recovered (by anyone with the requisite knowledge, mind you) in the event that something happens to the original Anna’s Archive team or the main database is lost/seized by “law enforcement”.

      It’s equivalent to how backup managers in ye olden days would make broken up piece files of a certain size that could fit onto a CD or DVD, so you could fit the entire contents of a large 20+GB hard drive onto multiple smaller media. The backup itself is not accessed unless your main hard drive crashes, in which case you reassemble all the individual pieces back into your complete OS environment after replacing the hard drive.

  • Bread@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    38
    ·
    8 months ago

    I have a spare 100 or so Terabytes and I can fit roughly another quarter petabyte in my server. I would like to help. I will look more into this potentially tomorrow when I have some free time. The preservation of knowledge is too important.

      • Bread@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        3
        ·
        edit-2
        7 months ago

        I have a few questions as you appear to be part of the archive or at least very familiar with it.

        Roughly how often are the archives updated?

        Do you guys already have a proper backup method or are your seeds acting as that backup?

        Any idea realistically how much bigger the archive can get data wise in the next few years? Estimates or educated guesses are fine. I want to know how much I need to plan in advance.

        If I take the whole archive, must I deploy it or can it be searched through if I have the whole thing and I want something specific out of it?

  • xantoxis@lemmy.world
    link
    fedilink
    English
    arrow-up
    21
    arrow-down
    2
    ·
    7 months ago

    Torrents that are crossed out have been superseded by newer torrents, for example because newer metadata has become available.

    Wait, fr? These aren’t even the final versions of the torrents? You might start seeding multiple terabytes of data and someone goes “lol nm here’s _final_final_3.aac” Absolutely bonkers way to do this. Put hard drives in an arctic vault or something, it would make more sense than this.

  • empireOfLove@lemmy.one
    link
    fedilink
    English
    arrow-up
    14
    ·
    8 months ago

    Id happily seed a tb or so of the most in-danger torrents. My internet aint much but my old pc is almost always on.
    How do I know which of the piece torrents are high or low on seeders? Maybe I’m just being special or can’t see it on mobile but is there no way to check each torrent’s health without actually downloading every piece and putting it in my torrent client?

  • TCB13@lemmy.world
    link
    fedilink
    English
    arrow-up
    16
    arrow-down
    5
    ·
    edit-2
    8 months ago

    These torrents are not meant for downloading individual books. They are meant for long-term preservation.

    What does this mean? If download one of the those torrents it won’t have usable PDFs/Epubs? Just random encoded garbage in some obscure format? So much for preservation. When their website is taken down and they’ll in jail nobody will be able to use the torrents , so why seed them anyways?

    • SinningStromgald@lemmy.world
      link
      fedilink
      English
      arrow-up
      20
      arrow-down
      2
      ·
      8 months ago

      The torrents are to preserve the archive as a whole and not individual books or documents. The entire archive is ~263TB’s which is far more storage than most people have in their home. So instead they broke it up into bits that were more palatable for most that when combined make the whole again. Like a huge .rar from back in the day.

      • Андрей Быдло@sh.itjust.works
        link
        fedilink
        English
        arrow-up
        14
        ·
        8 months ago

        I hope you are wrong. A big multi-part archive can’t normally be operated if any part is missing. I hope they do separate zips of a smaller size, like 100gb chunks of random books. By looking at one person’s comment it seems the largest compilations are very unpopular.

        • MachineFab812@discuss.tchncs.de
          link
          fedilink
          English
          arrow-up
          8
          ·
          8 months ago

          IME with libgen torrents, the filenames will be a random number-string generated by their database, and they might have extentions removed or garbled, but generally these files are actual eBooks/Articles/whatever.

      • TCB13@lemmy.world
        link
        fedilink
        English
        arrow-up
        10
        arrow-down
        1
        ·
        edit-2
        7 months ago

        You’re reciting their bullshit. I don’t care about the size, what I care is that there’s no documentation on the format they’re using and nobody speaking about how to use those torrents.

        Those guys made a lot with their website but that format is totally bullshit. If they truly cared about preservation they wouldn’t be sharing obscure format s instead they would just provide a simple sharded backup of the metadata and files - meaning that anyone could pick one part and the contents would be the actual PDF/ePub files without any special encoding. This approach would’ve been better in multiple ways:

        1. If most parts are lost the remaining would still be available and usable;
        2. If the guys vanish from the surface of the planet anyone could easily pick up the available parts and build a new library from them;
        3. Simple sharded backup means that people could use those torrents to download books as well, just add the torrent and pick the specific file to download. This would make the torrents way more likely to be shared and seeded as most people could afford to keeps parts or specific books they want and not an entire 300GB torrent. For what’s worth individualism and selfishness always win in society, giving people a usable sharded backup would make it more likely to last.

        What I see is the the guys running the website are just asking people to share HDD space to keep their database intact in case of legal trouble so they can rebuilt, but they choose to do it in a way that makes it really hard for others to replicate what they’ve build from their files effectively creating a monopoly on their stuff. Too bad this behavior just fucks up the community if they go away and can’t / don’t want to come back.


        Edit: let me even got further on this. If they were actually interested in preservation they would just provide all their files as individual torrents/magnets people could use both for downloading and seeding long term. Then they could just provide a file with a list of all magnets or a zip of all torrent files in order to allow seeders to preserve large chunks of the database with minimal effort - import all torrents in folder and done.

      • TCB13@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        7 months ago

        As if in 2023 you couldn’t add a torrent and pick individual files to download from it instead of the entire thing.

  • lixus98@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    10
    ·
    8 months ago

    Just got a 300gb torrent to seed for a while. It won’t be on 24/7 but most of the day it will.

    • can@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      1
      ·
      8 months ago

      If you don’t mind leaving your pc on and using bandwidth I don’t see why not?

      • Azzu@lemm.ee
        cake
        link
        fedilink
        English
        arrow-up
        16
        ·
        8 months ago

        Even that doesn’t matter, why would you need to keep your PC on. It’s about preserving the data, if it’s only online occasionally it’s still better than nothing.

        • canni@lemmy.one
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          7 months ago

          Functionally your set up is doing the same thing as a seedbox. They are generally thought of as remote and usually have a very good Internet connection. I think people tend to share seedboxes as well.

        • roofuskit@lemmy.world
          link
          fedilink
          English
          arrow-up
          2
          ·
          7 months ago

          I guess you could build a computer for the purpose with a very high speed connection and a VPN to match. But I assume people are usually referring to the ones you pay monthly for. They usually make managing private tracker access easier because you have a 24/7 machine with lots of storage and a high speed connection running constantly for keeping your ratio up.