So trying to not go into the technical details too much but when two instances federate with each other, they literally share all of the community, post and comment data with all other federated servers. But it’s the job of the host do manage that passing of data.
Now once the host decides to go offline, that activity of informing all other instances of “hey here’s something new about XYZ community!” no longer happens, but each instance still has the historical data from prior to them going offline. So you can still see that old data and still technically reply to it. Just that the host won’t tell other instances that you did reply.
So lets say lemmy.world and then lets take beehaw.org (I know they are defederated lets assume they are not) for example. All the posts and comments which are hosted by lemmy.world on hard drives or servers, are also hosted by beehaw.org and vice versa? So the amount of data is actually doubled in size?
Yep. Add in a 3rd instance and now you have 3 copies of the database, essentially. It’s just that each instance is responsible about telling the fediverse when updates occur to communities on their instance.
Thanks. Wonder how much it would be uncompressed but I guess that doesnt matter as you can probably compress it on your fediverse instance until it is required to be accessed by a user
OK thanks. I am still a bit confused at how it works tho, if they did nuke the website, where would the data from the post and comments be stored
So trying to not go into the technical details too much but when two instances federate with each other, they literally share all of the community, post and comment data with all other federated servers. But it’s the job of the host do manage that passing of data.
Now once the host decides to go offline, that activity of informing all other instances of “hey here’s something new about XYZ community!” no longer happens, but each instance still has the historical data from prior to them going offline. So you can still see that old data and still technically reply to it. Just that the host won’t tell other instances that you did reply.
So lets say lemmy.world and then lets take beehaw.org (I know they are defederated lets assume they are not) for example. All the posts and comments which are hosted by lemmy.world on hard drives or servers, are also hosted by beehaw.org and vice versa? So the amount of data is actually doubled in size?
Yep. Add in a 3rd instance and now you have 3 copies of the database, essentially. It’s just that each instance is responsible about telling the fediverse when updates occur to communities on their instance.
If the fediverse gets really big, lets say the size of reddit, it may be hard for all the different instances to store all that data on their servers
Ya, ActivityPub isn’t without it’s issues… but luckily it’s all just text. Much of that can be compressed significantly.
I wonder what the total data storage size is for all the publicly viewable content on reddit. I find it hard to even guess lol. 100TB? 10,000TB?
The compressed archive of reddit from 2005.5 until 2022 is 2 TB: https://academictorrents.com/details/7c0645c94321311bb05bd879ddee4d0eba08aaee
Uncompressed it is likely way larger though.
Thanks. Wonder how much it would be uncompressed but I guess that doesnt matter as you can probably compress it on your fediverse instance until it is required to be accessed by a user