Adding Relay Instances

A 'relays timeline' like @Matt Noyes mentioned sounds good to me. I never look at the Local (social.coop) feed, let alone the Fed feed, and only sometimes at my home feed (followed folks) which is pretty diverse, when I have time to browse. Notifications/@mentions is my primary window: my Mastodon is pretty much peer-to-recognised-peer. But it would be good to have a 'fresh voices' feed, of affiliated instances/relays.

Helping small instances achieve diversity and range sounds good too.

I think it's a great idea to allow other trusted servers to add us as a relay. From my understanding I don't think it causes any harm to us, as it just allows them access to more of the fediverse. I ran my own server for a bit and the lack of relays really hampers the experience. It's hard to find anything via search because hashtags aren't populated, you don't get insight into favorites or boosts on posts because your server doesn't know about them, your federated feed is only your follows, etc. It feels pretty isolated.

I wonder if there might be value in combining joining a relay network with switching to a fork of the Mastodon codebase that includes a privacy option for "Local" posting. The relay would make our public posts visible to more people. If this is a concern for anyone (and honestly, I don't know if it is, just trying out scenarios here) then local posting would continue to allow a posting option that's more public than "Followers Only" but less public than "Public."

I am supportive of the spirit of acting as a relay for new, small, aligned instances, as a way to help them get more content into their federated timelines (the impact on our own federated timeline will be negligible, as I understand it).

If/when we do want to act as a relay, in addition to the community chiming in on alignment with the instances we'd relay to, I'd like to see TWG weigh in on any additional burdens on our infrastructure, and the CWG weigh in on the suitability of the proposed instances (at least do some due diligence on #fediblock).

I am hesitant to offer this as a service to the community, as I would prefer there were some expectation of reciprocity. Perhaps what we want to do here is write a Relay Policy which allows Members to request Social.Coop serve as a relay, with clear guidance for how relays are added/re-evaluated/dropped.

I'm also in favor of the policy being "Relays are added only with x% of members voting, and y% approving plus a certification from the TWG that a particular relay is safe-to-add" (or an even smaller policy if it can be done)

I would be totally FOR relay for other democratically-run instance if they're bigger than a certain number of active users (like > 10 or > 100). If not, we risk accidentally propping up structures that run counter to our goals.

What sort of structures are you talking about here?

Nothing extremist, but non-coop structures. If we want social.coop to survive, we have to have new people coming in to replace those that leave or become inactive, and to do that people have to know coop model is possible and can work just as good as benevolent dictator.

Personally, I disagree. We should absolutely be encouraging the co-op model, but that doesn't mean not interacting with any other non-co-op folks. Small instances that do good things might use different models, that doesn't mean we shouldn't show solidarity.

I was too strong on my first statement - but if the users of the other models see the coop model as viable but not for them (right now, or ever), that's fine since for them it's still a real thing, just not something they're into. I'm more running into people thinking that a coop model is a fantasy or fake.

Hi all, after being referred to this thread from the friendly admins over at social.coop, I finally took some time to digest it all within the context of my original, somewhat narrower question there:

How can we better support (hashtag) search beyond our own instance?

Here's what I've gleaned and attempted to summarize based on the above, offered in hopes it can make this easier to understand and potentially vote on the relay question again in the future...

1. Currently, the federated timeline lists posts from people followed by members of our own instance

   From https://docs.joinmastodon.org/user/network/:

   When you browse the federated timeline, you see all public posts that the server you are on knows about. There are various ways your server may discover posts [are there other ways we have at our disposal???], but the bulk of them will be from people that other users on your server follow.

2. Currently, hashtag searches will only return results from your local instance [and individuals followed by members of your local instance?].

3. There seems to be very little documentation of what a relay is, and what it can and can’t do - see wiki: https://joinfediverse.wiki/Fediverse_relays - HT to @Drew Harry for sharing this above!).

4. But according to @Sam Whited, the Mastodon admin page does say:

   A federation relay is an intermediary server that exchanges large volumes of public posts between servers that subscribe and publish to it. It can help small and medium servers discover content from the fediverse, which would otherwise require local users manually following other people on remote servers.

5. In other words, a relay enables an instance to pull in and display (not push) posts from all members of another instance within its federated timeline (again, not just those followed by members of one’s own instance - see point #1 above).

6. And (at least theoretically?), a relay offers the ability to search hashtags across other entire instances, not just individuals followed by members of one’s own instance.

7. Both of these capabilities (hashtag search and full display within an instances federated timeline) COULD be reciprocal if individual instances published AND subscribed to each other’s stream via the relay.

8. What customization can be done to control what comes in and/or goes out is still vague, and might require forking.

   For example, (if I'm understanding it correctly) @Dynamic wonders if we could have more granular privacy options that would allow our posts to be shared at an instance level only, shared with other instances via the relay, or fully public (via federation).

9. Outstanding questions:

   a) Can a relay, perhaps operating as a kind of searchable database, also support search capabilities for individuals (not just hashtagged posts)?

   b) The Explore > Hashtags feature currently allows us to see what is trending across the fediverse (or at least those our local instance hasn’t blocked). Why can’t we also search for specific hashtagged posts across that?

   c) Can we set up a relay that allows us to only search for hashtagged posts from other instances without pulling all posts from that instance into our federated timeline? Or are there existing relays we might be able to draw upon for this search function, that don’t require us to create our own?

   d) More broadly, should the cost of setting up and maintaining relays be shared with (or born by, if not reciprocal) other relay members?

My Own Final Thoughts

After digesting all of this (or at least attempting to!), my feeling is that we should at least be open to setting up one or more relays, and/or joining others (especially if we have some control over how data gets imported, exported, filtered, or accessed). Discoverability is an important element of our core solidarity principles, and the functionalities relays offer seem to be an important tool for doing that in a well-managed and democratic way.

I also think we need to do a much better job of translating this stuff to our members. In a way that allows them to take advantage of the full affordances of the fediverse, while also be more informed participants in these discussions.

Your post is, to me, a good summary of the issues with relaying.

My concern with relaying is related to your 9c question. I want to prevent the pulling in of all posts, i.e. full text search, and vice versa. I'm completely okay with hashtags because we can choose to not use them. Pulling in all posts opens the door for abuse.

a) Can a relay, perhaps operating as a kind of searchable database, also support search capabilities for individuals (not just hashtagged posts)?

Do you mean full text search? Or searching for users? Either way the relay just pulls in posts, it doesn't change anything about how search works except gives you more posts to search as if they were local. This means that users on the other server will be searchable if they've posted and their posts have been relayed to us, but full text search will not be enabled (we could do that separately, but that's another discussion).

b) The Explore > Hashtags feature currently allows us to see what is trending across the fediverse (or at least those our local instance hasn’t blocked). Why can’t we also search for specific hashtagged posts across that?

I'm not really sure what this means; you should be able to search for any hash tag and get all the posts that the server knows about whether the hash tag is trending or not? It's up to the moderators to approve hash tags for this feature, and we often forget about it, so the "trending" pages may not be very up-to-date either way (sorry about that).

c) Can we set up a relay that allows us to only search for hashtagged posts from other instances without pulling all posts from that instance into our federated timeline? Or are there existing relays we might be able to draw upon for this search function, that don’t require us to create our own?

To do this you'd have to pull in those posts, otherwise they can't be searched. With the caveat that the tech team should make this call, not me: pulling in lots of extra posts likely isn't a problem and is a trivial amount of extra space taken up in the database.

Finally, the last sentence and the next question seem to be a misconception of how relays work, I'll answer it next:

d) More broadly, should the cost of setting up and maintaining relays be shared with (or born by, if not reciprocal) other relay members?

When I say "set up a relay" we don't have to run any extra software, pay for servers, etc. this is just a configuration option that has to be set in both Mastodon instances and then they start sending posts from all users instead of just followed users. It doesn't really have any cost for us; the smaller instances might have to increase the size of their database drastically if they relay with a much larger instance, but then they get the discoverability. The instances that asked if we'd relay with them are okay bearing this cost of a larger database because of this.

Pulling in all posts does not enable full text search, we can decide to enable/disable that separately. All it does is pull in posts to the federated timeline, nothing else changes. We also can select what servers we relay with, so it's not as if we'd do this for abusive instances, we can be selective about it.

Thank you for the clarification.

@Jeff Piestrak notes

Discoverability is an important element of our core solidarity principles

Yep. On this basis, yes, relays have a place in the ecosystem.

The complexity that Jeff analyses is one of the persistent troubles in the fediverse (and digital generally. And human (male) society generally 🙄 ) . . There are so many powerful tools, and so many complex interactions arising from all the tooling, that only 'wizards' can be literate in the ecosystem that results. I see the same thing occurring in federated wiki for example, where amazing capabilities invented in the past 18months (with graphs for example, or with scraped geographical data) are being leapfrogged by even more amazing capabilities (in collaborative network mapping) . . As a 'mere user' who tracks the tech conversations, I haven't a clue when, if ever, I'll find the headspace to learn to drive these powerful things.

So what? I don't know. But 'the simple user' is a basic orientation we need to somehow assert? Whatever that might mean. As regards searchability across Mastodon instances . . keep the UI simple? Keep the simple searches working well? As regards relays.. maybe add a 'guest instance' timeline?

Grateful to all for such a thorough examination/explanation of this idea, and to @Sam Whited for proposing it. It seems to me we are ready for a proposal, maybe using the "good enough for now, safe enough to try" standard for a limited time period (six months?).

Yes, much appreciation to @Sam Whited for the original proposal and detailed responses, and the many other contributions! I have a clearer idea of the mechanics of search and what a relay might offer. And to @mike_hales' points, hopefully we can improve our ability to translate these kinds of complexity to our members going forward (and simplify as appropriate).

Though there still seems to be some remaining concerns about the impact on our timelines, a trial period with member approved smaller instances (the focus of Sam's original proposal) seems relatively low risk while offering reasonable benefits (including the lessons all parties might learn from that).

My original interest in making a much broader range of fediverse posts searchable (by hashtag) without flooding our local or federated timelines with the actual posts would remain partly unresolved. Happy to work with others who share such an interest in framing it as a doable proposal (not sure if or how we might do this on a technical level...).

How many active members do we have now? It occurs to me that if the social.coop experiment thrives that we might reach a population level - 10,000 members? 5,000? - where it feels too big and we might want to calve off a second instance. social-too.coop? Anyhow, if we did that at some point in the future, it would make good sense to maintain a relay between social.coop and it's progeny.

Based on the comments here it seems what people are looking for is more like this fakerelay thing https://github.com/g3rv4/GMFActionDemo

Thanks to all who are trying to explain. I understood we would dominate such a relay, offering a service to small instances to help them to connect. That is fine, but I would like to see a tangible commitment to democracy from the benevolent dictators of the joining instances, and I oppose a 1 server 1 vote committee above this level of decision making. This would be the 2ct of a non-techy new member who welcomes educating texts and friendly critique where they might go wrong.