Add support for groups in diaspora

The technical difficulties amount to federating messages. By creating a groups feature, we are essentially extending the project to also be a large distributed forum. Similar to newsgroups, I assume, people would subscribe to different groups and receive all updates on those channels. If we want to model groups after Facebook, we'll need to federate group comments and group messages separately. We can also model groups more like Usenet and simply have 1 mode of transport, leaving the hierarchy of messages up to the view layer and disallowing commenters to alter the subject of their federated group message. Unlike sharing posts, which federate to a whitelisted select few people, group messages would be federating to a much wider group of people. I would raise that as a concern just as a gut reaction, but I'm not sure of the actual impact as I do not run a large pod.

@tomscott - thanks - I'm not sure I fully understand; what does federating messages (as opposed to posts?) involve?

My initial reaction is that group messaging is less important than group posts etc. Or could group messaging be implemented later - client-side (?) could take the list of group members and add it to a private message to: list [I guess this wouldn't currently work as you have to have mutual sharing to be allowed to pm someone].

In the longer term I would also be in favour of having plugins for groups to be able to have their own wikis or etherpads for collaborative editing...

@nickdowson well, it's more a matter of volume. if i post in a group that has 1000 subscribers, that message has to get federated out to a maximum of 1000 pods, quickly. that is a different scaling problem than, say, Facebook has, because they can just spin up a bunch of web servers that all connect to the same database cluster. with us, we're attempting to keep multiple databases in sync to ensure the data is distributed to each necessary outfit. by "group messaging", i meant group posts, by the way…i'm just using the term "messaging" here to describe how pods communicate. sorry if my language is a bit unnecessarily esoteric. :)

anyway, my major concern is how are we going to distribute this data quickly and efficiently across a wide range of pods. in the current federation protocol, we do have some issues wherein a post won't federate completely across everywhere it should be federating. i don't believe there's a fault-tolerance system baked in that ensures posts always make it to their destination. these features seem NECESSARY for group-based interaction, because suppose you make a post in a group and 50/100 of the pods that need to get it, don't. that discussion either dies, or worse, it gets replied to. the replies federate across the group and are referencing a post that for half of the pods doesn't even exist. before we implement a group feature, we need to figure out a solid solution to that problem.

So, one of the things groups currently would depend on is federating properly. We also need to think of a basic UX implementation.

Pistos had a groups feature on his fork of Diaspora that I really loved. The problem is that his feature is not compatible with Backbone.JS, and so would probably need to be rewritten, with tests.

What I liked particularly about Pistos' version of groups was that it was similar to how StatusNet implemented groups. Groups could be mentioned in a post using a bangtag, so ![email protected] would go to a cat group hosted by joindiaspora. Joining a group is kind of like a followed hashtag, except that there's an admin that can edit the page, or remove entries if they're offensive/spam/whatever.

The real barrier, I think, is federation support. At some point in the future, we may need to further consider our federation system and think about going with a better-supported protocol, such as OStatus as a whole (Diaspora only uses some of OStatus), or Tent, both of which are exploring solutions to existing problems that we ourselves have seen in Diaspora and other networks.

One of the long-standing problems of our federation system is that it currently needs refactoring. A general consensus is that putting federation into a layer of some sort may be beneficial, and even thinking about touching protocol work hinges on the state of our federation system and workers.

On that last point, @seantilleycommunit , would it be worth doing a call for volunteer developers to form a working party to look into the feasibility of putting federation code into its own layer, and implementing it if feasible? It should be headed up by one of our main current developers, but could include some people new to the project. It seems to be a high priority, but the size of the task seems to be holding up getting it done. I think it might be a good idea to try to form dedicated groups for some of the biggest and most important development tasks such as this and creating an API. What do you think?

That's really a separate discussion, but I moot the idea here and if you think it's a good plan, I can start a separate discussion.

Ok, thanks for running through some of the difficulties. @goob that sounds like a good idea - I would think that going with a well-supported protocol should also be high priority; do the federation layer and the protocol support need to be addressed at the same time or from a technical perspective does it matter in what order they are addressed?
@seantilleycommunit - as I understand it Ostatus is not privacy aware? correct me if I'm wrong! I would think that developing/supporting/collaborating on a protocol that goes wider than diaspora is highly important to the success of both diaspora and the social web (perhaps this page is not the place for that particular discussion though)

Great discussion here. I fully agree with Sean, but also want us to keep in mind, that groups in Diaspora* should probably something more than groups in OStatus. Groups in OStatus are like an open channel to which everyone could post messages to and anyone could subscribe that channel just like following a person. But when I think of groups, I'd really want a group to not always be public and give always public information to everyone. The group should especially control, who can join the group (admission-mode) and who can read which of the group's contents. So to me it seems as if we could take the groups from OStatus, but also need to add a privacy-layer.

Nick Dawson, adding further information to a group like etherpads, wikis or file-repositories isn't the very big deal after all. Since we rely on the activitystreams-protocol, we can support any activities for any modules in the future. So when Tom Scott says "message", he most likely means a generic message, that could be anything like a posting, a like or an editing of a wiki-page.

@rasmusfuhse I agree about the importance of privacy options (though as a start I think a blanket option to make all content private/public is fine).

Could you tell me more about the activity streams protocol?