Changes to the diaspora protocol

As you should be aware of if you want to take place in this discussion, diaspora never had or has a fixed, well defined protocol. Because of that, stuff is sometimes broken and writing actual documentation is impossible. In the past, other networks started implementing our protocol by basically reverse engineering our stuff. While this is fine, we obviously are not able to support this in any way. In fact, our wiki page about the protocols used clearly states

Implementers of this protocol should be advised, though, that Diaspora is in Alpha, and as such, this document is not authoritative, and may lag behind the reference implementation

This is a pretty clear disclaimer. Because of that, no core member of the project ever stated the communication with other networks is a thing we support and that we can/will assure this stuff does not break. In fact, I am not even aware of any of the third party networks devs (except Michael Vogel for some debugging) ever talked to members of the core team to discuss about best practices here.

In our ongoing endeavor to move parts of the federation code inside the new gem, some changes to the protocol have to be made. At the moment, all changes were made to get closer to the originating standards like Salmon to ensure even more compatibility in the future. Even though those changes have been made, we (as in Benjamin, Jonne and myself) ensured that no older diaspora pods communication breaks. The newer 0.5.0.0 releases are much closer to the originating standards but still compatible with very old diaspora releases.

Third party networks have implemented old diaspora federation parts in the past, but they have done a pretty bad job at it. The stuff Friendica sent in never was even close to match the stuff diaspora sends, but it has worked with some hacks on both sides.

This discussion is getting kinda funny here because you are talking about a "diaspora protocol" as if this is something that even exists. diaspora never had a fixed protocol and even the third party networks never did the same we did. So apparently, not even third party implementers were interested in keeping the same standards. Also, at this state, I would like to point out that YOU, Jason, even liked our plans which we are realizing at the moment three years ago. The first discussions about moving the federation inside a separate module were started 2 years ago and even some code was written by Florian. Think about that for a second. I consider three years a pretty good time.

Yes, there were changes made in the new Gem which is actually already included in the diaspora core. You even commented on some of Benjamins pull request and never raised issues and now you act like you never had the chance to participate. I am a bit confused now.

All changes made to the current gem are made in consideration of keeping old diaspora instances alive and at the moment, no one actively involved in those steps has plans on removing said compatibility, because that would be pretty much stupid. In addition, every third party network implementing exactly those old implementations on old pods will still work fine.

If someone added dirty hacks (that were never used by us) to their protocol and now it breaks? Well. Too bad. We keep our old stuff but we are not able to support hacks made by everyone. This would be pretty insane.

Next, the term "stakeholders" which actually drives me insane. Maybe you should read the definition first. We are the development team of diaspora and this is our project. The diaspora users are the stakeholders and not some third party projects developers who are not involved in our development. This is not the W3C and this is not the W3C Social WG. If you want to create a protocol that can be used with all systems, you have to take everyones needs into account and that cannot and will never be done by a single project. You are part of the Social WG and you should know that.

I am a bit angry right now. What you are just trying to do is basically blocking our work on a more stable federation inside the diaspora network and bringing our stuff back to already defined and widely used standards. We have discussed about the importance of the federation in the past and you were on the same side and now you're basically saying we should not be able to make changes without asking everyone on this planet.

So, just to summarize: We (as in the people who have actively participated in the development of the federation modules in the past) will continue to move the stuff into an own gem while keeping compatibility with old diaspora pods. Some time in the future (and this is going to take quite some time), we will gradually deprecate old federation implementations, thus forcing administrators of old diaspora pods to update. This will be done after a long pre-announcement, but again, we are not even close to think about that.

I would like to add something personal. Right now, we are moving from a pretty crappy implementation to a well-defined and well-developed ecosystem. If we decide to move to just another randomized implementation because this is what the community decides, fine, but then the project has to progress without my support, I fear.

“YOLO code”

ROFL.

I agree. Maybe it could be time to write down a formal protocol not tied to diaspora and invite other participants at the table to work with us.

As you should be aware of if you want to take place in this discussion, diaspora never had or has a fixed, well defined protocol. Because of that, stuff is sometimes broken and writing actual documentation is impossible.

I'm sorry to say, but that's the worst excuse I've ever read for not writing a documentation. The actual lack of documentation is probably the reason why there has been so few people working on the project for 3 years. This really is a problem and this really is the main recurrent criticism I've heard since I've been contributing. And I find it pretty accurate.

The fact that de federation protocol has flaws and is not guarantied to work is not to my eyes a good reason for not documenting, not communicated and not trying to be more open towards other projects. Otherwise, I don't think diaspora can be called an open-source project anymore.

I'd be less strong on the Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable part than Jason. But stil, that doesn't mean we can't communicate. We are not autists, are we!?

@jasonrobinson Before I'm able to vote you will have to define "changes" and "diaspora protocol" first.

Since there is no formal definition of the protocol right now, for me the protocol is defined by our own implementation. Do you have a different definition? Would you define the protocol as the "union" of all protocols used by social network implementations that claim to be compatible with diaspora? (friendica, redmatrix, hubzilla, pyaspora, ...) Would you define the protocol as the "union" of all protocols that are accepted (but not automatically spoken) by the current federation implementation of diaspora?

To define "changes" let's take the example you mentioned. Benjamin started to fix some bugs in the federation code (like #2440) but kept the federation code compatible with the old one. Pods using some old d* code are still able to communicate with pods using the latest code. He extended the protocol to allow communication between pods which is closer to the standards we use. Is the extension of the protocol already a protocol change or do we change to protocol whenever some old implementation becomes incompatible?

Benjamin also added some validations that were fine for our own implementation but not for others. Depending on the definition of "diaspora protocol" he made the protocol incompatible with other protocols or with other "flavors" of the d* protocol which were different from the one we use.

@dennisschubert

Also, at this state, I would like to point out that YOU, Jason, even liked our plans which we are realizing at the moment three years ago.

Sure, exciting stuff, still support it. All I'm asking is to respect other adopters and talk with them - not do one sided work with a "fix yours if you want to" attitude. This is the attitude which still you can see as negative towards the whole diaspora project that was caused by the original founders not promising to respect any other software stacks.

You even commented on some of Benjamins pull request and never raised issues and now you act like you never had the chance to participate. I am a bit confused now.

I don't implement the protocol. As stakeholders here I am talking of the other software stacks.

Maybe you should read the definition first.

‘an individual, group, or organization, who may affect, be affected by, or perceive itself to be affected by a decision, activity, or outcome of a project

Sounds like Friendica and Redmatrix.

This is not the W3C and this is not the W3C Social WG. If you want to create a protocol that can be used with all systems, you have to take everyones needs into account and that cannot and will never be done by a single project. You are part of the Social WG and you should know that.

Yes, being a part of the SocialWG is exactly why I feel that diaspora is the real hope of decentralizing the social web. Tbh, I'm not too sure what the result of the SocialWG will be. Really, the diaspora protocol, or "not protocol" as you want it, is already unifying three different projects and with improvements hey maybe it could be a real unifying thing. I heard even the XMPP networks are planning on federating over some gateway using the diaspora protocol (no, sorry, not a protocol, a "thing"). I know some people from GNU Social have had thoughts at least on this. I know a wip python library that a certain image hosting site was planning to use to federate profiles on their platform.

None of this can happen if we don't collaborate. You guys, the three who are working on improving the protocol, are in the position to do awesome stuff - but instead today I hear that the plan is to just move thinking only of the diaspora Rails based software stack with no care about any other implementers.

You find this justified so you can iterate faster and talk with less people. I find this the death of the dream of growing the decentralized social web around diaspora. It's the death of The Federation.

What you are just trying to do is basically blocking our work on a more stable federation inside the diaspora network and bringing our stuff back to already defined and widely used standards.

No, I'm saying respect is needed and dialogue.

Right now, we are moving from a pretty crappy implementation to a well-defined and well-developed ecosystem.

Great, why not do it right then?

@steffenvanbergerem

No, the protocol (or non-protocol thing) is not so well set in stone that one can look at a change in code and say for sure "this will or wont break something".

But you're not seeing my point. I learnt today that the 3 people working on the improvements will not consult, hear or care about changes that affect software outside of the Rails based Diaspora software stack. There will be no thought on whether Friendica or Redmatrix federation breaks because "hey we didn't declare support for it".

I never said in the OP and sorry if it sounded like that that the changes recently done by Benjamin would have even been possible to see whether they break the integration. The fellows from Redmatrix didn't come blaming the diaspora developers and quickly Benjamin also submitted a patch to revert some of the changes. Even not doing a hotfix to restore breakage is ok I guess because only a part of the pods are affected.

But the idea that in the future, changes will be made to redesign parts of the diaspora non-protocol thingy with only diaspora in mind and not even slightly giving a damn thought whether someone else might be interested is just disgusting.

Are you @jhass and @dennisschubert really going to go so far as to say yes to the above statement, that you have absolutely no desire for collaboration with other networks, that your only interest is diaspora and only diaspora?

If that is so, I think this project is not for me. I don't spend as much time on diaspora as at least Jonne does, but I spend enough that I want to make sure it is worth it. I care a lot for diaspora, but I care more about working towards collaborating and networking together. That is the real future not some single software stack. Alone diaspora is not worth the effort.

I do not want to spend time helping other projects adopt our broken "protocol". Instead I want to move to a protocol that's worth of being adopted by other implementations, asap. And then help other projects adopt it. The current one simply isn't, and I'm sure anyone doing it is simply looking to reach our userbase, not because it's in any way neatly designed, standards compliant, reliable, flexible or full featured. So yes, in the current state I do not want to collaborate with other networks in the aspect of helping them to support our current implementation, I think it'd be toxic for them and wasting both our time.

If you remember NodeInfo I very well wanted to collaborate on things that are actually properly designed. But all I got was interest after there was an implementation to copy, that's not really collaboration and certainly not the kind of collaboration your proposal demands from us before moving forward.

If you remember NodeInfo I very well wanted to collaborate on things that are actually properly designed. But all I got was interest after there was an implementation to copy, that’s not really collaboration and certainly not the kind of collaboration your proposal demands from us before moving forward.

Yeah and once you actually got feedback all you did was complain that "it's too late you should have given suggestions earlier". Sorry, if you can't take feedback then you shouldn't work on stuff that you want others to implement. The same is I guess going to be with the future diaspora protocol then?

Well, my faith in diaspora is dying fast. If you really want to exchange your technical excellence quest to kill the user base it would be great if you did it with some other project than diaspora. If you really feel this then I fear we've fallen too far apart. Well, at least it means you don't consider my contribution to be worth much to even consider changing your opinion.

I got angry not because of the feedback, but because I asked long long before for feedback, got none, then spend time on an implementation and then got feedback that would require me to spend time to at least half rewrite my implementation whereas if I had gotten that feedback earlier I wouldn't have to spend that extra time. In short I got my time wasted, and yes that makes me angry. If you lack the empathy to understand that then we maybe really should just say goodbye.

This is not a new problem or one that is even limited to Diaspora. Communication protocols (of many kinds) get deployed and improved, and even radically altered all the time. The way we traditionally do this is to provide backward compatibility for a period of time, while providing some kind of indicator within the protocol that shows what revision or expectations this server supports. It's unrealistic in any decentralised platform to require all stakeholders to upgrade en masse. This argument would be valid in either a homogenous or heterogenous network. Even Diaspora has outlying nodes that would like to upgrade on their own schedule - not yours. "Be conservative in what you generate and liberal in what you accept" and with mechanisms for backward compatibility to the extent that this is possible is the way good software has always been written.

As an example I recall the transition from SMTP to MIME and to ESMTP. We've gone through several flavours of IMAP and LDAP and TLS and HTML and yadda yadda. All went smoothly. Some protocols have even gone through practically "overnight" change, like SMTP and XMPP. I'm just suggesting there are ways of managing these changes. Just quietly changing something in an incompatible way and without notifying other stakeholders has never been particularly well received.

As one of the mentioned stakeholders I will not participate in this vote.

I'm fine with Diaspora changing their protocol. There are a number of things that need work. You certainly don't need my approval. A warning and a basic idea of what things are changing would be nice so we know what to look for. Otherwise it's just rude to wake up one day with all your members screaming at you that everything is broken - because of some decision another project implemented and didn't even warn you about.

I had no idea five years ago that Diaspora would want to be a walled garden even though there have been plenty of indications and warning signs. You can have your walls if you want them. My sincere apologies for trying to break down the darn things.

I can't add much here. We have currently no "protocol". With my work at the federation-gem I try to change this and move to a more defined protocol with better documentation. And all this without breaking changes.

But I can't continue my work if I need to test against (and support) 4 crappy implementations instead of only one. And if I would, the result would be crap again, because I need to add even more hacks to support 3 more implementations (which we currently don't support). At the moment, the federation does not work really well and I want to change that as soon as possible (it was broken long enough). Or do you think it is more important to "try" to support friendica, redmatrix and hubzilla than a better federation for all diaspora users? And I think at the end also friendica, redmatrix and hubzilla will profit from this.

The federation-gem will be (and currently is) backward compatible, but only with the old diaspora-implementation. I can't guarantee that friendica, redmatrix or hubzilla doesn't break if they do something different than diaspora.

The only planned changes are not new: #2440 and #5963. And as I said, these will be added backward compatible. I will notify friendica, redmatrix and hubzilla before I remove the old implementation of these issues.

After we have a real definition, better implementation and documentation of the protocol, there will be a better basis to discuss changes or a new protocol. And I can't wait 8 months for every little change, only to get no feedback ... (see nodeinfo).

TL;DR: I am OK-ish with "Changes to the protocol must be pre-communicated with ALL stakeholders" (if communicate means "notify them, before we remove something old") and the currently planned changes are already communicated on GitHub (#2440 and #5963). So I don't understand why this proposal is needed right now? But I am not OK with this from the OP "Even if it is "YOLO code", coming from one developer, that doesn't mean it doesn't have to be supported, maintained and more importantly, not broken for ALL stakeholders.". I can only guarantee that old diaspora pods don't break (and not even that with 100%, because the old implementation is already broken). Otherwise, I can no longer work at the federation.

@dennisschubert The current implementation of the Diaspora protocol in Friendica was done in cooperation with Ilya Zhitomirskiy and @mikemacgirvin. I never even knew that there were made some changes on the Diaspora side to support Friendica. I thought the implementation in Friendica was a (perfect) copy of the part in Diaspora.

For me this "We are the development team of diaspora and this is our project. The diaspora users are the stakeholders and not some third party projects developers who are not involved in our development." sounds like:

"We will change stuff without any pre-warning and we won't even tell this other persons. Maybe they will be aware of the changes when the federations breaks. But even then we won't tell them what we changed because we really don't care about anyone else. Maybe we lose some thousand contacts but we don't mind as long as they aren't using our software."

All I'm asking for is a short notification like what is done in the issue 6405 where there was told what was changed and how to cope with that.

This really shouldn't be a blocker for a further development of the protocol. I really would like to do all necessary changes on the Friendica side - I only need to know what to change. Please give me some time and some piece of information.

@jhass sorry that I replied so late to your nodeinfo proposal. I was really busy in the last months (not only because of Friendica but also at work) so I wasn't able to do everything in time.

To be honest, it looks like neither @augier nor @mikemacgirvin and @michaelvogel have actually read the comments and the original proposal, which makes me angry as hell. I wrote that long comment for a reason. Thanks for voting and commenting without knowing what is actually going on.

I think we made pretty clear that we will be compatible with older diaspora pods (and thus with your old implementation, if it is 100% compatible). I also made clear that we of course will issue announcements if we ever will remove old compatibility layers, which - again - we do not have actual plans right now.

Jason basically proposed that we should not be able to do any changes without discussion it with third parties first. This is pretty clear by stating:

Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable

And I am absolutely not fine with that. We are all very busy and I have no interest in working on a project where we need to wait 8 months just to make a little change. That's killing everyone.

Again, we will stay compatible with older diaspora versions but as much as I like that other networks have implemented the current state, if something is not able to keep up with the enhancements because the implementations differ, then too bad. We're simply not able to keep everything in mind.

@michaelvogel: I think you mix two different things here:

1. The planned changes which are still backward compatible (the guid and public-key: #2440) and which I planned to tell you, before the backward compatibility will be removed and after I wrote a proper documentation.

2. #6405: the stuff that broke here was not planned, so I couldn't tell you before it broke. I added some more validations, and didn't know that friendica and redmatrix wasn't valid against them. I removed some constraints now (because diaspora doesn't use these fields at the moment) to make friendica and redmatrix compatible again.

There's nothing wrong with informing other projects about changes in the federation, but when the devs have to wait for confirmation from other projects, it becomes a problem. It slows down development, and it may hinder improvements because other projects don't have the time to adapt to the changes.

Last but not least. Either you officially support third parties or you don't. Keeping crappy implementations just

This isn't even about compatibility within Diaspora but about compatibility to third parties; however, when Diaspora developers don't want to wait for approval from other projects they might start a fork, and then we'll have the problem of making different forks compatible to each other. Friendica support is nice, but it's not essential; and without a nicely working federation, it's not particularly useful.

All I’m asking for is a short notification like what is done in the issue 6405 where there was told what was changed and how to cope with that.

I'm pretty sure we could all agree that a notification is nice. In that case, a meaningful commit message and well commented code should be enough to do that. Also (I'm not sure if Diaspora already has that) building documentation from comments may be the perfect way to do this. That way, third parties would have docs they could consult.

Last but not least: Either you officially support third parties or you don't. Keeping crappy code because you don't want to break compatibility and don't want to deal with actually helping those third parties adapt to the new code doesn't help anyone. That's called being lazy, and it's a disservice to third parties and Diaspora itself.

If someone really wants to make sure that third parties can support Diaspora, he or she should help document changes made to Diaspora and help implement them in third party clients.

@dennisschubert I read all comments here. You wrote "we will be compatible with older diaspora pods (and thus with your old implementation, if it is 100% compatible)" but you also wrote "Third party networks have implemented old diaspora federation parts in the past, but they have done a pretty bad job at it. The stuff Friendica sent in never was even close to match the stuff diaspora sends, but it has worked with some hacks on both sides."

So it seems that Friendica isn't 100% compatible. Like I said: The only thing I'm asking for, is a notice when a part is removed that only concerned Friendica. I really would like to make Friendica 100% compatible. But since I don't know Ruby or the project structure, I will not be able to find all quirks on my own.

I do not want to veto on changes. I only want a fair chance to make a change before stuff breaks.

We do not have exceptions for friendica but we were pretty bad at validating inputs and actually reading and using the standards we use in the past, thus, we are simply not able to tell when stuff breaks. This is why, for example, #6405 happened. This stuff is totally unpredictable. Requiring us to provide notices for unpredictable stuff is nuts and you should understand that.

I do not want to veto on changes. I only want a fair chance to make a change before stuff breaks.

So you disagree with the proposal then. Why did you agree?

Edit: Just to make sure everyone read what I wrote in my first comment:

Some time in the future (and this is going to take quite some time), we will gradually deprecate old federation implementations, thus forcing administrators of old diaspora pods to update. This will be done after a long pre-announcement, but again, we are not even close to think about that.

This is as clear as it gets.

I do not want to veto on changes. I only want a fair chance to make a change before stuff breaks.

Well, but that is what the proposal suggests.

Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable

I agree with @jasonrobinson that a protocol change should be communicated before it is done. That's why I agree with the proposal.

@dennisschubert if you add some more validation there is always a chance to break things, that's clear. Just by knowing that you are working on (for example) more validations to the "host-meta" data would enable me to compare the Friendica part with the Diaspora part. I could compare if we are sending the same content type, if the XML is really validating, if every information is provided, ...

And an information like "we will remove the data for xyz at this place, please take the information from that place" would help as well.

BTW: You wrote "... but it has worked with some hacks on both sides" and then you wrote "We do not have exceptions for friendica". The first sentence sounded for me like "we made some exceptions". Maybe I was misinterpreting it.

We're not talking about supporting a broken protocol. It's great that you are working on a better protocol and better federation, but the only question is if you can (in time) communicate relevant changes to the other projects in 'the Federation'. So these project can adapt and people on these project can still communicate with Diaspora users. Eventually it's all people who wants to keep communicating.

That said, I think it will be a splendid idea if Diaspora can officially make some kind of documented federation API available.

That said, a separate project/library for federation should be our final coal.

I guess we have the feedback of the people that matter.

Since it's commit messages that are need to be followed (according to Benjamin), maybe someone can start following the work so everybody interested doesn't have to do it separately. I'm not interested in trying to follow the things re The Federation any more (which I guess is dead on arrival) with the attitude on our side.

I guess also this means that we're not any more interested in the SocialWG work since, unknown to me, you're talking about making a proper implementation of the current non-protocol. I was under the impression, as a core project member, that we were only extracting the current code out and then possibly adopting something else like the SocialWG work. I guess before you spend countless hours on the protocol work it would be nice to discuss these things within even core team members - or is that just Jonne and Dennis now?

Thanks Dennis for your constant attitude regarding anyone else, including other core members.

I think I'll take a little breather on the 1-2 hours a day I spend on mainly promoting diaspora and the federation, and supporting diaspora users via various channels. If someone else wants to do some of that, let me know and I'll set you up with some access to the accounts. Clearly, consulting other core team members is not needed in this project where communication is not wanted by a certain part of the core.

Sorry for dragging you all into this. Dennis is btw the reason we are talking here because he refused to argue these points in private messages where this all started.

I’m sure anyone doing it is simply looking to reach our userbase

Not our userbase. The users. They don't belong to us. And no, I don't find revolting they are trying to build bridges between networks.

Or do you think it is more important to “try” to support friendica, redmatrix and hubzilla than a better federation for all diaspora users?

I think what he found more important was just talk. And I don't understand why it is making you angry...

To be honest, it looks like neither @augier nor @mikemacgirvin and @michaelvogel have actually read the comments and the original proposal, which makes me angry as hell. Thanks for voting and commenting without knowing what is actually going on.

You seem angry very often... And I thank you considering me stupid... I voted before you wrote your comment an even when I read it, I still find it irrelevant. Sorry you don't like it, but it stays this way.

Well, but that is what the proposal suggests.

Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable

I don't understand it this way. If one of the stakholders clearly don't give a fuck and never give any feedback, then ok, this won't be a veto and I really don't think @jasonrobinson has ever considered this way. But there's a huge gap between this point and @jhass or @dennisschubert's position of "I don't care about others. All that counts is diaspora".

that we were only extracting the current code out and then possibly adopting something else like the SocialWG work.

I had too.

I would propose a new mailing list especially for developers of all different projects who are coding with the protocol. This new list should exclusively be used for having a shortcut when (by mistake) something broke in the communication between the systems or when there are questions concerning the implementation. I guess this would be a very low volume list since mostly the communication is working fine between the systems.

@jhass Once again: I'm sorry that I participated that late on the nodeinfo stuff. Due to too much work on my side it went out of my focus. Additionally I'm getting lost in the various communication channels that are used at Diaspora.

this all seems to tie into the development of an API, i mean to some extent the application developers and collaborating networks are in exposed to these issues because there is no API for them?

No. The missing of an API has nothing to do with it. An API is for using your existing account on a system in another way (maybe from a mobile client). Here it is a question concerning the protocol.

@michaelvogel

Just by knowing that you are working on (for example) more validations to the “host-meta” data would enable me to compare the Friendica part with the Diaspora part. I could compare if we are sending the same content type, if the XML is really validating, if every information is provided, …

The relevant issue is #5114 and the repo for the gem code is SuperTux88/diaspora_federation. Maybe you could just follow that issue and we will leave a comment whenever there is a pull requests that removes federation code from the diaspora codebase and uses the gem instead? One could then discuss on github or on IRC if those changes will affect Friendica.

Of course, planned breaking changes (which make the protocol incompatible with old d* versions) will be announced early enough so you can adopt your implementation.

@jasonrobinson

Since it’s commit messages that are need to be followed (according to Benjamin)

Where did he write that? Benjamin wrote

The only planned changes are not new: #2440 and #5963. And as I said, these will be added backward compatible. I will notify friendica, redmatrix and hubzilla before I remove the old implementation of these issues.

I guess also this means that we’re not any more interested in the SocialWG work [...]. I was under the impression, as a core project member, that we were only extracting the current code out and then possibly adopting something else like the SocialWG work.

Benjamin is extracting the current code into a gem. While doing that he also started to fix some open issues. The SocialWG works is not done yet and we don't know if we can use it for diaspora*. Moving the federation to a gem makes it much easier to adopt another protocol later. We might even support more than one protocol, who knows. The SocialWG works sounds promising, but I don't think we should stop fixing our own protocol.

okay. well. i like the idea of a heads up mailing list. the devs make a huge investment in time and all contribute unique skills, so it is a shame when efforts nullify instead of being added together to improve my Favorited social platform :)

@steffenvanbergerem My problem is that I don't know any Ruby at all. I don't know the structure of Ruby (how the different parts are working together) and I don't know the structure of how Diaspora is working. Just having the the pull requests doesn't help me much.

I will follow that issue but will need explanations.

@jasonrobinson

Since it's commit messages that are need to be followed (according to Benjamin)

I have never said that, but if you want to be informed about every little change, then it is the easiest way to follow the development (I try to write informative commit messages). If you only want to be informed about breaking changes, then it is NOT necessary to follow the commits.

you're talking about making a proper implementation of the current non-protocol.

Yes, this is the first step. After the federation code is in its own gem, it is easier to replace the protocol or add a second.

The diaspora core developers do not want to communicate changes.

This is not true! But I can not wait for approval of everybody. I WILL inform everybody BEFORE something old is removed.

As I understand it now: friendica and redmatrix don't want a veto and we don't want to wait for a veto. They want to be informed before something is removed, and we will inform them before something is removed. Where is your problem?

@michaelvogel

I agree with @jasonrobinson that a protocol change should be communicated before it is done.

There is not ONE "before it is done". The protocol changes will always be in two steps: 1. add something new and still support the old. and 2. remove the old. The proposal was about inform you before step 1 and wait for your feedback, before we even do some changes. My plan was to inform you before step 2, while the old stuff still is supported. But before step 2 we have a working reference implementation, documentation and enough time for you to add the changes to your software.

Additionally I'm getting lost in the various communication channels that are used at Diaspora.

I don't know if it makes it better by adding another mailing list ;)

My problem is that I don't know any Ruby at all.

You don't need to know any Ruby. I will inform you about the big changes. And if you want to be informed about every little change, then it is maybe indeed better to follow the commits. But the commit-messages should say enough for you to see if it is maybe important for you. And for example the validators are easy to read: webfinger and HCard and if you have any questions you can always ask me. There is also a documentation of the gem where the entities are documented (webfinger and HCard) with deprecation warnings. But also here, you can always ask me, if you have a question.

Dennis is btw the reason we are talking here because he refused to argue these points in private messages where this all started.

So are we ad hominem now? You wrote a private message to Jonne and me claiming we do decisions without involving the community. I refused to discuss this in a private conversation you started and take it into the public as you wished. And now I am the bad guy for bringing this into the public? What is wrong with you?

I am seriously shocked. I thought we had a nice collaboration going and now I feel like you played me for a sucker. This whole discussion which started as an important discussion for this project turned out to be a campaign to make people look bad. This is a very uncool move, Jason. I am disappointed and I am very sad about this has come so far. What's your point of doing that? Seriously?

We're now discussing in an endless loop about stuff we already do agree (that is, announcing breaking changes before doing them). We never planned to break stuff without announcing it but you claimed otherwise. Now other people think we're breaking stuff without notice and thus accepting this proposal without getting the core of it. Your proposal is not about communicating changes, it is about waiting for everyone else to say "okay, you are allowed to make changes" which will kill this project.

Why did you even close this vote? You complain about our lack of community involvement and now you are the one closing this discussion? The outcome you set was not even part of the proposal. I thought I knew you, but this is an action I'd expect from a troll or someone who is seriously trying to hurt the community and not from a valuable community member.

I did not even block the proposal because I would be able to accept the results. I made clear I would not be able to justify my time anymore when the project decides to accept your proposal, but I never said this is something that could not be done.

I made it clear before, but as long as I am part of the development team (that is, as long as there is no proposal against me), I will add my opinion to discussions. In fact, you even asked Jonne and me for our opinions and we made it pretty clear. And now you're not able to accept it because you think we are wrong? And because you think we are wrong you are not able to discuss in a peaceful matter?

This is just sad.

@supertux88

This is not true! But I can not wait for approval of everybody. I WILL inform everybody BEFORE something old is removed.

Great, I like the "everybody"! Why did you disagree on the vote then? Maybe you can talk Dennis and Jonne to agree with you then. Since they only promise Diaspora support, not recognizing any other implementers.

@dennisschubert

I am disappointed and I am very sad about this has come so far. What’s your point of doing that? Seriously?

I guess I'm just tired of your constant hostile attitude and this can be considered just tipping over. I opened this proposal to get some kind of promise that there is no need to get back to this subject but as you basically presented a big NO brick wall as response - well, no point in discussing then is there?

We never planned to break stuff without announcing it but you claimed otherwise.

You specifically said privately and publicly you will not care about other than Diaspora implementers - is that correct or not? You also defined no need to support external services, by for example doing a patch when federation breaks. I even said I would do the work. But to you anything outside Diaspora is not important.

Why did you even close this vote?

I closed it so that it will not pass. So that you will not stop work on the Diaspora protocol, as per your very strong ultimatum. I think that would be worse for Diaspora than forcing the vote through.

So your ultimatum worked.

And now you’re not able to accept it because you think we are wrong? And because you think we are wrong you are not able to discuss in a peaceful matter?

You're not discussing - you are putting your foot down with a strong opinion which I tend to disagree very strongly on. The Federation is more important than Diaspora. You and Jonne think it is not important at all and you at least said clearly if you have to consider federation with the other implementers, you will quit working on the protocol. That is a pretty damn strong statement with very little discussion in it.

I strongly agree work has to be done on the federation protocol. I totally support moving it out to a gem. I want it to continue.

What I don't agree with is your "fuck all the rest" attitude when it comes to anyone else.

I'm not the easiest person to deal with but you are a fucking brick wall. When there is no dialogue, why try? I'm at least not going to waste any more time on it. You guys are much more important to Diaspora than I am.

Currently I don't see the point of promoting The Federation as it clearly is not something that is important for all the project members.

Great, I like the "everybody"! Why did you disagree on the vote then? Maybe you can talk Dennis and Jonne to agree with you then. Since they only promise Diaspora support, not recognizing any other implementers.

I disagreed because I don't agree with "Deliver a plan of changes being done to stakeholders when those are planned or latest when work begins on them" and also I don't agree with this from the OP "Even if it is "YOLO code", coming from one developer, that doesn't mean it doesn't have to be supported, maintained and more importantly, not broken for ALL stakeholders."

And I think Dennis and Jonne agree with me here, because Dennis wrote "This will be done after a long pre-announcement ..." in his first comment. The "we don't care" (and this is not a "fuck all the rest") is because we can't guarantee that the other networks don't break because the current protocol is unstable, and so we can't support them officially. We don't plan to break it without announcement, because all planned breaking changes will be backward compatible for some time. But we can not guarantee that any other "non-breaking-change" doesn't break something on their side (like #6405, this was not a planned breaking change).

I think you are mixing these two things here: 1. planned breaking changes (this will be backward compatible and communicated early enough) and 2. unintentionally broke something (we can't guarantee that this won't happen again with third party networks at the moment)

I agree with you, that we need to officially support friendica and redmatrix, but we are not there yet. We need first a defined protocol which works stable, and then we can support this protocol. If others then implement this protocol, they are automatically supported (if they implemented it correct).

I agree with you, that we need to officially support friendica and redmatrix, but we are not there yet.

What does "support officially" even mean if Friendica and Redmatrix are the ones supporting Diaspora? That wouldn't even be possible. I don't even know what that would mean and I certainly haven't requested it or seen anyone else request it.

What I see is a lack of will to collaborate at all, even to the level of being hostile, unwelcoming and publicly announcing not caring at all. And no, I'm not talking about you.

Maybe things will grow to be better but tbh I don't have that much faith. Take NodeInfo for example, again. Jonne writes the spec for it and asks me to pass it on. I pass it on at some point afaicr, but didn't have any immediate concerns with it. Later, we agree that it is time to implement it. Then when people actually give feedback, it is too late and not welcome.

I fear the same will just happen with the protocol changes. There is just no interest to ease interop. Any comments will be shot down as "not your business, implement if you want but we don't care" - and I wouldn't be surprised if the exact words wouldn't be just those.

Well, these are my feelings on the comments made by certain core developers here and in the afore mentioned private discussion, and before too and they are frankly killing my interest in contributing to the project at the moment.

I'm not going away (sorry), or trying to hurt the project, but I think I am allowed to say what my feelings are towards things, to statements being made and if I'm unhappy about something and the people I think are important to the project don't give a damn - well what can I do?

Please, do what you want with the protocol stuff, I'll stay out of the way and do something else, directly or indirectly related to diaspora. The-Federation.info expires in December - I'll decide by then if it is worth to keep the domain or not..

What does “support officially” even mean if Friendica and Redmatrix are the ones supporting Diaspora? That wouldn’t even be possible. I don’t even know what that would mean and I certainly haven’t requested it or seen anyone else request it.

With "support officially" I mean that after we have a well-defined protocol, we can support this protocol, so that we can guarantee that the federation with third-party-networks doesn't break.

Take NodeInfo for example, again. Jonne writes the spec for it and asks me to pass it on. I pass it on at some point afaicr, but didn’t have any immediate concerns with it. Later, we agree that it is time to implement it. Then when people actually give feedback, it is too late and not welcome.

Jonne waited 8 Months for feedback, and nothing came. Then he released the 1.0 Version, and THEN the feedback came ... I can understand him. BUT he has taken the feedback and added it to the already released 1.0 version. It was OK, because nobody already used the 1.0 release (only diaspora), but normally you can not change something that is released. So it is correct, it was "too late". But it was not "not welcome".

I don't really see your problem. NodeInfo and the try to create a stable well-defined protocol goes towards "The Federation" not away from it.

Great, I like the “everybody”! Why did you disagree on the vote then?

I'm pretty sure most people disagreed because of

Do not merge in changes before listening to all the stakeholders and noting their concerns with implementation and timetable

No one here disagrees with informing others about breaking changes.

I find it great that changing a button style or placement in the UI requires an ok from just about everyone but asking people for opinion on changing the heart of diaspora and the federation is "too much effort". The proposal doesnt say "wait 8 months". Jonne wanted to wait 8 months before merging NodeInfo in - I never understood why he started blaming me for what happened then.

You know what? Maybe you guys are not even the know everything re improving the federation code? Maybe, just maybe, code review could help? Except if you dont have any respect for the other parties, which I fear is partly the case here.

I find it great that changing a button style or placement in the UI requires an ok from just about everyone but asking people for opinion on changing the heart of diaspora and the federation is "too much effort". The proposal doesnt say "wait 8 months". Jonne wanted to wait 8 months before merging NodeInfo in - I never understood why he started blaming me for what happened then.

That is not correct: a UI change doesn't need the OK from everyone. BUT everyone is welcome to give feedback, and if needed, a loomio thread is created. The same applies to the federation gem, EVERYONE is welcome to give feedback, and some people already have done this, and I respected their feedback. But I can not wait until everyone agrees with every little commit. The proposal doesn't say "wait 8 months", it says "Do not merge in changes before listening to all the stakeholders" with no time limit, this can even be longer than 8 months? The only thing I have "changed" (it is still backward compatible, so nothing really changed yet) so far is #2440, this issue is open for almost 4 years, do you think I need to wait longer?

Sorry, I don't understand the second comment, who should review which code?

@supertux88 Like I already said, I totally forgot about the nodeinfo stuff. When it was short before release I heard of it again (I guess @jasonrobinson pointed me to it) and implemented it in Friendica. By doing this I realised two things that could have been improved to it. One of these things were realised by @jhass very quickly. (I really would have understood it if he just had told that this would be changed in a version 1.1 but not 1.0)

I definitely need persons in the background who constantly point me to things like this so I don't miss them. (Something like a project manager) For this we need persons like @jasonrobinson. Without him I guess I will miss very much. I simply can't concentrate too long on a single thing and I can't overview too much things at the same time.

I'm really sorry that @jhass waited so long for a reply to his proposal and that it came so short before release.

@michaelvogel don't worry so much, the issue wasn't that you gave no feedback, the issue was that nobody did. And with this history a proposal like this which demands to wait for feedback from everyone that could possibly be interested for code we never officially declared third party support for and that has design flaws which need to be addressed is just unacceptable.

@jhass I guess we could had agreed to something like "notifying all stakeholders and giving them a fair chance to reply and to listen to their concerns".

Of course this can't mean waiting for several months.

Concerning this "not officially declared third party support" we should all accept that the connection between the systems is a fact.

I just had a short look. In my data I found about 240 active Diaspora servers, 112 servers running RedMatrix and Hubzilla and 395 servers running Friendica. (Of course there are much more Diaspora users than Friendica users because Friendica servers are mostly used by only a few people)

All these systems can interact by now.

The number of developers who are working on internals on these systems are low. At Diaspora there are 2 or 3 persons (AFAIK) working at the communication core. In Friendica it's mostly me and on RedMatrix it's mostly @mikemacgirvin.

So I guess that in a "case of emergency" it should be no problem to speak with each other. And I guess that in this phase of the communication we all can learn about what could be done, once the protocol is really well defined and there are (hopefully) masses of developers from others systems willingly to implement the protocol.

All these systems can interact by now.

They can interact by now, more or less stable ... and I can't guarantee that this doesn't break, because the implementation on the diaspora end is buggy and undefined, and I don't know the implementation at the friendica end. But I try to stay backward compatible as much as I can.

If you want to be informed about every little step, then it is the best, when you follow the federation-gem commits. But if you don't have that much time, then I will inform you when you really need to change something (because we are removing backward compatibility) and give enough time to change it.

So I guess that in a "case of emergency" it should be no problem to speak with each other.

If something unexpectedly break, then of course we can talk to each other.

I'm happy with some warnings when you know there could be compatibility issues, but also a bit more sensitivity when it comes to fixing issues - waiting a month to revert something that breaks federation in a big way isn't a suitable outcome. I'm not happy adding a salmon declaration to our XRD to keep it working because we don't support salmon. (And neither does Diaspora). I'm not happy adding a url alias because we removed lrdd some years ago (and Diaspora never implemented it). Both are a huge support burden for our over-stretched devs, as is Diaspora itself, but our members seem to want Diaspora; even though half of the useful features they insist on having won't work with Diaspora. I need to figure out how to make post/comment edits and nomadic identity work with Diaspora or we might have to disconnect - we can't support our core features across that platform and this is a HUGE support burden for us. So I understand that trying to maintain compatibility with "reverse engineered Diaspora" implementations on other platforms is a support burden for you.

If the API was complete we could just use that and be done with it. But in the end we have to respond to our members and fix their problems. Whether you like it or not, we and many of our members are all members of the Diaspora community in some degree, just as you are members of our community. We need to work together to figure out how to lower the support burdens for everybody. Theoretically all you need to do is maintain a consistent protocol for us to be happy and attempt to provide backward compatibility as you move forward. As we've discovered that isn't always as easy as it sounds.

On our side we have to find ways to support stuff that you haven't even thought of yet and hence don't have any support for. We have to somehow turn all of these things into something that maps to the Diaspora protocol - or our members complain. It's daunting. I've got no idea how to support magic-auth or nomadic identity on Diaspora and I've been struggling with trying to find a way for a few years now. I also don't know how to make event participation work from your side. Maybe we could parse keywords in comments. We did get forums working across platforms (at least public forums). And polls are easy. I'm mostly waiting for somebody to want those bad enough to do the work, as I'm tired of doing it all.

I sort of question the need to go back now - four years later and fix issues with the XRD hacks Diaspora initially put in place. (I mentioned these to Ilya long ago and he was aware of and trying to figure out what to do about them before his unfortunate demise). XRD has been obsolete for three years and most everybody has moved away from it. I'd rather spend the time converting our Diaspora stuff to rfc7033 so we won't have to waste resources with 5 or six different URL fetches for every handle lookup. That and scraping HTML webpages for info. Yuck. Adding unimplemented stuff to or requiring unimplemented stuff in the XRD or moving it to an hcard we have to scrape is really missing the point and just making work for everybody. Nobody uses that stuff anymore. Anyway I'm not being confrontational - just trying to provide my viewpoint so perhaps we can all understand each other; even if some of us don't get along and have deep hatred towards each others' projects. If I had the deep hatred of Diaspora that some think I do, I wouldn't have implemented cross platform federation on three different projects. Frustration? Absolutely. Hatred, no.

@supertux88 Following the commits isn't something I would like to do - especially because I won't be able to exactly see what has changed.

Is there a documentation of the used endpoints? (Like /.well-known or /p?) I would like to compare them with the equivalent implementation in Friendica to make it more compatible.

BTW: I agree with @mikemacgirvin that the scraping of HTML pages is a strange way to fetch information. I really would suggest something more structured like XML or JSON.

P.S.: I had some suggestions for the protocol. Is Loomio the right place for it?

I disagreed because I don’t agree with “Deliver a plan of changes being done to stakeholders when those are planned or latest when work begins on them” and also I don’t agree with this from the OP “Even if it is "YOLO code”, coming from one developer, that doesn’t mean it doesn’t have to be supported, maintained and more importantly, not broken for ALL stakeholders.“

How would you call that if not communication? So, we agree, you're not even willing to communicate.

That is not correct: a UI change doesn’t need the OK from everyone.

Is that a joke!?