Kendraio
Hello All,
To our surprise, we are building a "data browser". A personal mashup engine: an antidote to big data: #littledata. The App plugs into any API; bi-directional. The App's API clients (Adapters and Flows) are scriptable configuration (not code) that can be easily modified and shared across communities. Kendraio App is now mature enough to show you. And I'd like to do that. So, please get in touch.
When we started building Kendraio App we thought we were making an asset/rights management dashboard app for artists/managers integrating with service provider APIs. So far we have integrated with YouTube, Spotify, Teosto (Finland's CMO), DDEX (strictly speaking a protocol not an API), Bandsintown (an event listing service), MusicBrainz (a complex wiki for music data) and more in the pipeline.
Then we realised that the tools we are creating could work in any environment and any sector. Hence #databrowser. We are building interface first. Human centric. Not service centric. Hence a local app. For people.
We'd like you to challenge us, my Kendraio team. We still have some funding from the EU (until August 2020). We'd like you to set us tasks. Could be to integrate an API. Could be to model a protocol. Could be to build a dashboard for a sector or service. Go wild! Go blue sky! Challenge us!
This is alpha software but we are already operational and working with real APIs/protocols and real people/organisations. If you want to take a look see:
• Online demo: https://app.kendra.io
• More information: https://www.kendra.io/kendraio-app
• Read the documentation: https://kendraio-app.readthedocs.io
• For help and support, join the conversation on Slack: http://slack.kendra.io
• To download the source code, visit the repo: https://github.com/kendraio/kendraio-app
• To report bugs, suggest improvements, or request features, use the issue tracker: https://github.com/kendraio/kendraio-app/issues
This a time limited offer (if we don't get more funding beyond August 2020). But there is still time to develop amazing worlds. Let me know what you are up to and how we can collaborate.
Cheers Daniel
Daniel Harris Sun 12 Jan 2020 2:46PM
Hello @Bob Haugen , thanks for all your awesome links and information. Yes! However, I'm realising, as much as I'd love to obsess over the finer details of the subtleties and variations of how people interact with each other, there are some blockers for me in doing this: mostly time.
I want to realise (make real) and get adopted a system for self organisation. I am no scientist. I'm a dreamer. I'm not a developer. And I find myself being rather one tracked in my interactions. Do I feel a sense of time slipping away? And therefor giving myself less meandering time?
Yes, @Lynn Foster I think this
might end up being in the killer app category
too. But how does one get other people to see that? And what does one do once they do? I've been exploring this idea for over 20 years and there's a big trick that I've been missing. It doesn't matter how good the thing is, the goodness doesn't, in itself, make for success: creation and adoption. So, this is what I'm drawn to: the art of manifestation. I don't know how this will be built or adopted. But I do know it needs people to work together and feel emotionally involved. So, now after building a very rudimentary alpha demo of Kendraio App, my task is in fostering working relationships around it.
It doesn't really matter exactly what those working relationships are just as long as the combined thing has (perceived) benefit to both parties. So, yes, @pospi let's build the simplest and quickest thing first: the last one on your list:
inventory management.
Let's start doing something that we can collectively build on.
I guess I say all this because I want to give you all background on my current thinking and focus. And perhaps this is an apology for not being able to dive into semantic detail on any one workflow, and why this necessitates me being effectively dismissive of more complex ideas at this stage in the game. I am focusing on the mechanics of manifestation, creation and adoption. This area is so complex. And human psychology (tribal allegiances and energetics) has such a huge part to play here too. And I feel that part of our collective emancipation is dependent on us naming the human influence on success and bringing clarity and light to it. And once we understand it, use it to our collective advantage.
Anyway, all that said, let's play – and build little toys that people can interact with. Boom! 🎉
Praise be to APIs! 🙏
@pospi and I will report back after our Tuesday call. Onward! Cheers!
pospi Tue 7 Jan 2020 10:52PM
@Daniel Harris there are a few bits of UI that we're keen on developing as initial projects... it's hard to get a gauge of what is useful in terms of pushing Kendraio forward but I can at least try to describe them a bit in terms of our needs. We can expand more when we chat next week (:
Flow visualisation / authoring UI. There's a fairly long thread talking about the requirements for this visual design language here. That's a pretty big project with complex UI requirements, but if that's the sort of thing you want to push ahead with then it's a great candidate. I would start with "view" before moving on to "edit", but that's just me.
The "personal agent app" @Bob Haugen is talking about is also quite a big project, but definitely easier to build. Mostly just forms. An older project which was building towards this can be found here for comparison / exploration.
For our own needs, and to get us dogfooding the software sooner, I'd really like to see a work logging app. The queries to support that might be quite complicated though, because you'll have to "think in VF concepts" when mapping queries to the simplified UI that a Kanban board offers. But @Lynn Foster can probably help with that (;
The easiest one to do is probably inventory management. That only needs to query resources and create events that manipulate them.
Hope that helps to give some sense of our immediate goals!
pospi Tue 7 Jan 2020 10:42PM
The spec for the ValueFlows API is at https://github.com/valueflows/vf-graphql/ if you're interested in integrating it. That will give you some stuff to work with in Holochain REA networks (core features only at present, but work is ongoing), as well as eventually the ActivityPub. While that would be very cool for us, I do wonder whether the present state of the project makes it worth your while. You probably want something production-ready to plug into, huh? In which case https://thegraph.com/ would be a worthwhile candidate because then you can do Ethereum stuff. Which ones to pick would depend on your ideological positioning, but things like voting systems, token-based community currencies & donations could be useful to some?
Bob Haugen Tue 7 Jan 2020 7:26PM
@Daniel Harris That list is describing a Personal Agent app. You can read more at https://github.com/valueflows/vf-apps-agents/issues/2
Lynn Foster Tue 7 Jan 2020 7:13PM
@Daniel Harris I like @Bob Haugen 's suggestions more than where my head went first (project level visualization). It fits well into your statement "The stated aim of Kendraio App is to provide a dashboard for a person to better manage their digital life: whether their business life or their personal life.". I'm not too good at predicting this kind of future, but I think the "personal dashboard" idea with control of your data and access to whatever your life involves might end up being in the killer app category. :)
Bob Haugen Tue 7 Jan 2020 6:44PM
@Daniel Harris here are some ideas about how a person might want to interact in a social+economic network:
https://cryptpad.fr/pad/#/2/pad/view/64P7o1b-7n8WCppvlhlHoC6xqs6TYqgfuQsE7rrMxn8/
The basic ideas there are that people want to:
send messages to each other, which could be purely social, or could also be economic.
post economic actions like recording their work or what they created.
see all the economic actions that they could be or have been involved in. For example, all the available offers and requests, or all the work they have done or exchanges they have participated in.
comment on economic actions where their comment links to the economic action, and the presentation of the economic action shows the comments that have been posted about it. This would similar to the usual comments on a blog post.
see and manage all the resources they are responsible for or have access to.
see where they fit into their larger contexts, which could be an organization or a process flow visualization or a map. The cryptpad shows an example of a process flow, and here's an organization visualization: http://holodex.enspiral.com/
Daniel Harris Mon 6 Jan 2020 2:58PM
What kinds of visualizations can you or do you want to show?
Anything that a human would do. How does a human interact with a scenario? I mean the human user interface. Like a set of screens. Mockups. Anything?
The logic is not so important to me at this stage. What is important (to me) is what it looks like to an end user, and what they can functionally do with the interface. Even in the roughest sense is fine. Any scenarios spring to mind?
cc @Lynn Foster @Bob Haugen @pospi
Lynn Foster Mon 6 Jan 2020 1:31PM
>For completeness, @Lynn Foster has been defining validation logic for ValueFlows apps.
Just looked and this is not really very usable, github issues with lots of discussion. I'll get it on my list to pull it into a real document somewhere, with notes as to what is not nailed down.
Bob Haugen Mon 6 Jan 2020 12:32PM
Re visualizations: I did a bunch of experiments with them in http://valueflows.pythonanywhere.com/
For example, http://valueflows.pythonanywhere.com/vocab/egg2worm-flow/
I can help with back-end logic.
Lynn Foster Mon 6 Jan 2020 12:17PM
>What I would now like to do is identify a practical thing we can do together. You've already seen we can build forms from the ValueFlows schema. What next? What else do you want to visualise? What else do you want the user to be able to do? Maybe think of something specific in one realm/domain and we can riff on that theme?
@Daniel Harris yes! ❤️❤️❤️ What kinds of visualizations can you or do you want to show? There are some structures in VF that would seriously benefit from some good visualization. For example, the economic activity tends to form directed graphs. One thing people have wanted for a while is something to fill the same requirement as a gantt chart like for visualization of your project or some piece of work. Or, what other kinds of UIs are you most interested in building? But @pospi let's chat on the side and see what you think makes sense for HoloREA projects, and might fit into the UI component approach....
Daniel Harris Mon 6 Jan 2020 11:25AM
Good information @pospi . We are not bound by what JSON Schema or GraphQL support. We can find other ways of encoding configuration for rules or validation or error or interface. For example UI Schema is another up and coming way to add interface cues to JSON Schema when rendering in a form. So, we can play other tricks like that.
What I would now like to do is identify a practical thing we can do together. You've already seen we can build forms from the ValueFlows schema. What next? What else do you want to visualise? What else do you want the user to be able to do? Maybe think of something specific in one realm/domain and we can riff on that theme?
Daniel Harris Mon 6 Jan 2020 11:11AM
@pospi Can you think of one GraphQL API that you would want us to integrate into Kendraio App? Something that is close to what you are doing? Something that would be useful to you to visualise? Something that we could perhaps work on collaboratively now?
Would be really amazing if there was an open source version of something like this!
Could that be what we are doing? Just a possibility given the resources. OneGraph looks great. Nice name too. 🙂 We've been thinking about integrating these iPaaS/iSaaS providers into Kendraio App – to give us a 'leg up' so to speak. I've forked a list in one of our docs. Feel free to add any others you can think of at Kendraio Adapter API Registry | API gateway providers .
pospi Mon 6 Jan 2020 12:45AM
It's not just the ValueFlows community that is aligning with GraphQL... it's a trend in distributed systems in general. Scuttlebutt folks just built PatchQL and there are projects like https://thegraph.com/ in the blockchain space. There has also just generally been movement away from RESTful APIs towards GraphQL over the last 5 odd years since Facebook started marketing the tech more.
If you connect to an existing backend, validations will probably happen there and you'll get an error message back if something won't work.
Check. The nice thing about GraphQL is that it has a standardised way of handling errors. So that means you can integrate upstream errors from the services you're wrapping pretty easily into your UI.
I think json-schemas may have some syntax for validation beyond required fields
They do, but I'd debate the usefulness of implementing them. There might be reasons to avoid back-end round trips for invalid payloads if dealing with blockchain systems (very long request times); but for other stuff like Holochain and Scuttlebutt there's really no reason to since the "backend" is on the local machine anyway and requests are pretty much instantaneous. There are quite a few oneOf
type validations in ValueFlows (which I know JSON Schema supports) - you could potentially move those into schema validations, but IMO it's just another place you have to maintain. GraphQL itself doesn't support validations that are any more complex than "required"- its goal is to be a minimal backend-agnostic schema layer and so it foregoes a lot of this kind of power for flexibility.
For completeness, @Lynn Foster has been defining validation logic for ValueFlows apps. Most are quite simple, but there are also some complicated ones which need to check against several related records.
pospi Mon 6 Jan 2020 12:22AM
Awesome. It'd be worth checking out https://www.onegraph.com/ btw- they have done a heap of work to integrate an array of web2 services into a single query API. Would be really amazing if there was an open source version of something like this!
Daniel Harris Sun 5 Jan 2020 7:52PM
Re: ValueFlows, Kendraio, Validations and Tasks...
Hi @Lynn Foster
One question I have has to do with validations and making sure what gets saved makes sense and will work. Let's say you are creating form and list interfaces from a specification, like VF in json-schema and graphql are on the table in this discussion. The economic stuff is a bit less forgiving than say your basic social interaction, and needs to be way tighter than "anyone can say anything on the web".
So, how does that fit in with how you think about your work? If you connect to an existing backend, validations will probably happen there and you'll get an error message back if something won't work.
Yes, if not right now, we definitely will be able to handle error codes sent from a remote API endpoint that is doing the validation logic.
But, I'm wondering, and don't have the experience to know - I think json-schemas may have some syntax for validation beyond required fields, but I'd have to look again, never actually used them. I don't know what can be done along those lines for graphql? The only one I wrote relied on compiled code for that.
And we can already do type and pattern matching validation logic within the local App and this can expressed either within JSON Schema or UI Schema, I can't remember right now.
Maybe the question is, what if the thing you are doing on the web is more than what you have observed most of it is, lists and create/update/delete? With apologies if I missed something you said about that.
Ah, but that's actually an entirely different question and one that probably deserves a whole new post in itself but I can't be bothered right now. ;-) So, I'll answer here...
The short answer is that we'll create an interface to let you interact with the service that is performing the logic of the "thing you are doing on the web" that "is more than what you have observed". ;-)
One of the things I really want to have in the App is a list of all the tasks (human and machine) that I am involved in on the various services I connect into. Maybe a GitHub issue, or a complex P2P search, or uploading my latest song release to 10 different online stores, or observing the state of holochain/blockchain transactions. So I can see progress as they happen (or not/fail). Not sure if that relates to what you are asking but the question inspired me to mention what we are calling Tasks – instances of Flows. Maybe. We're all working it out. ;-)
Daniel Harris Sun 5 Jan 2020 7:22PM
Re: ValueFlows, Kendraio and GraphQL
Hi @Lynn Foster
I'm bumping this to the top because comments were getting lost at the bottom. If I was viewing this in a future Kendraio App (or similar) then I'd plug into the Loomio API and I'd be able to set the rules about how comments were listed and in what order. Oh heaven! Anyway...
Right now most of the people using ValueFlows are using GraphQL, partly due to @pospi 's influence on several projects. And it's a nice tool. So that means you could potentially plug something from your user interface(s) directly into some soon-to-be-existing-at-least-to-experiment-with "backend" and it would work.
That's great. I spoke to Kendraio lead developer and he says that we can do GraphQL without much/any changes. ;-)
Lynn Foster Sun 5 Jan 2020 3:59PM
>PS: I think "sybmol" is spelt wrong but I don't have the guts to change it on GitHub. 🙂
Fixed it 😬 - thanks @Daniel Harris !
Anyhow, I got through the whole thread, and appreciate all the thoughts from everyone. As for the big picture, I understand what you're doing could be a piece of the "let's make it lots easier and safer for users to connect into the economic networks we choose" or something like that. Economic as in trading, sharing, creating together, using, consuming, etc. etc., with a goal of being within our ecosystemic bounds and outside the bounds of surveillance capitalism. Or something like that. Little bits now, so we can create some fair tools that will work larger scale when we need them. I hope and think that fits with what you and @mike_hales are both saying.
But, I'm gonna be focused mostly on the details. 🙂
>Would it be better for us to speak native GraphQL?
I don't know, and you may want more than one interface, but here are some thoughts and questions, maybe for both you and @pospi.
Right now most of the people using ValueFlows are using GraphQL, partly due to @pospi 's influence on several projects. And it's a nice tool. So that means you could potentially plug something from your user interface(s) directly into some soon-to-be-existing-at-least-to-experiment-with "backend" and it would work.
One question I have has to do with validations and making sure what gets saved makes sense and will work. Let's say you are creating form and list interfaces from a specification, like VF in json-schema and graphql are on the table in this discussion. The economic stuff is a bit less forgiving than say your basic social interaction, and needs to be way tighter than "anyone can say anything on the web".
So, how does that fit in with how you think about your work? If you connect to an existing backend, validations will probably happen there and you'll get an error message back if something won't work. But, I'm wondering, and don't have the experience to know - I think json-schemas may have some syntax for validation beyond required fields, but I'd have to look again, never actually used them. I don't know what can be done along those lines for graphql? The only one I wrote relied on compiled code for that.
Maybe the question is, what if the thing you are doing on the web is more than what you have observed most of it is, lists and create/update/delete? With apologies if I missed something you said about that.
Daniel Harris Sun 5 Jan 2020 11:47AM
@mike_hales So many vectors. So little time. ;-) Some random thoughts on what you said.
I'm enjoying you posts. I did ask for a challenge I guess. ;-) Every other word I'm having to look up! ;-)
Remember, we may be trying to solve different problems. If we are to know if something is "appropriate" or not then we need to know the context/aim/goal before we can judge. I'm not sure of your goal yet. What's your goal? In simple terms. I mean, what do you want your experience to be? Yes, I think that's the better question. The human experience?
For me, I am trying to solve the problem of app bloat for users and duplicated functionality across multiple services and lack of data portability for users. It is from my (the user) point of view that I am coming at this. Hence my first interaction with any service is via an interface. I have a gut feeling that this approach may contribute to a much bigger goal of freedom for individuals to decide how they live their lives but I'm not sure yet.
Don't get me wrong. I love architecture. My Dad and brother are architects. And I am not advocating one method over the other. We all have different skill sets. I was never going to be the one to build an open protocol for media/music but I was really good at getting people together to think about the problem for a while. And then I realised my skills might be far more about adoption strategy. We can build the best technology but if it don't get used then what's the point? So, now I'm working in another way: develop a little, integrate real people/organisations, develop, adopt, develop, adopt. In Kendraio's case the adopters are now driving development. And that's what I want. Also, in some ways, this is much more of an art methodology of the journey. But it's a bit of both.
There is not only one way. All ways are stronger if they interconnect how they can. Each can help uplift the other. That's my strategy for Kendraio: get as many adopters as I can and make it work for them. It's paying off.
Things we can talk about in our call:
Interface vs architecture: both are necessary and needed but not necessarily in that order. ;-)
Software development: I have a hunch that development is not efficient. There is a tendency to build discrete chunks that can be branded and marketed. This is just as much the case for FOSS too. Most likely due to ego and money. If I just say "yes, like, I'm just interconnecting everything" then it's hard to sell the idea to a funder (either grant or investment). But if you say I have a thing and it's got a nice blue badge then it sells like hot cakes. Yum. Yum. But is that the most efficient? I don't think so.
One tool vs many tools (monolith vs interoperability): interoperability is hard and takes agreements between people – which is always hard. So, much easier to build a tool to rule them all. Much harder to get all existing tools to work together. I think I've found a middle way.
Goal orientated vs art orientated: err... both?
Ground, ground, ground!: I'm really cautious about working with developers. My experience has been mixed. Suffice to say I now have a rule: whatever you do make sure it works in the app so I can test and demo it each day. Of course I'm flexible with the rule. But I'm very cautious of letting a feature take ages to make work. And I'm all for keep it tidy as we go. All this takes longer. Many developers love to hide away for months and then come out with the thing. That doesn't work for me because, though we have alpha (experimental) code. We are live and have users and I am demoing every new feature every new day.
Look forward to our call in a month.
Cheers
mike_hales Sat 4 Jan 2020 12:58PM
interface first . . interaction is about listing stuff they have or want, performing an action on it, adding a new one or editing an old one. And I want the same interface no matter what service I am interacting with
My intuition is signalling some kind of hacking aesthetic here (which I resonate with -bricolage, maybe. Something fundamentally materialist, stuff-oriented. Not unconnected with the WIMP-desktop paradigm). I have an intuition that practice within a ValueFlows network - organising the provision of X in a manifold of supply possibilities, from here, now - would have something of this kinaesthetic quality of handling stuff. Equally intuitively, I have a sense that the experience of working with federated wiki has something of this in it. Maybe that’s why I brought the fedwiki thread into the OAE forum.
But @Daniel Harris I don’t think it’s appropriate to speak of this as ‘interface’. Seems to me, this is basic architecture for a profoundly powerful kind of tool, needed in a world of basically ‘wild’ or feral material. The One Tool to Rule Them All. Well maybe not! But at least, a Swiss Army Knife. I’ve been searching, all my computer-using life (user and active configurer, not coder) for this kind of bricoleur, palimpsest-making, scrapbook-compiling, hypertexting toolspace. ‘Interface’ is such a mistaken construct here. This is basic prosthesis-design for life in a world of feral digital material? A prosthesis can be regarded as interface, I guess. But for the ‘wearer’ it’s Self? It’s the means of constructing the entire practical world, in actual, hands-on, autopoetic practice. Interface, schminterface 😉
Daniel Harris Sat 4 Jan 2020 10:29AM
@Jon Richter Thanks for all the great pointers. There is some really good synergy with all these.
I really must update the repo descriptions and Kendraio App Features page. We are no longer using Drupal for main development. Our main website is Drupal but that's not where our main development is happening. We are using Angular for Kendraio App. We could have used any PWA framework but for various reasons choice Angular. We are building an App first and foremost. It has to be able to run locally and in offline mode. There may be a scenario where each person has the Kendraio App running on their multiple devices and headless in the cloud – each acting as an agent for the personal collective.
In the Kendraio world, Adapters and Flows are configuration (currently JSON) not code so no need to push a new version of the App (through the app stores) to integrate with a new service. We are not integrating any service provider SDKs – which they are so fond of producing – that way lies bloat and we have to stay lean.
We are building the App: interface first. This is about humans doing what they do now but better and more efficiently. So this is much less about automation and much more about how humans interface with their online life. One thing we've found is that most human interaction is about listing stuff they have or want, performing an action on it, adding a new one or editing an old one. And I want the same interface no matter what service I am interacting with. And I want to control the interface.
In my ideal world I have one "App" where I manage all my online interaction. Yes, I know this is just an analogy for a semantically aware world. But that world is not there yet. So, we have to "fake" it. But I think the world is there enough (with APIs) to do this. And regardless I want to control my own data and my own interface.
And yes I am now intrigued about how we align Adapters and Flows with other projects which have equivalents. And at least have a way to translate between them.
Yes I think that #smalldata is better than #littledata. I was looking for the opposite of "big" but little could be sparse but small could be lots of tiny things and, of course, it iss beautiful. 🙂
Jon Richter Fri 3 Jan 2020 10:42PM
I second the notion that it is small data that we are aiming here for.
I'm receiving interested feedback by practitioners from the field, so your app seems to resonate with people out there.
For me personally, the technological underpinnings are of much more interest. I see that you remained on the drupal path (kendra_home, kendra_hub)and use its API mode to drive a web-native app, neat.
After reading through many of the links, it appears to me you are building an automation platform/workflow engine like Apache Airflow, grappe.io / https://github.com/assemblee-virtuelle/Semantic-Bus, using serverless "lambda" components on now.sh as adapters in an ETL pipeline, which brings specific features for the music business domain.
Indeed the Signpost with mapping vocabularies similar to what dswarm.org / demo.dswarm.org does is of high interest.
What would a data browser do for you, i.e. visualise data as Apache Superset does?
I'm interested in seeing where this conversation goes, esp. with aligning data models and protocols.
Daniel Harris Fri 3 Jan 2020 12:14PM
Hi @pospi thanks for all the info. I will see if it makes sense to me but I'll also pass this on to my devs. My takeaway is to implement GraphQL in Kendraio App! ;-) In terms of coding/language/framework, I wouldn't get hung up on it. Yes, we are building in Angular right now. It's given us some speed of development. But I don't think we are religious about it. One point of interest might be: we have taken a solemn oath not to include a service provider's SDK within the App. Otherwise we'll bloat before we've even started. So, I think I'm correct in saying, as far as we're concerned we are only interested in API/protocol definitions and endpoints and secrets and we'll configure the Adapters and Flows with all the logic. That way we can add new service provider APIs without the need to push an update through the app stores. That way anyone can add or modify the Adapters and Flows and share them to others. Cheers!
Daniel Harris Fri 3 Jan 2020 11:56AM
@Lynn Foster @pospi reading between the lines, it could be that if Kendraio App could speak GraphQL then it may alleviate the issue. I'll check out with my devs if that is feasible. Would it be better for us to speak native GraphQL?
Lynn Foster Fri 3 Jan 2020 11:49AM
@Daniel Harris and all, sorry I've been totally tied up for several days, but will happily wade through everything here soon and catch up. Just a quick note. I don't mind adding titles to all the data elements in the VF json-schemas. Manual atm (as you can tell by "sybmol", aack), but that's ok. :) Or maybe we'll come up with something automated, thanks for looking into it @pospi .
Daniel Harris Fri 3 Jan 2020 11:48AM
Hey @mike_hales I'd love to see your design blueprint. Do you have mockup screens of what it looks like? I mean how do people interact with systems in your utopia? I'd love to see what those look like. If not, I can help you design them, yes? It's from the human interface that I understand what people are talking about.
mike_hales Fri 3 Jan 2020 11:47AM
Thanks Daniel. I like the idea but I have a monthsworth of thinking and writing commitments that I have to get some order into, before I can afford to go into free fall 😕 Fun, another time. Do come back on this.
Daniel Harris Fri 3 Jan 2020 11:44AM
Cool @mike_hales . I would like to a call and document party with you. It's where we have a Zoom call and talk through the various vectors that have come up here and write these down in the document and give our opinions on them. I think it would be useful and fun for both of us. Yes?
mike_hales Fri 3 Jan 2020 11:02AM
Thanks Daniel for taking this time and moving through my load of stuff. Lots to digest - and I likewise have a lot of slow reading and responding on my plate. So perhaps no more responses from me for a while - you’ll note that while you were replying I added further points in other comments!
I appreciate the manner of your responding and likewise hope for dialogue and practical alignment, if not agreement or full approval. For now, best wishes.
PS Metaphysics . . I don’t invoke ‘masses’, quite the opposite: “the future is a pluriverse” of many lives lived in many ways. It’s just that I promote a mutualist, not an individualist, engagement. And ‘realism’ isn’t supposed to mean buckling-under to what’s dominant, it’s supposed to mean the very largest (poetic, dreaming) imagination that mobilises absolutely all capabilities and takes the fullest account of how big and weighty (and often toxic) stuff in fact is even if it was well-intended and seemed to be under control. It’s a pessimist trouble-scanning layer on top, with a profound optimist layer beneath.
mike_hales Fri 3 Jan 2020 10:43AM
I certainly get the intent, and the need, for ‘small data’ tools that an individual actor can use to ‘scrape sense’ off-of platforms that were not designed to present the desired, mashed-up perspective and meet that particular need, in that particular location. That’s how the world is, most of it wasn’t designed to meet the particular need that you or I happen to have right here. But I don’t support the basic anarcho-libertarian, object oriented ideal of a universe of monad-agents, each autonomously owning and explicitly (perhaps algorithmically) manipulating a system of private assets. To that extent, I regard the whole semantic web project as deeply problematic.
It’s not that we don’t need such tools. But this is not the world imo we need to live in, it’s the world that we need to escape from (of privatisation, of small vision, of fantasies of 100% control over objects via super-smart objects that scrape and crawl the web). So . . we also need the equivalent of Big data delivered by public platforms, which face platform users (publics, confederations, communities, activist formations, mutuals, associations) with mirror-analytics on the entire collective and its major components and the place within that of the individual actor, rather than serving platform owner-exploiters only, as current, commercial-platform, Big Data is designed and funded to do.
It’s called Commons. Everybody contributes to what is in the commons (data, explicit metadata, tacit traffic data), everybody mobilises & enjoys what is commoned, everybody directs and regulates the commoning: not as a pyramid of autonomes but as mutualised participants in collectives, in which stewarding and mutual aid rather than ownership and control are primary. Commons of data and metadata are not at all the same thing as universes of powerfully tooled-up individuals? Commons of skilfully stewarded material are what our species and other species need, not a fully digitised supermarket of traded property and tooled-up shopper-vendors. That’s what online commodity trading does, for goodness sake - and who trusts that trustless system?
We need both projects running alongside each other. Where are the big data commons projects?
Daniel Harris Fri 3 Jan 2020 10:40AM
Hi Mike,
This looks intriguing, thanks Daniel. After checking out your links, here are a couple of Qs. Pardon my lack of understanding, I’m a culture hacker and ’new economy’ activist, rather than a code hacker or tech wannabee.
I really appreciate that you took the time to investigate. Thanks for all your questions. I'll go through one by one.
I don't think there's one way to get to utopia. And there is safety/resilience in many ways being pursued simultaneously. And better still if all those "ways" interconnect and are complimentary and overlap too.
I am a slow reader and writer. I tend to read through everything many times to check and recheck. So, if you send me a big message then it will take me a while to respond. That's why I prefer small messages. It's also why I prefer building documents collaboratively rather than messages/email. I love talking about the content of a document – then we are collaboratively building a common creation. And then that document becomes a resource for others to use/read. Just so you know.
Your questions are deep. But there's no way I can give you all the info you need in one shot.
Me? I don't code. I studied art for one year at Camberwell. Didn't do a degree. Phew! ;-) Then worked in video as a lowly tape-op. Then started an ISP in the mid 90s with some friends. Lucked out. Sold it. Been Kendraio and many other projects ever since. Yes, I've spread myself thin. And that could be frustrating for people. But I like relationships and building them. That's what I'm doing now. Onwards...
When I found the Internet I fell in love because it was the same peer to peer construct that I'd been creating with interconnecting matchstick sculptures. Then realising that the Internet is an open protocol – a language. I fell in love with that too. Then I thought we should have open protocols for everywhere. Then I spent 20 years trying to get the media/music industry to build an open protocol. But along the way I realised that it wasn't really the open protocol part that I was interested in but the benefits and freedoms that it brings me and others as sovereign human beings. I want to be in control of my life. So, we started building interface first. And realising that the Semantic Web was difficult to build I wondered how we could get there another way. By meeting people at their own doorstep. Luckily, many companies have APIs but they are all using different objects and methods. What if I could do the translation within my own app? Then they wouldn't need to change...
• What kind of organisation are you? A coop? A startup with commercial venture funding? What kind of ownership and governance structure of the company, and what kind of assets? (Obviously code product is open source.)
Kendraio Foundation is a UK company limited by guarantee (not by shares). Hence, there are no shares to sell and so no investment can be made. I am the sole director and sole signatory to the legal construct. Assets are just code and all open source on GitHub. Yes, that can bring up governance issues but we'll deal with that when it becomes a deal breaker. It hasn't yet AFAIK.
D’you have a business plan
No, it seems like we are working from more of a place of having a grand vision and seeing what iterative steps can be taken to get there. But the grand vision is not fleshed out totally in any one place. So, yes, I get your comments about lack of transparency. It's not intentional. More just the way it has grown.
and does it involve core revenue
No, no revenue model. Phew! I can pay developers if I have money from grants, sponsorship or donations. And that's that. We are funded by grants and sponsorship and when there's no money development stops and we take a break. That's the story so far but hopefully people will see the value and the universe will provide consistent funding from now on. Funding so far:
from specific services added on to the opens-source code, which are labour- and skill-intensive or rooted in proprietary process knowhow?
Hence: nope.
Why is info on Kendraio-the-venture (which needs material resources and funding) - as distinct from Kendraio App, the code repo - not on your website?
Because no one ever asked before. And also because it's very very simple. But it's there:
https://www.kendra.io/orgs/kendraio-foundation
Is it that you believe that open source code output is a sufficient definition of the nature of a project, on which folks might decide to foster links and contribute collaboration?
I don't believe or not believe. There is only so much time in the day and we are generally responding to who makes suggestions or asks questions. So, thanks for yours.
imo this is not so, the story is a bigger one.
I agree. I have seen far too many "open source" projects being sold (they were hosted by for profit companies) and then forked by the community leaving multiple and duplicated effort. Which I guess is OK given my earlier statement about multiple ways. But it just seems like wasted effort.
• What would a user actually do with the app and its scripted variants? This doesn’t seem obvious. “A personal mashup engine” is a description that seems wthout an obvious meaning - mashing-up what, why, what for?
You're right. It's not obvious. If you'd tried explaining a web browser to people in the early 90s they won't understand. Look up IPaaS. I only found the term a week ago. I'm learning so much on the job. These IPaaS are systems that businesses are using to connect with each other: to interoperate. I now realise that I want to bring the power of those IPaaS systems to individuals and groups – from cloud to app. Also, have a look at IFTTT and Zapier. These are building blocks. But they are all services and so can be switched off. I want to use tools that can't be switched off. As long as the Internet stays switched on. ;-)
A dashboard “for rights owners . . enabling users to manage and track their digital media assets [and] collaborations” is a description here that’s not transparent to someone who isn’t in the commercially-exploitable rights business. For example, not transparent to a commons developer
Do I? Don't I? Do I? Don't I? Go down the rabbit hole of conversational techniques? I could let it go but then is that an opportunity missed? Have you asked me for this? I feel into my body and I am nervous because I don't want to create antagonism. So, I'm taking a chance, that we can talk about "talking about". Because I find it fascinating. I am interested in the power of language and how we use it. So this might be off topic but so be it. I find I expand when asked a question. And I contract when told about other people who think or react in some way to something I'm involved in or have responsibility for. As I understand it "invoking the masses" is a conversational technique that is used to gain power by proxy where an unspecified number of unspecific people are used to justify an argument. The technique is based on the fact that these "people" are not in the conversation and they cannot be questioned nor negotiated with – they are an immovable block. But we're just having a gentle conversation and finding out about each other. I value your opinion. I don't need you to justify your opinions by citing others that agree with you, just so you know.
That said, any open source developer knows exactly what rights are about. Oh dear, am I invoking the masses too? ;-) Open source licenses are deep legal constructs that only work because there is are solid legal frameworks in our world.
All that said, I hear that you don't understand what the heck I'm talk about. And that's OK. We can work together to put it in language that you can understand. I'm up for doing that work.
It is the choice of any person to decide how they want to live their life. If they want to sell their creations then great. If they want to give it all away then great. If they want to do a bit of both then great. I want and desire people to be free to decide who they are and what they do. And I want and desire to assist in that process.
or a person who believes that digital media can’t be exploited as commercial assets without being tied to associated, chargeable, custom labour services or process technologies that are supplier-specific, and in that sense non-public, if not actually proprietary.
There's a lot of juice in what you are saying and I really hope that we can get to a point where we can get an understanding going because I think it would be beneficial to both of us.
I'm not trying to tell anyone what they can do with their creations. I love all art and I want to support all artists/creators in doing what they want to do. In fact, I want to support all people in self organising, whether as individuals or collectives or sometimes in between. I want to see interoperability and data portability and open protocols and APIs everywhere. I want people to be able to do what they want to do. I want technology to enable that freedom of expression and ideas.
• What documentation exists, for the broader ‘new’ interpretation that you offer above, of Kendraio “in any environment and any sector. Hence #databrowser”? There doesn’t seem to be any such documentation on your site, which seems to be cast in terms of the original narrower vision.
Yes, that is true. I'm only selectively talk about the bigger vision within environments where I know they will be more appreciated – like this one! Also, this revelation that what we are building is bigger than originally thought, is very recent. I just haven't had time to update much text. But also I'm cautious and I would much rather have conversations like this right now than launching big bold statements that I'm not even sure are true. So, peer review and peer testing is what I want to do now.
impute a generic relationship between #data and ‘rights/assets’: how do you make this bridge? ‘Rights/assets’ is a quite specific (propertarian, individualist) framing, while ‘data’ typically invites a far broader, less value-laden interpretation.
There are many opinions. It's all data I guess and some people will see value in some data. And maybe they'll share one of their home grown carrots for a song?
Is the app only useful for folks trying to trade in (develop revenue streams from) digital media within a larger, dominant, commercial, digital media market?
I guess that's for us all to decide. I see that there is a possibility that the app (and apps like it) could become our main interfaces for dealing with services we use.
Most people making most money these days are doing through large aggregator constructs. I want to allow them to experiment with new, more fluid, ways of doing things whilst still working with these old/current constructs. An easeful bridge rather than an all or nothing leap.
Is the #data you refer to (to be browsed by media asset-owners using Kendraio App) the specific spectrum of data published in the apps/interfaces of market players in rights-based media industries, accessible thro their APIs? Or is it the data accessible thro the API of any platform - in which case, this needs some use cases before a nonhacker might begin to imagine its significance?
Yes, we need use cases. Yes, we need collaborators. Yes, we need to have conversations. Yes, this is a journey of discovery.
For example, instead of me buying stuff from Amazon – yes I use Amazon, it's so darn convenient, sorry – I want to connect to 100,000 shops directly from my own private app. And it's very doable. Most of those shops have an API because most of them use an off the shelf shopping/cart system and most of those have APIs.
Maybe that’s what you’re inviting us to provide: use-cases, where Adapters and Flows might be scripted and widely shared? In that case, I feel Adapters and Flows need more description than your site provides, of a more expansive, less tech kind: use-case oriented rather than oriented to API script-hacking.
What I'm really enjoying at the moment is building lots of mini projects with people. Some of these are with organisations. Some of these are with individuals. In each case we are creating use cases which we can build too. That's what gives me the most joy: actually building what we are dreaming about with more and more people and the app becomes more and more useful. It is through building and then using what we have built that inspiration happens and lessons are learnt. So, we are learning about where we are going at every iteration. It's art I tell you! ;-)
Overall, I have a sense of Kendraio as a venture that’s basically code-oriented (with UI-hacking apparently at the core) rather than oriented to the various practical worlds of non-tech, non-dev folks. And perhaps oriented to attempts at revenue thro the enclosure of digital media assets - or of auxiliary services associated with such assets - rather than to the construction of commons of digital media. Am I mistaken Daniel, in either of these perceptions?
I don't want to tell you or anyone how you should live your life. What I want to do is provide you with tools to make it easier for you to make choices. Is this tool going to help you make choices? How will we find out? It's a ride.
It would be great to feel that there was a core here, of technology for person-centred, locally focused, ‘small data’ derived by data-scraping from corporate web platforms, and easily useable by non-hacker (non-scripting) users in diverse communities,
What would it "feel" like? What would that feeling feel like? And what would give you that feeling? Can you describe it in human terms? As in: my hand touches a screen. And drags an icon. And creates a search. And shares it with others... If we can dream about what the interface does then we can build it from interface first.
in significant opposition to the Big Data approaches of the platform-capitalist corporations we love to hate.
I don't hate anyone or anything. I love everyone and everything. We are all doing the best we can given the information that we have to hand. I'll grant you that sometimes it seems that the "ones in power" seem to have very limited "information" indeed. ;-) And, yeah, I think they do. But we are all humans. And if we want to live on a planet that has a human sustaining sustainable biosphere – and I do – then we all have to win. The strategy of how we get to utopia is just as important as the blueprint for utopia itself. The means creates the ends. The energetics of positive change are extremely sensitive – everyone has to feel like they are winning. Otherwise people feel resentful and the whole cycle starts again. We need balance. Love is the only way here. For sustainability. That's logic by the way. Not hippy emotional. Just raw logic.
Is the potential of Kendraio really THAT big?
What if I could shop the world through my own interface that I could configure to look how I wanted? What if I could change all my notification settings for all my services in bulk on one page, with one click? What if I could see/filter all my outstanding issues/tasks on all my platforms in my own app? What if I could administer all my financial accounting in one app that I was in control of (that wasn't a service)? You tell me!
Is it the case, that you need more applications before you know the answer to this?
Yes.
Isn’t there something in the huge algorithmic capability and traffic-scraping throughput of the corporate back-office systems and server farms, that makes for a huge assymetry here?
If I understand your question then there is a lot on enterprise tools to riff on.
It would be good to be realistic in our expectations?
Interesting statement, no question, no statement, no question... Whatever it is, it's interesting. Again, I'm calling you on a conversational device called "let's be realistic". ;-) No, let's not! At all. Ever. Being realistic is limiting and keeps us in our boxes. We cannot know what is possible. But to dream is a wonderful thing. And to be lucky enough to build the dreams is incredible. Artists don't start a creation with the idea of being realistic. They dream big. And sometimes get some of the way there. And also, let's not have expectations. Expectations only lead to frustration when they are not met. Instead let's have desires. Let's dream big.
And also, let's not do anything one way. Diversity is safety. ;-)
Cheers Daniel
mike_hales Fri 3 Jan 2020 10:11AM
Seems to me that Kendraio Signpost may be of more significance than Kendraio App. Specifically, this seems to be where your speculation about the large impact of the app/architecture might be founded, outside the digital music sector, as regards #data and semantic web in general. @Daniel Harris tell us more about Signpost and translating between ontologies? Ontology is certainly a big deal in ValueFlows @Lynn Foster? Is there documentation of Signpost? Is Signpost funded along with App, or a distinct project, with or without funding and partners?
mike_hales Fri 3 Jan 2020 9:47AM
@Daniel Harris How about a link to your EU funding source/program/grant? Cheers
Later: thanks for your funding link Daniel. You’re doing a whole bunch of Euro grant juggling there! For me, one project and six partners was enough to do my head in! Lots of Southern and Eastern Europe I see - still a strong EU criterion for funding. Do they feel like major partners? - I mean, like, I sense there’s a whole bunch of stuff going on in Turkey.
pospi Fri 3 Jan 2020 2:37AM
Thanks for connecting us with this work... it's very exciting! Kendraio looks like the kind of effort that could likely save everyone months of effort, if not years (:
I can see how the project has evolved in the way it has, and I share your perspective that implementation and experimentation at the UI level is where most innovation is likely to occur in the next few years. As you say- interface first, human centric, not service centric. You might be interested in this article from one of the Holochain core devs, or some of our team's reflections on what this implies in terms of system modularity and UI application dependencies.
More broadly, along with ourselves most distributed systems projects I'm seeing are converging on GraphQL as an API interface. This is a great trend as it means that plugging together disparate systems is going to get easier. I have also observed that network-to-network protocol bridging carries a high technical complexity and yields narrow progress. Examples of this can be seen in the development timelines that yielded Ethereum-to-Bitcoin transaction bridges, for example: very slow work for a single pairing, let alone multiple integrations.
So my educated guess is that most multi-platform apps will handle integrations between distributed systems at the UI layer. Much easier to translate between two data formats at the app level than trying to make different internals talk to each other. And that gets easier again if apps are being built to align to the same GraphQL schemas to begin with... indeed, it's just a few lines of wiring to combine multiple data sources once the foundations are laid, and no other application code needs to be modified outside of the schema bindings.
One of the other main goals of the ValueFlows UI project we're just now starting is to remain platform-agnostic. We don't want vendor lock-in at any layer of our stack. It looks like Kendraio is using Angular, and though we aim to support Angular, we want to do this in a way that can also integrate with React and other ecosystems. I have started feelers into the Svelte community to explore what a truly universal UI component build system might look like, but have been unimpressed with the responses so far. We're happy to build Angular or React apps whilst prototyping, but eventually we want this framework to deliver packaged React, Angular, Vue & WebComponents (at a minimum).
pospi Fri 3 Jan 2020 2:06AM
Hey @Daniel Harris (: This is really exciting, and prompted me to do something. I haven't pushed one to NPM yet, but if you clone the vf-graphql
repo and npm run build
you should now end up with json-schema.json
that contains a definition for the entire API. I have noticed it doesn't seem to generate title
attributes for fields (found 2 different libraries to handle the conversion on NPM; neither added them)... you might want to consider a pre-processing step that injects an "uppercase first" version of the field name as the title for the form to pick up?
mike_hales Thu 2 Jan 2020 7:27PM
This looks intriguing, thanks Daniel. After checking out your links, here are a couple of Qs. Pardon my lack of understanding, I’m a culture hacker and ’new economy’ activist, rather than a code hacker or tech wannabee.
What kind of organisation are you? A coop? A startup with commercial venture funding? What kind of ownership and governance structure of the company, and what kind of assets? (Obviously code product is open source.) D’you have a business plan, and does it involve core revenue from specific services added on to the opens-source code, which are labour- and skill-intensive or rooted in proprietary process knowhow? Why is info on Kendraio-the-venture (which needs material resources and funding) - as distinct from Kendraio App, the code repo - not on your website? Is it that you believe that open source code output is a sufficient definition of the nature of a project, on which folks might decide to foster links and contribute collaboration? imo this is not so, the story is a bigger one.
What would a user actually do with the app and its scripted variants? This doesn’t seem obvious. “A personal mashup engine” is a description that seems wthout an obvious meaning - mashing-up what, why, what for? A dashboard "for rights owners . . enabling users to manage and track their digital media assets [and] collaborations” is a description here that’s not transparent to someone who isn’t in the commercially-exploitable rights business. For example, not transparent to a commons developer, or a person who believes that digital media can’t be exploited as commercial assets without being tied to associated, chargeable, custom labour services or process technologies that are supplier-specific, and in that sense non-public, if not actually proprietary.
What documentation exists, for the broader ‘new’ interpretation that you offer above, of Kendraio “in any environment and any sector. Hence #databrowser”? There doesn’t seem to be any such documentation on your site, which seems to be cast in terms of the original narrower vision. You impute a generic relationship between #data and ‘rights/assets’: how do you make this bridge? 'Rights/assets' is a quite specific (propertarian, individualist) framing, while ‘data' typically invites a far broader, less value-laden interpretation. Is the app only useful for folks trying to trade in (develop revenue streams from) digital media within a larger, dominant, commercial, digital media market? Is the #data you refer to (to be browsed by media asset-owners using Kendraio App) the specific spectrum of data published in the apps/interfaces of market players in rights-based media industries, accessible thro their APIs? Or is it the data accessible thro the API of any platform - in which case, this needs some use cases before a nonhacker might begin to imagine its significance? Maybe that’s what you’re inviting us to provide: use-cases, where Adapters and Flows might be scripted and widely shared? In that case, I feel Adapters and Flows need more description than your site provides, of a more expansive, less tech kind: use-case oriented rather than oriented to API script-hacking.
Overall, I have a sense of Kendraio as a venture that's basically code-oriented (with UI-hacking apparently at the core) rather than oriented to the various practical worlds of non-tech, non-dev folks. And perhaps oriented to attempts at revenue thro the enclosure of digital media assets - or of auxiliary services associated with such assets - rather than to the construction of commons of digital media. Am I mistaken Daniel, in either of these perceptions?
It would be great to feel that there was a core here, of technology for person-centred, locally focused, ‘small data’ derived by data-scraping from corporate web platforms, and easily useable by non-hacker (non-scripting) users in diverse communities, in significant opposition to the Big Data approaches of the platform-capitalist corporations we love to hate. Is the potential of Kendraio really THAT big? Is it the case, that you need more applications before you know the answer to this? Isn’t there something in the huge algorithmic capability and traffic-scraping throughput of the corporate back-office systems and server farms, that makes for a huge assymetry here? It would be good to be realistic in our expectations?
Bob Haugen Thu 2 Jan 2020 7:15PM
I took a look, but will wait for Lynn or Pospi to respond with more intelligence. In the meantime, thanks a lot, it's a nice demo of your form builder.
But but what, you don't like sybmols? 😉
Daniel Harris Thu 2 Jan 2020 5:52PM
Hi @Lynn Foster . I'm looking at ways we can take your ValueFlows schema and integrate it with Kendraio App. By way of an initial example I'm taking a JSON representation from your GitHub repo and seeing how our form builder interprets it. If you click on this you'll see what form is generated. If you click on the cog you'll see your JSON schema driving the form. Like this:
What would it take to add titles to all of the JSON elements in the schema? Is it a purely manual thing?
PS: I think "sybmol" is spelt wrong but I don't have the guts to change it on GitHub. 🙂
cc @Bob Haugen @elf Pavlik @pospi
Lynn Foster · Sun 12 Jan 2020 2:55PM
>It doesn't matter how good the thing is, the goodness doesn't, in itself, make for success: creation and adoption.
That is an overriding point, isn't it? We do have to build for people who already want to use the app and are ready to work with us as users. Maybe a good way to simplify our thought processes too.