Follow

Question: could an AP implementation of exist? I can see many advantages, but I am not aware of the tech limits.

Ideas?

@Antanicus

You mean distributing changesets or diffs over #activitypub ?

@61
Something like that, yeah. A federation of servers where hyperlocal data reside, maybe even maintained by local councils

@Antanicus

Ok I see. That is an intriguing idea indeed!

@Antanicus I would really really like to see some way to "decentralise" #OpenStreetMap (I think there's "POSM"?)

But I'm unsure really *how* to do it, or *how* #ActivityPub fits into it, or how to make AP work with #OSM. (Partially I don't know enough about AP).

@rory @Antanicus If you can just send geojson or other osm-compatible node data over ActivityPub and let users incorporate that as personal subscribed (and locally backed up? maybe retransmittable? or refined with tags?) map layers, it should do the trick.

@feonixrift @Antanicus 🤔
#OSM doesn't have layers, and doesn't use geojson type format.

OSM tags are like geojson properties, but the geometry is topology, and references other objects. e.g. ways/lines are made up of ids of nodes/points. So doing a "distributed" "ids" is hard...

@rory
Yeah this is the kind of problems I have no knowledge about but can potentially prevent any form of decentralization within OSM. I'd love it if this simple question evolved into a more elaborate thread

@feonixrift

@Antanicus I think that the internal chat could be easily federated. Or no? @rory @feonixrift

@Ca_Gi
@Antanicus @feonixrift @rory

Are we talking about end-user federation or node-to-node federation?

I note that end users are rarely direct consumers of #openstreetmap data.
“The Map” is the huge #Postgres database serving the data or alternatively, the planet.osm dump or its diffs.
This data is served via APIs to various consumers and also used to produce raster tiles (via another DB) - 1/3

@Ca_Gi @Antanicus @feonixrift @rory
@rory
Some actors also process planet.osm into their own custom formats (e.g., #OsmAnd, #Geofabrik maybe?) which are then distributed to consumers - 2/3

@Ca_Gi @Antanicus @feonixrift @rory
@rory
Consumers usually receive:
* Raster tiles (e.g., openstreetmap.org/ visitors)
* Data from various APIs (e.g., #Nominatim, #Overpass users)
* Vendor-specific data bundles (e.g., #OsmAnd, #Maps.me)

@rory can you confirm my understanding of the above is reasonably accurate? - 3/3

@Ca_Gi @Antanicus @feonixrift @rory

Where it says “Consumers usually receive” it should say “End-users …”

Think of the end-user as the person using an application that presents, in some form or another, map data coming from #OSM.

@Ca_Gi @Antanicus @feonixrift @rory

What is probably not too much work is to get #OSM servers to broadcast updates to each other. Please see ultra-dodgy modification of the component diagram (original here wiki.openstreetmap.org/wiki/Co) for a graphical explanation.

@Ca_Gi @Antanicus @feonixrift @rory

In theory this could also be done via #Postgres replication, but with less control by admins as to what gets replicated. You probably do not want someone's DROP DATABASE to replicate across all instances. 🙂

I do note that the #federation protocol could be anything that does the job, perhaps as easy as a plain old rsync, HTTP PATCH, whatever. It doesn't have to be #ActivityPub.

@61 @Ca_Gi @Antanicus @feonixrift 😆 wow that's complete and complicated. It almost lists every piece of #OpenStreetMap library under the sun.

For "federation", it's on the bottom left corner that's relevant really

@rory

The real question is, who else is running a full(-ish) #OSM instance that openstreetmap.org can federate with?

@Ca_Gi @Antanicus @feonixrift

@61
I believe this is a self-reinforcing problem: OSM is hard to federate, therefore nobody runs full(-ish) versions of it, therefore no developer puts effort into "federalization" and the loop starts again

@rory @Ca_Gi @feonixrift

@Antanicus

Why do you say it's hard to federate? Planet.osm allows anyone to run a full instance, reasonably well synchronised with the main OSM server.

Is it not possible that the core OSM setup (Postgres + Mapnik or whatever covers your needs) looks hard to install and maintain and that's why there aren't that many known instances about? I don't know, just speculating.

@rory @Ca_Gi @feonixrift

@Antanicus @rory @Ca_Gi @feonixrift

Anyone here with hands-on experience setting up an #openstreetmap instance?

@61 @Antanicus @rory @Ca_Gi @feonixrift yes I've done it once or twice. It's surprisingly not that difficult.

Hi @rory. Do you have a chitsheet or similar that you can share?

How long did it take for you?

@Antanicus @Ca_Gi @feonixrift

@61
I don't have any specific notes. I don't think there was a lot of specific problems. It didn't take me too long.
Then again, I have a lot experience installing web applications and unix services, so it might have been easy for me for that reason! 😆
I just ran it in "foreground" debug mode, which was "good enough" for simple usage. Didn't do any email stuff either.
@Antanicus @Ca_Gi @feonixrift

@61 I'm curious about Planet.osm (never heard before). How does it work? Is it just a backup of the main data or what? @Antanicus @rory @feonixrift

@Ca_Gi

Huge #XML file with all the map data in it.

Used for backup and replication.

@Antanicus @rory @feonixrift

@61 Therefore: 1) Is one-directional 2) I can access to the planet.osm DB but also if I changed something on it that wouldn't affect the main DB. Right? @Antanicus @rory @feonixrift

@Ca_Gi

AFAIU that's correct. Any changes that you do want to share would be pushed onto the main instance and changes you want to keep local would be applied directly to your database.

A possible improvement (but something like this may already exist) would be to have instances broadcast local changes to other federated nodes in real time. This way there wouldn't necessarily be a single authoritative instance. This may or may not be desirable.

@61 Ok, now it's more clear. Yeah, I like the idea. Different Instances that keep a copy of the main, basic data (that will be the same for every Instance - maybe the sync process could be verified via a blockchain verification system). And nearby a p2p federation that shares local changes

@Ca_Gi @Antanicus @rory @feonixrift

> I think that the internal chat could be easily federated.

Yup 👇
en.osm.town/

@Ca_Gi @Antanicus @feonixrift What do you mean by "internal chat"?

#OSM uses many communication channels (IRC, mailing lists, slack, twitter, telegram, mastodon (ie here)), the OSM user diaries, OSM private messages. In theory you could try to "federate" them, but they are a tiny tiny tiny part of OSM, and nothing to do with edits, or the map data. So I'm not sure what benefit it would bring? And the main thing (the data) would still be centralized.

@rory I mean the discussion platform on openstreetmap.org Yes, the data would stay centralized, but is the only implementation that I see of ActivityPub with OSM. Decentralize the map database could also be interesting but I think that that will bee too complex for AP. @Antanicus @feonixrift

@Antanicus To be serious, you could federate #OpenStreetMap by changing all object ids into a content bashed hash, and "OSM servers" could talk to each other and pull/push data to/from each other.

There is a proposal to remove node id and replace them with some sort of hash (partially for performance). Change OSM data mode can take years and years though (inertia!). We still don't have a (proper) area type.

/cc @61@feonixrift@hackers.town

@rory so _theoretically_ a decentralized OSM is possible.... Very interesting stuff!

@Antanicus @rory Just a "feeling": Consistent IDs could be decentraly managed by a Block-Chain?

@karlos @Antanicus nooooo! Not a blockchain! 🙂

(hahah. Buzzword! In theory it could be done with that, I just hate blockchains)

@rory

Besides, it would be a terrible solution anyway. As you mentioned and as git has proven since 2005-04-07¹, hashes are the way to go.

¹ git.kernel.org/pub/scm/git/git

@karlos @Antanicus

Sign in to participate in the conversation
Mastodon Bida.im

E' un social network autogestito. L'amministrazione tecnica e' supportata dal collettivo bida. Per maggiori informazioni leggi il nostro manifesto. Il nodo e' autogestito anche attraverso la mailing list. Per contattarci mastodon [AT] bida.im