@sir IRC is kind of broken as designed; the server pretty much expects to be talking directly to the user, so there are fundamental things missing, like knowing which past request an error message is a response to, or knowing whether a message you see or sent has been truncated because it was too long (or even what the length limit is)
@sir @lain @ciaby on the contrary: scaling problems are very often protocol problems, and even more often are much easier to solve by protocol changes than by implementation changes. HTTP scaled better than telnet, gopher, or FTP for protocol reasons; NNTP scaled better along some dimensions than HTTP for protocol reasons; HTTP/2 scales better than HTTP/1 for protocol reasons; TCP/IP scales better than NCP for protocol reasons; and I think, though I'm not sure, that the IRC protocol beats AP
@kaniini @sir @lain @ciaby Well, I was thinking also about AP's individual messages being a bit heavierweight than IRC's, and about it not having a native channel mechanism to limit the number of copies of a given message that any given server has to make. But at least it doesn't have netsplits, right?
@kaniini @sir @lain @ciaby The example messages in https://www.w3.org/TR/activitypub/ have 160 bytes or more of apparently mandatory per-message overhead, which is about 4× the size of an average IRC message. At the point where we hit bandwidth limits this means a 5× scalability penalty. But presumably we have to improve other bottlenecks a lot before that becomes the problem
@kaniini @sir @lain @ciaby But I was saying that the IRC protocol seemed likely to be more scalable than AP, not that AP's improvements over IRC and other messaging protocols weren't worth those putative scalability penalties. And as you probably saw in the other thread, IRC has substantial scalability problems of its own.
@kaniini @sir @lain @ciaby Scalability is only one axis of design quality. It's totally possible for one design to be more scalable than another despite being worse in other ways. Happens constantly, in fact. IRC servers routinely serve 10,000 concurrent clients (and, say, 1000 concurrent active users, and about 10 k messages per second) on sub-gigabyte hardware. ØMQ and AMQP implementations can beat those message rates by an order of magnitude; Kafka by 2.
@lain presumably someone has done load testing? I'd like to have some kind of vague order of magnitude idea of when things start to break: on a 1000MIPS machine with 1GiB, is it when statuses are flowing in at 1 Hz, 10 Hz, 100 Hz, 1 kHz, 10 kHz, 100 kHz, 1 MHz? Is it when there are 10 clients, 100 clients, 1000 clients, 10k clients, 100k clients? Does it matter how many other servers you're directly federating with, and where does that break, roughly?
@kragen the limit is well defined and you just seem to have encountered a bad client if it doesn't split it up/tell you it's too long.
@sir your client silently corrupting your data before sending it to the server is a solution in limited circumstances, but even so, it doesn't solve the problem, just makes it less frequent; and the institutionalized behavior when you violate the well-defined limit is to undetectably corrupt your data further, which is not reasonable behavior
@sir also, as explained in more detail in http://web.archive.org/web/20180911051912/https://gist.github.com/grawity/3257896 and https://developer.pidgin.im/ticket/4753, you can't reliably anticipate how much extra data the server network is going to tack onto your message before sending it to other clients, so the client isn't in a position to reliably solve the problem
@kragen I don't know what these people are talking about. The server tells you when your hostmask changes.
@sir that's not what a hostmask is; maybe you're thinking of a cloak?
@sir anyway, the IRC protocol has some serious deficiencies that can't be fixed in a backward-compatible way, but I'm not convinced that there's another protocol out there that's a Pareto improvement
@kragen this particular problem can be fixed in a backwards compatible way, though. What other problems do you have to complain about? Think carefully about whether or not they actually require incompatible changes before you answer
And for the record, IRC has been working well for over 30 years despite these "deficiencies", far longer than any of its competitors have come close to
@sir What is your proposal for fixing it in a backwards-compatible way?
IRC has *never* worked *well*. I've been using it since before Undernet existed (about 25 years ago) and running a server with some friends for most of that, I've written a couple of (shitty) IRC clients, and I'm hanging out with the ircII maintainer right now (ironically, on ICB rather than IRC). I could be mistaken about its problems but if so it's not due to the kind of profound ignorance you seem to be assuming :)
@kragen just send clients their hostmask so they can compute the length limit?
IRC has always worked more than well enough. I use it all day every day and pretty much never have issues. I used Matrix for all of 20 minutes and I was disconnected twice, had constant latency spikes, and couldn't join channels or start PMs without opening up a resource intensive web giga-application.
@sir That's not what a hostmask is :)
When was the last time you were on a million-user IRC network?
Yeah, it doesn't sound like Matrix is the solution we're looking for.
@kragen IRC can totally handle millions of users. Networks handle upwards of a hundred thousand every day without breaking a sweat. Some single channels have thousands of users alone. The design is quite scalable
@sir I know very smart programmers who devoted years of their lives to trying to get IRC to handle millions of users without success, though hey, maybe that's improved since then. Current networks are typically under 100K users and still have netsplits; fixing netsplits probably requires a new server-to-server protocol. Skype, by contrast, has handled tens of millions of users for a bit over a decade. I don't use it anymore because it's proprietary, but it does exist.
@kragen netsplits are pretty rare these days. I see them maybe once every 3-4 months
@sir yeah, that's definitely an improvement over 1994, and it roughly matches my experience. (I'm also on IRC all day every day.)
@sir I want to point out that, as far as I can tell, AP, Skype, and SSB don't have netsplits at all, though. (Not sure about Matrix.)
@kragen they do, they just don't call them netsplits and they don't necessarily have the same symptoms
@sir Yeah, of course any distributed system on a partitioned network will suffer a network partition. By "netsplit" I was referring to IRC's handling of that situation by irreversibly losing messages. (At least nowadays it doesn't enable randos to steal your channel — but fixing that required a backwards-incompatible protocol upgrade.)
@kragen this is still a client problem, not a protocol problem. Like I said, the limit is well defined, if your client doesn't deal with it the way you like you should ask them to change it.
All friendly creatures are welcome. Be excellent to each other, live humanism, no nazis, no hate speech. Not only for nerds, but the domain is somewhat cool. ;) No bots in general! (only with prior permission)