By Simon Woodhead
Simwood Meet caught on fast! We’ve commented before about some of the use cases you’ve told us about, from Government to remote education. What started as a highly available deployment of Jitsi Videobridge to help ourselves and our customers communicate privately, securely, and cost-effectively through these crazy times, quickly grew legs.
When we published our app last week, we got a glimpse of the global nature of what we’d started. Sure, there were loads of downloads from the UK, but it was not the top country. Nor was the US. The main takeaway though is that it has been downloaded worldwide, from Argentina to Vietnam (not quite to Zimbabwe yet!). And it is being used, the average user having seven conferences per device in the last week. These are Apple stats based on opt-in users, not sneaky tracking.
We’re still seeing the median conference size at three which is quite significant, not least because most conferences have two. The distinction makes a huge difference technically. At two participants, media usually flows peer-to-peer, encrypted regardless but taking the direct path. This should optimise privacy and performance, especially if those two users are local to each other.
At three or more participants in a conference, sessions move to the server. Everyone sends a few streams (high res. and low res.) and then an appropriate selection of streams is sent to each participant – usually one high res. and several thumbnail-sized. The role and placement of the server node is thus vital.
Last week, all our nodes were in London. That meant three people in the US having a conference would be bridged in London. Inefficient, suboptimal, and potentially leading to a diminished experience.
So, over the weekend, we’ve deployed new nodes across Europe, Asia, and North America. As some of these are beyond the reach of our own network, we’ve leveraged AWS’s global anycast service for the first time. The whole world is now within 100ms of an entry point, from where traffic is backhauled over private connectivity to our nearest node. This means everyone hits their closest node, anywhere in the world, in theory.
But there’s a problem. By default, Jitsi Videobridge maintains an entire conference on a single node, and that is the node on which it was first created. So on our Simwood weekly stay-sane-at-home call with the whole team, if our US colleagues get there first, the conference will be homed in the US. That is great for them talking to each other, but the experience for the majority in the UK is diminished. Not only does everyone have media coming transatlantically (more prone to packet loss and at reduced bandwidth through the laws of physics) but any UK-to-UK interactions are all bridged through the USA. This isn’t really very nice and is far from optimal.
What we really want is every participant to hit their local node – lowest latency with least opportunity for packet loss, and highest resulting bandwidth. If the US guys hit a US node, and the UK guys hit a UK node it’d be the best it could be, regardless of what order people join the call. Local conversations (e.g. US to US) feel local, irrespective of who is on the call or where it is set up. That is essentially what we’ve enabled this weekend!
To make this happen, those local nodes of course need to communicate and this is where the magic happens. All nodes that are participating in the call exchange all media streams directly and across private connectivity (itself achieving transfers 20-52% faster than the public Internet between the same nodes). The only way we can improve on this is to make light quicker or put nodes closer to end users. Whilst some competitors indirectly claim to have achieved the former (if they did but know it), here in the real world, we’re going to continue deploying nodes where the demand is. There are loads of parallels and lessons here for the voice world too.
So, to the acid test: does it benefit user experience. In our tests it most certainly does but what really prompted me to write this was the usage, which if it is a proxy for satisfaction means people are very happy! We’re seeing 70x the usage we were at the end of last week and most of that is coming from one part of the world that now has a regional node. Another region of the world is also showing massively more usage (well 2x) but to be fair they’re still asleep! Gulp.
