Introduction

Have you ever wondered how sites like Twitch and eBay Live handle so many users, keeping them updated and connected without breaking or slowing down?

The demo above is the front-end to a very real distributed server network that's built the same way that large real-world systems are, in this case using ASP.NET Core, SignalR, Redis, GraphQL, and other technologies.

The server network is composed of four servers: a primary, a secondary, and two external servers.

Each server communicates in different ways to the other servers and to front-end users, showcasing different technologies.

You communicate with the servers by posting GraphQL statements from the front-ends. This simulates how outside servers would talk to them, like a message that eBay Live would send when a stream goes online.

The servers communicate back to you and other users by updating the stream statuses at the top of each front-end, and through livestream-style notification bubbles.

You can have as many front-ends open as you like, to simulate many users. Just open this page in multiple browser windows and devices, and you'll see changes propagate across the entire network!

I invite you to play with it, to post different updates to different streams and servers, and to see which servers communicate with which other servers.

Click colored links on each front-end to see a detailed discussion of how each communication method works.

The available GraphQL commands are:

mutation {
  setStreamStatus(
    stream: streamName,
    isLive: true/false
  )
}

mutation {
  clearStreamStatus(
    stream: streamName
  )
}

Monitoring is provided by a separate ASP.NET service worker named Monitoring Service. You can find a discussion of it below.

Limitations

This is a conceptual demo of server architecture and messaging, not a production-grade system. I've omitted or simplified several critical concerns you'd face when building something real. Here are a few of the limitations:

• No persistent storage:For demo purposes, these servers store stream statuses locally in memory. A real system would use durable, queryable databases like MSSQL or PostgreSQL.
• No server authentication:To simplify access to the demo, anyone can issue GraphQL mutations. Production systems obviously need robust auth layers like OAuth or JWT.
• No testing or logging:I opted not to implement significant testing or logging, and omitted the remainder from code samples below for clarity. A production system would need logging like Serilog and intense unit, integration, and load testing.
• No horizontal scaling:This is a 1:1 mapping of server roles, designed to show how a network could comfortably scale horizontally using load balancers like ALB and Elasticache, but obviously it doesn't implement those.
• No containerization:For simplicity, these servers coexist on the same VM, rather than being deployed as independent containers into an environment like ECS or EKS.
• No access protection:In a real environment, these would be hosted behind Cloudflare reverse proxy for various safety measures, with an additional middleware defensive layer using AspNetCoreRateLimit, and IAM restrictions on all services and containers.
• Limited input validation:In the interests of experimentation, I opted not to limit and validate data entry on the server side, but it would normally be performed in the setStatus/setStatusAsync functions.

Primary Server

The Primary Server runs on port 5221 and acts as a hub server; while it shares system and user load with the Secondary Server, only the Primary Server sends messages to External Servers.

The Primary Server hosts this demo's primary front-end, composed of this webpage, and the three iframes on it which contain the front-ends of the three other servers. I carefully designed the user interface to be an interactive technical diagram, and I'm pleased with how it turned out.

Like the other servers, it stores stream statuses locally, in memory, to better illustrate server communication and directionality. In practice, they would be in an independent data store like an MSSQL database, and we wouldn't need a hub server.

  private readonly Dictionary _statuses = new()
  {
    { "Stream 1", false },
    { "Stream 2", false },
    { "Stream 3", false }
  };

Also like the other servers, it acts as a GraphQL server that users can post to as if they were external services providing stream updates. Each server exposes two operations:

mutation {
  setStreamStatus(
    stream: "Stream 1",
    isLive: true
  )
}

mutation {
  clearStreamStatus(
    stream: "Stream 1"
  )
}

Secondary Server

The Secondary Server runs on port 5224 and acts only as a secondary server; sharing the user and messaging loads but NOT communicating with External Servers, which lets us easily demonstrate and prove server communication and directionality.

(In a real deployment with an independent messaging queue, every server could be a Primary Server.)

Like the two External Servers, the Secondary Server exposes a small web interface for demo purposes, cross-linked to this discussion using Javascript postMessaging to cross the CORS boundary:

  function parentScrollToId(idToScrollTo) {
    parent.postMessage({ scrollTo: idToScrollTo }, '*');
    return false;
  }

External Server

The two External Servers run on ports 5222 and 5223 and emulate external servers or services, which are not tied in to the SignalR or Redis messaging between the Primary and Secondary servers, and do not communicate with each other.

Because they store stream statuses locally like the other servers, but do not communicate with other servers, updates made through their GraphQL API's do not propagate anywhere but to their own front-end users.

Source: SignalR w Redis backplane

The Primary and Secondary Servers send updates to their user front-ends via SignalR, using a Redis backplane between the servers to distribute messages.

  builder.Services.AddSignalR()
    .AddStackExchangeRedis("localhost:6379");

SignalR is an ASP.NET WebSockets technology that allows front-ends to subscribe to the server for messages:

  const connection = new signalR.HubConnectionBuilder()
    .withUrl("/streamStatusHub")
    .configureLogging(signalR.LogLevel.Information)
    .build();
  connection.on("ReceiveStatusUpdate", (statuses) => {
    console.log("Primary Server SignalR received:", statuses);
    if (initialUpdateReceived) {
      for (const [key, newValue] of Object.entries(statuses)) {
        const oldValue = currentStatuses[key];
        if (oldValue !== newValue) {
          console.log(`Primary Server ${key} changed from ${oldValue ? 'ONLINE' : 'offline'} to ${newValue ? 'ONLINE' : 'offline'}`);
          var bubbleContainer = document.getElementById("PrimaryServerDiv");
          if (newValue) {
            showNotificationBubble(`${key} is LIVE!`, true, bubbleContainer);
          } else {
            showNotificationBubble(`${key} has ended`, false, bubbleContainer);
          }
        }
      }
    }
    statusDiv.innerHTML = Object.entries(statuses)
      .map(([stream, isLive]) => `${stream}: ${isLive ? "ONLINE" : "offline"}`)
      .join("<br>");
    currentStatuses = statuses;
    if (!initialUpdateReceived) {
      initialUpdateReceived = true;
    }
  });
  connection
    .start()
    .then(() => {
      console.log("Primary Server SignalR connected");
      connection.invoke("RequestInitialStatus");
    })
    .catch(err => console.error("Primary Server SignalR error:", err));

Whenever the Primary or Secondary Server receive a status update, they broadcast the full updated status list to their connected users:

  private async Task BroadcastStatus()
  {
    Console.WriteLine("[SignalR Hub Broadcast] Sending ReceivedStatusUpdate signal through SignalR hub:");
    foreach (var kv in _statuses)
    {
      Console.WriteLine($" - {kv.Key}: {kv.Value}");
    }
    await _hubContext.Clients.All.SendAsync("ReceiveStatusUpdate", _statuses);
  }

Redis is used as a "backplane" for this signaling mechanism, allowing the servers to act as relays to their own users for broadcasts from other servers in the network, completely distributing the user load throughout the network:

Broadcasting to users via SignalR/Redis

When you post a mutation to either the Primary or Secondary Server, it updates its own local data:

  using PrimaryServer.Services;

  namespace PrimaryServer.GraphQL
  {
    public class Mutation
    {
      public async Task<bool> SetStreamStatus(string stream, bool isLive, [Service] IStreamStatusService service)
      {
        return await service.SetStatusAsync(stream, isLive);
      }
      public async Task<bool> ClearStreamStatus(string stream, [Service] IStreamStatusService service)
      {
        return await service.ClearStatusAsync(stream);
      }
    }
  }

  public async Task<bool>
  SetStatusAsync(string stream, bool isLive)
  {
    lock (_lock)
    {
      _statuses[stream] = isLive;
    }
    ...
  }

And then broadcasts the message through SignalR both to its own front-end users and for relay through the other SignalR/Redis hub-connected server to that server's front-end users:

  private async Task BroadcastStatus()
  {
    Console.WriteLine("[SignalR Hub Broadcast] Sending ReceivedStatusUpdate signal through SignalR hub:");
    foreach (var kv in _statuses)
    {
      Console.WriteLine($" - {kv.Key}: {kv.Value}");
    }
    await _hubContext.Clients.All.SendAsync("ReceiveStatusUpdate", _statuses);
  }

Syncing between servers via Redis Publish/Subscribe

SignalR is not designed for server-to-server communication. When the Primary or Secondary server is asked via Redis to broadcast an update to its users, it does NOT update its OWN data.

This isn't a problem with independent data stores, but we want some communication problems to be visible so we're using in-memory data storage.

To update that memory in the other server after broadcasting an update to its users, we publish the update through Redis, which updates all the other subscribed server:

  using StackExchange.Redis;
  using System.Text.Json;

  namespace PrimaryServer.Services
  {
    public class RedisSyncService : IHostedService
    {
      private readonly IServiceProvider _serviceProvider;
      private readonly ISubscriber _subscriber;
      private readonly ConnectionMultiplexer _redis;

      private const string ChannelName = "streamStatusUpdates";

      public RedisSyncService(IServiceProvider serviceProvider)
      {
        _serviceProvider = serviceProvider;
        _redis = ConnectionMultiplexer.Connect("localhost:6379");
        _subscriber = _redis.GetSubscriber();
      }

      public Task StartAsync(CancellationToken cancellationToken)
      {
        _subscriber.Subscribe(ChannelName, async (channel, message) =>
        {
          Console.WriteLine($"[RedisSyncService] Received message on channel '{channel}': {message}");
          try
          {
            var statuses = JsonSerializer.Deserialize<Dictionary<string, bool>>(message!);
            if (statuses != null)
            {
              using var scope = _serviceProvider.CreateScope();
              var streamService = scope.ServiceProvider.GetRequiredService<IStreamStatusService>();
              streamService.UpdateStatusesFromRemote(statuses);
            }
            else
            {
              Console.WriteLine("[RedisSyncService] Warning: Deserialized statuses were null.");
            }
          }
          catch (Exception ex)
          {
            Console.WriteLine($"Error processing Redis message: {ex.Message}");
          }
        });
        Console.WriteLine("[RedisSyncService] Subscribed to Redis channel: " + ChannelName);

        return Task.CompletedTask;
      }

      public Task StopAsync(CancellationToken cancellationToken)
      {
        _redis.Dispose();
        return Task.CompletedTask;
      }

      public static async Task PublishAsync(Dictionary<string, bool> statuses)
      {
        using var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379");
        var subscriber = redis.GetSubscriber();
        var json = JsonSerializer.Serialize(statuses);
        await subscriber.PublishAsync(ChannelName, json);
      }
    }
  }

  public void UpdateStatusesFromRemote(Dictionary newStatuses)
  {
    lock (_lock)
    {
      foreach (var kvp in newStatuses)
        _statuses[kvp.Key] = kvp.Value;
      var keysToRemove = _statuses.Keys.Except(newStatuses.Keys).ToList();
      foreach (var key in keysToRemove)
        _statuses.Remove(key);
      Console.WriteLine("[RedisSyncService] Local statuses updated:");
      foreach (var kv in _statuses)
      {
        Console.WriteLine($"  - {kv.Key}: {kv.Value}");
      }
    }
  }

Propagating to External Servers via GraphQL

When the Primary Server receives data updates, we want it to notify External Servers and services, as an example of how to communicate to servers and users outside of our distributed network.

For this, we use asynchronous HTTP posts of GraphQL, and we do not wait for responses since we wouldn't use them:

  private async Task ForwardMutationAsync(string query, string url)
  {
    var payload = new
    {
      query
    };
    using var httpClient = new HttpClient();
    var content = new StringContent(
      System.Text.Json.JsonSerializer.Serialize(payload),
      System.Text.Encoding.UTF8,
      "application/json");
    try
    {
      var response = await httpClient.PostAsync(url, content);
    }
    catch (Exception ex)
    {
      Console.WriteLine($"Exception forwarding mutation to {url}: {ex.Message}");
    }
  }

Monitoring

A separate ASP.NET service worker monitors the four servers above as well as the Redis server and sends email updates on startup and on any status changes.

The service worker runs on a 5-minute debounce, only sending alerts on changes after a 5-minute period to avoid email spam on multiple server changes.

Actual status email after manually
turning off Redis and External 1.

In production environments, the monitoring role would probably be filled by interfacing with a separate monitoring application like DataDog or Thousand Eye.

Below are two key functions in the monitoring service, CheckAllServicesAsync and IsGraphQLServerOnline. These test the ability of each server to receive and respond to requests, and connectivity with Redis.

public async Task CheckAllServicesAsync()
{
  var sb = new StringBuilder();
  bool somethingChanged = false;

  //  Check Redis server
  bool redisOnline = false;
  try
  {
    using var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379");
    var pong = await redis.GetDatabase().PingAsync();
    redisOnline = true;
  }
  catch
  {
    redisOnline = false;
  }
  if (_redisServer.LastStatusWasOnline != redisOnline)
  {
    somethingChanged = true;
    sb.AppendLine($"Redis: {(redisOnline ? "online" : "OFFLINE")} (was {(_redisServer.LastStatusWasOnline ? "online" : "OFFLINE")})");
  }
  else
  {
    sb.AppendLine($"Redis: {(redisOnline ? "online" : "OFFLINE")}");
  }
  _redisServer.LastStatusWasOnline = redisOnline;

  // Check GraphQL servers
  foreach (var server in _servers)
  {
    var url = $"{_baseAddress}:{(_isProduction ? server.ProductionPort : server.LocalPort)}/graphql";
    bool online = await IsGraphQLServerOnline(url);

    if (server.LastStatusWasOnline != online)
    {
      somethingChanged = true;
      sb.AppendLine($"{server.Name}: {(online ? "online" : "OFFLINE")} (was {(server.LastStatusWasOnline ? "online" : "OFFLINE")})");
    }
    else
    {
      sb.AppendLine($"{server.Name}: {(online ? "online" : "OFFLINE")}");
    }

    server.LastStatusWasOnline = online;
  }

  return new MonitoringReport
  {
    Summary = sb.ToString(),
    HasStatusChanges = somethingChanged
  };
}

public async Task<bool> IsGraphQLServerOnline(string url)
{
  using var client = new HttpClient
  {
    Timeout = TimeSpan.FromSeconds(2)
  };

  var content = new StringContent(
    "{\"query\":\"{ __typename }\"}",
    Encoding.UTF8,
    "application/json");

  try
  {
    var response = await client.PostAsync(url, content);
    return response.IsSuccessStatusCode;
  }
  catch
  {
    return false;
  }
}

Messaging Restrictions

It would be difficult to understand these communication flows if all servers and end-users updated perfectly.

To better illustrate the messaging successes, we treat the Secondary Server exclusively as a non-hub server, and restrict it from communicating updates to the External Servers. We also restrict Primary Server from relaying updates to the External Servers from the Secondary Server.

We also restrict the External Servers from communicating to any other server, including each other, although in practice they would normally respond to messages.

Hire Me!

You can find me on LinkedIn, at https://www.jcrichman.com/, and on Twitter/X.