Weekly Update: W/C 05th October, 2020

As part of our ongoing commitment to transparency and development in the open, our founder writes weekly updates to the Edge community. This is the 81st of these updates, originally posted to Telegram.

Lead article image

Hi everyone 👋

It’s somehow Friday again – time flies when you’re busy!

The first implementation of Edit’s eCom and subscription solution goes live in a week. Once it’s bedded down I’ll share the link.

The regional site for Leo Burnett mentioned last week is now live: https://leoburnett.com.pl

It’s built on Edit, with API in the backend and CDN for media.

Here’s the Edit interface for the site:

Image

We received a copy of the TNC Testnet node and have been looking through the explorer. It works, and appears extremely fast. We’re going to be setting up a copy in the network backbone next week.

And we’re expecting their explorer to be released publicly soon.

This week the network team rounded the corner on the last stages of the first phase of the Consul KV removal process. All system configurations are now made via gRPC connections between Hosts, Gateways and the parent Stargate, meaning there’s no longer any reliance on Consul in the network.

After about a week of trying to figure out a good pattern for handling asynchronous app and subscription updates, the team settled on a solution which limits the amount of times Hosts write their application status manifest to Stargate.

Here’s an explanation of the issue and the resolution:

Hosts create separate gRPC connections with Stargate for each type of configuration data. In this instance, we’re looking at applications (CDN for example) and application configurations (cdn.somedomain.tech).

Rewinding slightly to how things used to work when we used Consul, here’s a checklist outline of the process of configuring an app:

  1. Host finds a Gateway
  2. Host checks the KV store to see what the GW is running
  3. It sees an app (CDN for example)
  4. It looks for CDN in the configurations directory
  5. It reads the latest arch digest, fetches (if required) and launches the image
  6. It waits for the status of the image to become ‘healthy’
  7. It starts to watch the CDN subscriptions directory
  8. It syncs each CDN config with the container
  9. Once each config is synced, it writes to Consul KV store e.g. host/xxxx/apps/cdn with a ‘true’ value, to show that CDN is running properly and configured

This system doesn’t work very well now that subscriptions are no longer a child of applications. Why? Because Host may receive subscription configurations before it has received CDN configurations, so the subscriptions will start as orphaned.

Here’s the solution, broken down into the two journeys:

App Config syncs

  1. Pull and start app container
  2. Trigger a call to initialise any orphaned subscriptions
  3. Add a callback function to run init on every subscription once the application is ‘Ready’

In this process, whether the subscriptions are received before or after the app config, they will always be initialised, and started when the app is ready.

Subscription config sync

  • If an app can’t be found, do nothing
  • If the app can be found and this subscription is not yet initialised, initialise it
  • Watch for the application to become ‘Ready’, then send subscription config to it
  • If this is successful, set the external loop value of ‘notify’ to true
  • After cycling all subscriptions, if notify is true, sync subscription status’s with Stargate

These two protocols make it possible for us to guarantee that Hosts will wait for both the application and the subscriptions to complete their setup process before telling the SG that the Host is a valid contributor.

Next up is making sure that:

  • Host does not dequeue requests for applications that aren’t fully configured
  • Host stops applications when a new version is released

test.network stats are down from 30-40% CPU (Consul) to 1-1.25% CPU (network native solution).

In other network updates, as part of the changes to the way configs are synced, before DNS has flushed Gateways will now handle invalid DNS resolution.

And an issue was fixed on Host where a new CDN config was not triggering a service state callback to Stargate (meaning that Stargate was not aware that it had successfully received and deployed a new config).

Work on the reduction of the filesize of CDN is ongoing. And we’re also mid introducing documentation for CDN to the Edge website.

And that’s it for this week.

Enjoy your weekends.

Related articles

More updates articles
Mail icon

Like what you are reading? Want to earn tokens by becoming an Edge Node? Save money on your hosting services? Build amazing digital products on top of Edge services? Join our mailing list.

To hear about our news, events, products and services, subscribe now. You can also indicate which services you are interested in, which we use for research and to inform the content that we send.

* You can unsubscribe at any time by emailing us at data@edge.network or by clicking on the unsubscribe link which can be found in our emails to you. Read our Privacy Policy.
Winner Best Edge Computing Platform Technology & Innovation Awards
Presented by Juniper Research