• Published
  • 1 min

Handle hostnames, schedule jobs and role access when synchronizing content

Content copy - Handle hostnames, schedule jobs and role access when synchronizing content

The Environment Synchronizer module helps you to set your environment into a known state after synchronizing databases between environments. In this blog post we’ll describe built in functionality to handle hostnames, schedule jobs, roles and access rights.

When you set up a new Optimizely website, or when you are synchronizing content between environments you need to update hostnames, turn on/off schedule jobs or set roles permissions depending on each environment. This is usually done manually, or sometimes even not done at all, which might have negative consequences. With the Environment Synchronizer module, you can configure this to be done automatically.

Some use cases:

Maybe you need to turn off the “Optimizely notification” schedule job in integration and preproduction environment.

A database synchronization usually needs a “Search reindex” job to run in order for this to be in sync with the updated content?

Different environments need to have specific “Edit" or “Primary” URLs.

You might not want environments other than production to be publicly available without forcing a login?

You want to add a role in preproduction for your test user?

What usually happens when this is not automated is two things:

  1. Content synchronization is not done frequently, which can have bad side effects since you are testing with old content.
  2. You have unwanted side effects after a content migration has been done. For instance, I don’t know how many times that I have had to deal with “bugs” in preproduction or integration that are really just due to the Seach and Navigation index not being aligned with fresh content.

When doing this manually you need to have clear routines of what needs to be done after a synchronization, and even with this in place it’s error prone. Also, when this is done manual it’s not possible to schedule automatic harmonization/synchronization, for example every Sunday 21:00 so that all environments are in sync each Monday morning.

“It would be nice to be able to configure this, so that it will automatically be handled and work every time.”. That was the use case when Epinova created the Environment Synchronizer addon.

Features

What can Environment Synchronizer do then? Time to have a quick look at the features.

Run as initialization module

You can specify if the environment synchronizer should run on startup and update configuration if needed. This will basically just trigger a scheduled job, so this can also be handled manually if desired. This comes with the positive side effect that you have a history of when it has done updates.

Run every startup

You may want the run environment synchronizer on startup but only “execute” when that database has been synchronized. The Environment Synchronizer addon will keep track on the current runtime environment vs what the database considers to be the environment. If these differ, most likely the database has been harmonized/synchronized from another environment and the synchronization will run.

Hostnames

You can specify the hostnames that the current environment should handle including support for multiple sites and multiple languages.

Force login

It’s possible to force loin by removing access rights for the everyone role, for instance if you have an environment where you don´t want external visitors to be able to reach the site, for example during development or environments “behind” production. This function will remove Everyone – READ permission on the site(s).

Turn on/off or execute schedule jobs

You will be able to schedule jobs turn on and off in an environment. You will also be able to execute a schedule job, for example “Search reindex” that maybe start a reindex of the “Find/Search and navigation” index. This configuration includes the ability to turn all scheduled jobs off, and then activating specific jobs, which is a common case for non-production environments.

Set / Remove roles

You can also add, update, or remove roles for a site.

Run manually

You can run environment synchronizer with a schedule job. So, if you don´t want to run it automatically on startup, you can run it manually with the schedule job when its suites you.

Custom synchronization code

You can implement an interface “IEnvironmentSynchronizer” from Environment Synchronizer that includes a method with the name “Synchronize”. That will be executed during startup or manual execution. With this function you can write your own code to do whatever you want. And Environment Synchronizer helps out to execute that code.

Showcase

So let us have a little showcase where we can use this addon. I have a website that is called “website.com”. As a developer you work on localhost and have a “development” database that you use when develop/maintain the website. Then you have CI/CD build and release pipelines that deploy to the Optimizely DXP environments Integration, Preproduction and Production. I have added some states/configuration that I want/need in each environment. Let’s have a look at how we implement this.

Environment specs

The first step is to install the NuGet package Addon.Episerver.EnvironmentSynchronizer in the website project.

Now you can start adding the configuration for each environment. So, you will need to make sure that you have appsettings.json files that will be used for each environment. Let's say that we have appsettings.json as default configuration and then have appsettings.*.json that represent each environment where localhost will be called development. Then we will end up with:

appsettings.json

appsettings.development.json

appsettings.integration.json

appsettings.preproduction.json

appsettings.production.json

Depending on what you like and what you have decided in your project there are many ways to split the configuration in different json files and let other json files add or override the basic configuration. To try to make it as clear as possible, I will add all needed configuration for each environment in respective configuration file. Then you can break it down and add your config in base file and add/override all you want.

Localhost/development

Development spec

On your localhost/development environment we want to:

  • work with hostname localhost:5001
  • The “Everyone” role should have READ access on all pages, so we don’t need to login to see the website (Still need to login for edit and admin CMS interface).
  • Remove some GDPR data in the database if it exists.
  • Turn off all scheduled jobs.

In the appsettings.Development.json file you can add the following configuration:

{
  "EnvironmentSynchronizer": {
    "RunAsInitializationModule": true,
    "RunInitializationModuleEveryStartup": false,
    "ScheduledJobs": [
      {
        "Name": "*",
        "IsEnabled": false
      }
    ],
    "SiteDefinitions": [
      {
        "Id": " 117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://localhost:5001/",
        "ForceLogin": false,
        "Hosts": [
          {
            "Name": "*",
            "UseSecureConnection": false,
            "Language": "en"
          },
          {
            "Name": "localhost:5001",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ],
        "SetRoles": [
          {
            "Name": "Everyone",
            "Access": [
              "Read"
            ]
          }
        ]
      }
    ]
  }
}

If we break it down:

RunAsInitializationModule

Attribute “RunAsInitializationModule” will make sure that the Environment Synchronizer tries to run automatically every time application starts.

RunInitializationModuleEveryStartup

The attribute “RunInitializationModuleEveryStartup” will make sure not to run synchronization if the database is holding an environment flag that is equal to the environment you are running on. Example: If you have just started the application after a harmonization of database from production => preproduction for example. This logic will then understand that the database flag holds “production” but startup/running on “preproduction” then it will execute the synchronization. After execution it will set database flag to “preproduction” so it will not run next startup with the same database.

ScheduleJobs

"ScheduledJobs": [       {
        "Name": "*",
        "IsEnabled": false
      }
    ]

The section “ScheduleJobs” holds the configuration about what to do with scheduled jobs. In this case we have only one item. That item is using a wildcard that means that it should be run on all schedule jobs that can be found in the database. And the attribute “IsEnabled”= false means that we will turn off all schedule jobs in this environment. Pretty nice when you never want the schedule jobs to run automatically in your development environment.

SiteDefinition

"SiteDefinitions": [
      {
        "Id": " 117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://localhost:5001/",
        "ForceLogin": false,
        "Hosts": [
          {
            "Name": "*",
            "UseSecureConnection": false,
            "Language": "en"
          },
          {
            "Name": "localhost:5001",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ],
        "SetRoles": [
          {
            "Name": "Everyone",
            "Access": [
              "Read"
            ]
          }
        ]
      }
    ]

The section “SiteDefinition” holds the configuration for the sites in the CMS. If you have multi-site setup Environment Synchronizer supports that as well. Site Id or the name will be used for matching the correct site to configure, we recommend matching on site id. And if you have different site id in different environments it could be good to look into first harmonizing the databases in the different environments so that you have same site id and content in your environments.

SiteDefinition: Hosts

When the logic has found SiteDefinition object in Optimizely CMS matching your SiteId or/and Name it will update:

  • SiteUrl to https://localhost:5001/
  • It will remove all existing hosts and then add
    • Host “*”
      • with no secure connection (http)
      • With default language “en”
    • Host “localhost:5001”
      • with secure connection (https)
      • With default language “en”

And the result is like the picture below.

Site definition - admin view

SiteDefinition: SetRoles

"SetRoles": [
          {
            "Name": "Everyone",
            "Access": [
              "Read"
            ]
          }
        ]

Last thing in the config is “Set roles”. Here we can see that we should set role “Everyone” to have READ access on the website.

Everyone - read

That will make it possible to visit the website without login. (Still need to login into edit and admin interface)

Check requirements

If we have a look again in the requirements…

Development spec

We handle host localhost:5001, we set the role “Everyone” – READ to not have forced login on website and we are turning off all schedule jobs. But we have not solved “Remove GDPR data” yet.

Custom code

In Environment Synchronizer there is an interface “IEnvironmentSynchronizer” that you can implement in your own class. Then you can write your own code that handles your GDPR data in this case.

using Addon.Episerver.EnvironmentSynchronizer;
using EPiServer.ServiceLocation;
using System.Text;

 

namespace YourWebsiteNamespace
{
            [ServiceConfiguration(typeof(IEnvironmentSynchronizer))]
            public class GdprEnvironmentSynchronizer : IEnvironmentSynchronizer
            {
                        private StringBuilder resultLog = new StringBuilder();
                        public string Synchronize(string environmentName)
                        {
                                    if (!EnvironmentHelper.IsProductionEnvironment(environmentName))
                                    {
                                                //Add logic to delete/handle GDPR data from application.
                                                resultLog.AppendLine("Removed GDPR data");
                                    }
                                    return resultLog.ToString();
                        }
            }
}

Now we have added a GdprEnvironmentSynchronizer that removes the GDPR data in all other environments except production. Notice that the “Synchronize” method returns a string. That is the result information that will be shown in schedule job result text for Environment Synchronizer history.

Result view

Change SiteId

If you have different SiteId between your environments. You can synchronize the database so that all environments in the end derive from the same database. We recommend that this is the best way. But if you can ´t do that you can change it manually in the database.

It is not impossible to change the UniqueId in the database if you want to do that instead of synchronizing databases. I tried this quickly and it worked after restarting the application. But I have no idea if that can break stuff in the CMS!

Table view

UPDATE tblSiteDefinition SET UniqueId = '117d8c57-f514-408e-b2a5-1bcad84de13c' WHERE pkId = 2

Note: I don´t know if you have access to the databases in the DXP environments. But if you find a way the manual change could work.

 

Integration

Integration spec

Now it is time to have a look at the setup for the Optimizely DXP Integration environment.

On the Integration environment we can see that we want:

  • work with hostname inte.website.com
  • Force login: remove Everyone role with READ access on all pages. That forces visitors to login before they can see the website.
  • Add a test user role and give it correct permissions
  • Remove some GDPR data in the database if it exists.
  • Turn off schedule job “Optimizely Notification”.
  • Execute the schedule job “Search reindex”

In the appsettings.Integration.json file you can add the following configuration:

{
  "EnvironmentSynchronizer": {
    "RunAsInitializationModule": true,
    "RunInitializationModuleEveryStartup": false,
    "ScheduledJobs": [
      {
        "Id": "8bd1ac63-9ed3-42e1-9b63-76498ab5ac94",
        "Name": "Optimizely Notifications",
        "IsEnabled": false
      },
      {
        "Id": "cf6e8504-1237-4f4a-a5a7-93169cd3ef63",
        "Name": "Search reindex",
        "IsEnabled": false,
        "AutoRun": true
      }
    ],
    "SiteDefinitions": [
      {
        "Id": "117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://inte.website.com/",
        "ForceLogin": true,
        "Hosts": [
          {
            "Name": "inte.website.com",
            "UseSecureConnection": true,
            "Language": "en",
            "Type": "Primary"
          },
          {
            "Name": "custxmstr972znb5inte.azurewebsites.net",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ],
        "SetRoles": [
          {
            "Name": "Tester",
            "Access": [
              "Read",
              "Create",
              "Edit",
              "Delete",
              "Publish"
            ]
          }
        ]
      }
    ]
  }
}

RunAsInitializationModule

Run automatically every time the application starts.

RunInitializationModuleEveryStartup

Only synchronize if database flag is different from current environment.

ScheduleJobs

"ScheduledJobs": [

      {

        "Id": "8bd1ac63-9ed3-42e1-9b63-76498ab5ac94",

        "Name": "Optimizely Notifications",

        "IsEnabled": false

      },

      {

        "Id": "cf6e8504-1237-4f4a-a5a7-93169cd3ef63",

        "Name": "Search reindex",

        "IsEnabled": false,

        "AutoRun": true

      }

    ]

The first item is specifying the scheduled job with id “8b…” and set the scheduled job to IsEnabled=false. If the scheduled job were running automatically, it has now been set to off. Since both Id and Name are specified, it will use Id to match against scheduled jobs from database. If Id is empty, it will try to match the name of the schedule job.

The second item will also set the scheduled job “Search reindex” to not run automatically. But there is also an attribute “AutoRun” = true that instruct Environment Synchronizer to execute that schedule job. In this case the job should reindex all pages on website in the Find/Search and navigation index.

SiteDefinition

"SiteDefinitions": [
      {
        "Id": "117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://inte.website.com/",
        "ForceLogin": true,
        "Hosts": [
          {
            "Name": "inte.website.com",
            "UseSecureConnection": true,
            "Language": "en",
            "Type": "Primary"
          },
          {
            "Name": "custxmstr972znb5inte.azurewebsites.net",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ]

The Environment Synchronizer will now search for the site definition with the id = “117…”. If it finds it, it will set the SiteUrl and “Force login”. “Force login” means that it will make sure that role “Everyone” is removed from permissions list for this website start page and subpages. Visitors most login to view/see the website.

SiteDefinition: Hosts

  • It will remove all existing hosts and then add
    • Host “iwebsite.com”
      • with secure connection (https)
      • With default language “en”
      • Set this hostname as Primary
    • Host “custxmstr972znb5inte.azurewebsites.net”
      • with secure connection (https)
      • With default language “en”

The new attribute here is the “Type” of hostname. You can set the same values as in EPiServer.Web.HostDefinitionType [Enum]. That is Primary, Edit, RedirectPermanent, RedirectTemporary.

SiteDefinition: SetRoles

Last thing in the config is “Set roles”.

"SetRoles": [
          {
            "Name": "Tester",
            "Access": [
              "Read",
              "Create",
              "Edit",
              "Delete",
              "Publish"
            ]
          }
        ]

Here we can see that we should set role “Tester” to have READ, Create, Edit, Delete and Publish access on the website.

Since this role does not exist, Environment Synchronizer will create the role and set the specified permissions on the website start page and all subpages.

Custom code

Since we want to delete GDPR data in this environment as need to make sure that our “GdprEnvironmentSynchronizer” class is still in the code base so that it gets executed on this environment.

 

Preproduction

Preproduction spec

On the Preproduction environment we can see that we want:

  • work with hostname prep.website.com
  • Force login.
  • Add a test user role
  • Remove some GDPR data in the database if it exists.
  • Turn off schedule job “Optimizely Notification”.
  • Execute schedule job “Search reindex”

In the appsettings.Preproduction.json file you can add the following configuration:

{
  "EnvironmentSynchronizer": {
    "RunAsInitializationModule": true,
    "RunInitializationModuleEveryStartup": false,
    "ScheduledJobs": [
      {
        "Id": "8bd1ac63-9ed3-42e1-9b63-76498ab5ac94",
        "Name": "Optimizely Notifications",
        "IsEnabled": false
      },
      {
        "Id": "cf6e8504-1237-4f4a-a5a7-93169cd3ef63",
        "Name": "Search reindex",
        "IsEnabled": false,
        "AutoRun": true
      }
    ],
    "SiteDefinitions": [
      {
        "Id": "117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://prep.website.com/",
        "ForceLogin": true,
        "Hosts": [
          {
            "Name": "prep.website.com",
            "UseSecureConnection": true,
            "Language": "en",
            "Type": "Primary"
          },
          {
            "Name": "custxmstr972znb5prep.azurewebsites.net",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ],
        "SetRoles": [
          {
            "Name": "Tester",
            "Access": [
              "Read",
              "Create",
              "Edit",
              "Delete",
              "Publish"
            ]
          }
        ]
      }
    ]
  }
}

If we look at the configuration for preproduction, we can see that it is almost as a copy of the integration environments configuration. The only difference is that hostnames points/use “prep” in the name instead of “inte” as in integration environment.

Production

Production spec

On the Production environment we can see that we want:

  • hostname prod.website.com
  • No - Force login.
  • Remove test user role
  • Turn on schedule job “Optimizely Notification”.

In the appsettings.Production.json file you can add the following configuration:

{
  "EnvironmentSynchronizer": {
    "RunAsInitializationModule": true,
    "RunInitializationModuleEveryStartup": false,
    "ScheduledJobs": [
      {
        "Id": "8bd1ac63-9ed3-42e1-9b63-76498ab5ac94",
        "Name": "Optimizely Notifications",
        "IsEnabled": true
      }
    ],
    "SiteDefinitions": [
      {
        "Id": "117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://website.com/",
        "Hosts": [
          {
            "Name": "website.com",
            "UseSecureConnection": true,
            "Language": "en",
            "Type": "Primary"
          },
          {
            "Name": "prep.website.com",
            "UseSecureConnection": true,
            "Language": "en"
          },
          {
            "Name": "custxmstr972znb5prod.azurewebsites.net",
            "UseSecureConnection": true,
            "Language": "en"
          },
          {
            "Name": "oldwebsite.com",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ],
        "SetRoles": [
          {
            "Name": "Everyone",
            "Access": [
              "Read"
            ]
          }
        ],
        "RemoveRoles": [
          {
            "Name": "Tester"
          }
        ]
      }
    ]
  }
}

Let us break down the configuration:

ScheduleJobs

"ScheduledJobs": [
      {
        "Id": "8bd1ac63-9ed3-42e1-9b63-76498ab5ac94",
        "Name": "Optimizely Notifications",
        "IsEnabled": true
      }
    ]

The first item is specifying the scheduled job with id “8b…” and set the scheduled job to IsEnabled=true.

SiteDefinition

"SiteDefinitions": [
      {
        "Id": "117d8c57-f514-408e-b2a5-1bcad84de13c",
        "Name": "Website.com",
        "SiteUrl": "https://website.com/",
        "Hosts": [
          {
            "Name": "website.com",
            "UseSecureConnection": true,
            "Language": "en",
            "Type": "Primary"
          },
          {
            "Name": "prep.website.com",
            "UseSecureConnection": true,
            "Language": "en"
          },
          {
            "Name": "custxmstr972znb5prod.azurewebsites.net",
            "UseSecureConnection": true,
            "Language": "en"
          },
          {
            "Name": "oldwebsite.com",
            "UseSecureConnection": true,
            "Language": "en"
          }
        ]

The Environment Synchronizer will now search for the site definition with the id = “117…”. If found, it will set specified SiteUrl.

SiteDefinition: Hosts

  • It will remove all existing hosts and then add.
    • Host “com”
      • with secure connection (https)
      • With default language “en”
      • Set this hostname as Primary
    • Host “website.com”
      • with secure connection (https)
      • With default language “en”
    • Host “custxmstr972znb5prep.azurewebsites.net”
      • with secure connection (https)
      • With default language “en”
    • Host “oldwebsite.com”
      • with secure connection (https)
      • With default language “en”

SiteDefinition: SetRoles

"SetRoles": [
          {
            "Name": "Everyone",
            "Access": [
              "Read"
            ]
          }
        ]

Here we can see that we should set role “Everyone” to have READ access/permission on the website. So now unauthenticated visitors can view the website. And once again. If the roles did not exist before, it would create it. If it exists, it will update and set new permissions.

SiteDefinition: RemoveRoles

"RemoveRoles": [
          {
            "Name": "Tester"
          }
        ]

Custom code

In the production environment, we don’t want to remove any GDPR data. Since we will not specify on which environment “IEnvironmentSynchronizer” classes will be executed or not. It is important to add logic for that in your own class, so we have added an if-statement that handles that for us:

if (!EnvironmentHelper.IsProductionEnvironment(environmentName))
{
            //Add logic to delete/handle GDPR data from application.
            resultLog.AppendLine("Removed GDPR data");
}

 

Environment

Something we haven´t described is how Environment Synchronizer know the current environment. The simple answer is it looks on the environment variable “ASPNETCORE_ENVIRONMENT”. It is the same variable that a lot of other applications use as well. Example Optimizely CMS https://docs.developers.optimizely.com/digital-experience-platform/v1.2.0-dxp-cloud-services/docs/environment-configurations

If you need to create your own handling around this. You can implement the “Addon.Episerver.EnvironmentSynchronizer.IEnvironmentNameSource” interface and take over how you want to return the current environment name.

Summary

In this blog post we have set up the Environment Synchronizer addon and done configuration for each environment to show a pretty regular setup. With this configuration the project can start using automatic harmonization of content between environments right away. Note: you can setup automatic harmonization/synchronization of content in Optimizely DXP environment in Azure DevOps with “Epinova DXP Deployment extension”.

At Epinova, we are working with a lot of projects where we move content from production “down” to all other environments every week, including development environments. Having real up to date content to work with when we develop new features helps a lot, both for testing but also to be able to demo internally or to the client.

Outro

I hope that you see the potential in using Environment Synchronizer addon in your projects to make your life a little bit easier.

For the latest documentation and guides please visit the repo on GitHub. Where all documentation and code exist.

https://github.com/Epinova/Addon.Episerver.EnvironmentSynchronizer

https://nuget.optimizely.com/package/?id=Addon.Episerver.EnvironmentSynchronizer

Epinova DXP Deployment extension

https://github.com/Epinova/epinova-dxp-deployment

https://marketplace.visualstudio.com/items?itemName=epinova-sweden.epinova-dxp-deploy-extension