Items as resources by Sitecore part 1: warnings
A common scenario for upgrading a Sitecore instance is to first upgrade your custom code with the latest binaries of the targeted Sitecore platform. You can test your code by deploying it to an empty local Sitecore environment. Once the code is locally deployed, you can push your serialized items to your local environment to test if the upgrade has succeeded.
However, within a production environment this is not as simple. You have existing databases that need to be upgraded. These databases contain the following type of items:
- Sitecore native items, which are managed by Sitecore
- Configuration items, which are managed by developers and are connected to the customers implementation
- Content items, which are managed by editors
While upgrading, Sitecore native and configuration items needs to be changed or moved to support the latest version of the platform. Also, items can only be updated or changed when Sitecore is started. In a classic Sitecore upgrade you would apply Sitecore Content Serialization packages to upgrade Sitcore native items ans item serialization tooling to update the configuration items.
Sitecore 10.1 introduced a breaking change, (441838) Items are now stored as resource data and are no longer stored in the database. So, instead of storing the Sitecore native items in a database, they are now stored in a .dat file. This is better known as items as resources. You can read more on this in the blog post of Martin Miles.
In addition, Sitecore Fast Query has been deprecated, as can be seen in the release notes of Sitecore 10.1 highlights. It has been deprecated because Fast Query is directly querying the database. As the items no longer resides in the database by default, but on disk, these items can no longer be retrieved by Fast Query. For more details you can read my blog around the end of Fast Query.
Items as resource lets you load a subset of items from precompiled resource items on disk and merges them into the user-visible content tree. Items are loaded while Sitecore starts up and are presented in the content tree. This makes upgrading much easier as the Sitecore native items are loaded from disk and already present, when the application starts. The need for applying ItemPackaging after an upgrade is no longer needed.
When Sitecore CLI 4.0 was released, a new ResourcePackaging plugin became available to create your own resource files. This makes it possible to also distribute the configuration items of your custom implementation in .dat files. Making upgrades and releases much easier. Now all the resources that are needed to run a Sitecore implementation can be placed on an image that can be uses to start a containerized environment. A separation between configuration and content is now also available on Sitecore item level. Making it much easier to upgrade.
Sitecore UpdateApp Tool
As mentioned earlier, items as resources are presented in the content tree and can still be edited. This is where it gets interesting. When you make a change and save the item, it is copied to the content database. When Sitecore loads, the item in the database overruns the item in the resource file. This can lead to unexpected behavior, as you can imagine that a newer version of an item was deployed, but you cannot see the changes.
To ensure that you know what items have been changed, you can run the Sitecore UpdateApp tool during an upgrade. Based on the report you can see what items are stored in the database and what changes were applied. Based on this knowledge you can decide what to do with the stored item in the database. Removing an item from the database, automatically restores the item from the resource file.
The Sitecore UpdateApp tool only comes standardly with the resource files for the Core, Master and Web database. Some installations also use other Sitecore modules and come with additional resource files, the resource files for modules can be downloaded and placed within the Sitecore UpdateApp tool. By doing this, the resources for the modules can also be checked if there are items stored in the database.
You can of course do the same thing for your own resource file and include these in the Sitecore UpdateApp tool. The report will now also show what items, belonging to your custom implementation, have been altered.
In our release pipeline, we included an init image that runs the Sitecore UpdateApp tool and writes the report to our DevOps. So, with each release we can see what resource items have been changed on production. This makes it easier to track unexpected behavior. However, this sometimes is the latest step you check when looking into an issue.
To keep future upgrades simple, as little as possible resources items should be changed. You want to prevent items from being unnecessarily changed and entered into the SQL content database. Sometimes scriban from a rendering variant is changed on production to solve an issue, or the name of a template is altered of items that are in a resource file. It would therefore be desirable to have a warning or create a gutter if items are in a resource file.
So I decided to create warnings in the content editor, when a resource item is changed and stored in the database.
Determining if an item originates from a resource file is not as simple as it seems. The properties that indicate if the item is a resource item are only known in the Sitecore.kernel and are not public, this could not be used easily, looking deeper into the items as resource architecture, I found a workaround in the uiDeleteItems pipeline.