Epinova QA checklist for EPiServer CMS 6 projects

At the EPiServer meetup in Oslo a few weeks ago, I presented how we work with Quality Assurance in EPiServer CMS 6 projects at Epinova. We always strive for solid craftsmanship through the entire project scope. To achieve this, we have developed and refined a set of frameworks, tools and checklists which define our desired level of technical quality.

With the release of EPiServer 7 we're currently revising the entire system, and we feel it's time to share some of our QA systems with the community, including our complete development checklist for EPiServer 6 projects.

Custom EPiServer project template

All our EPiServer projects start out using the Epinova Visual Studio project template - a custom made, ever-evolving, barebones project containing only what we consider best practise configurations.
Key features include:

• a database pre-configured with user/group permissions and language settings
• config files tuned for optimal performance and scalability
• a small set of important plugins
• carefully selected base classes and extension methods for search, xforms, SEO, WebParts, breadcrumbs/menu controls, etc

Using this template, every project starts off right, quick, and ensures a basic uniformity between projects. It also ensures that the foundations of the project will pass the QA audit.

Automated Visual Studio tools and CI tests

Some of the audits we perform on our projects are extremely time consuming and inaccurate when done by a human. As part of constantly improving our quality, more and more checkpoints are audited automatically. To achieve this, we are using Continuous Integration systems and custom-built, automated tools integrated with Visual Studio. Developers use these tools during all stages of the project.

Development checklists

Checklists are a big part of the QA system, and they undergo continuous improvement. At Epinova we consider back-end and front-end to be separate specialist disciplines, and we have dedicated checklists for each of them. I've seen a couple of other checklists out there, but we decided to create more detailed lists for internal use. The checklist covered in this blog post is for back-end development only, although we might share the front-end checklist at a later time.

During the development phase, these checklists act as important guidelines. They are NOT quick-fixes to apply in the final stages of a project. We make sure developers know them well and continually use them during coding. Towards the end of every project, we produce two QA reports - one each for back-end and front-end. The reports are based on the checklists, and evaluate the overall technical quality before a site goes live.

In the report, the auditor marks each checkpoint as Passed, Failed, Not Applicable, or Subject For Discussion. If a checkpoint fails, the auditor adds a comment explaining why. Developers are given a short amount of time to correct errors or debate that particular decision, after which the report is finalized and a QA score is calculated. This stimulates a healthy internal competition to achieve the best scores - finding scape goats is never the focus.

Auditing is not always black and white though. For instance, while I initially expect to find zero TODOs in a completed project, finding a few will not always fail the checkpoint if there is a good explanation. Other issues detected by the auditor may not fit under any of the checkpoints, and are summarized to be debated at the end of the report. We also use the checklists when taking over a project from a third party to uncover potential flaws.

Our strong belief in QA systems comes from experience. Over the years I've seen several EPiServer solutions with high code quality on both back-end and front-end. However, the end product often smells of being created by engineers, for engineers - resulting in a bad user experience and dissatisfied customers. In my opinion, this is one of the most important (and neglected) areas of quality assurance.

An improved user experience for editors is one of the big new features in EPiServer 7, with more focus on details like On Page Editing and highlighting required properties. As details like these have always been covered by our EPiServer 6 checklists, these solutions are already well prepared to take advantage of the new features and prove that focusing on quality and doing things right the first time makes your solutions more viable over time.

The Epinova QA checklist for back-end development

Source code

• The solution has basic documentation for future reference
• The solution is easily retrieved from source code repository
• The solution builds without any errors or warnings
• All files included in the solution are used
• The solution contains no empty folders
• There are no files in the repository which are not included in the project
• Commits are organized in logical chunks with description and task id
• A separate production code branch exists
• The solution is automatically build on a Continuous Integration server
• All built assemblies have correct version and revision numbers set automaticly

Configuration files and other configuration

• The solution runs in IIS Express and IIS 7.x 
• Web.config is deployable without requiring a lot of transformations
• Separate config files for transformations to all deployment environments.
• Contents of AppSettings is kept to a minimum. Perhaps config section or PlugInSettings are better options?
• VPP paths are on a shared source
• PlugInSettings are reasonable and the controls are appropriate
• When using PlugInSettings remember to broadcast remote events when updating
• Content of FileSummary.config is customized to the solution
• httpCacheVaryByParams contains all necessary querystring parameters

User experience for editors

• The number of dynamic properties are kept to a minimum
• No unused pagetypes, webparts, blocks or dynamic content
• Properties are logically sorted for pagetypes, webparts, blocks and dynamic content
• Properties are logically arranged under different tabs for pagetypes, webparts, blocks and dynamic content
• Required and Searchable attributes are purposely set for all properties on pagetypes, webparts, blocks and dynamic content
• Default values are purposely set for all properties on pagetypes, webparts, blocks and dynamic content
• All properties including help texts have user friendly translations for all properties on pagetypes, webparts, blocks and dynamic content
• Visitor groups and gadgets have user friendly translations
• All plugins, modules, etc. have user friendly translations
• All tabs have user friendly translations for pagetypes and webparts
• Available pagetypes are logically limited
• Page versions are limited
• Webpart editing is enabled in most popular browsers
• OPE (On Page Edit) is functional for most properties on all page types
• Page guides are implemented or considered obsolete
• Quick publishing is implemented or considered obsolete
• Image editor is activated with settings according to site design guidelines
• Numbered properties (ContactPerson1, ContactPerson2, etc) are kept to a minimum

User experience for administrators

• All unused users and roles are removed
• Access to edit and admin can easily be done from admin and doesn't require changes in web.config

Technical quality

• The solution can be configured to run without external dependencies/integrations
• Projects and libraries are logically separated
• Logging is set up according to best practices
• Page type inheritance follows a logical hierarchy
• Large numbers of pages on the same level is avoided by using e.g. archive mechanisms
• CurrentPage.LogicalStartPage extension is used instead of PageReference.StartPage
• Base classes for materpages, templates, user controls, views and controls are used
• Descriptive namespaces and names are used
• Design- or position-specific names are avoided, e.g.  FrontPageLeftBlueMenu
• URL rewriting to external is avoided unless really necessary
• Viewstate is kept at an absolute minimum for visitors
• Appropriate modifiers (public, protected, private, internal, abstract etc) are used
• Date, time, currency, numerical values etc. are formatted using CurrentUICulture and never like "dd.MM.YYYY"
• No unhandled exceptions in javascripts
• Third party addons and modules are up to date
• Webparts and blocks are generic and not specialized for a zone or pagetype etc.
• Webparts uses "WebPartTransparentExceptionTrap"
• Project includes at least one global error page for all exceptions.
• Automatic IDs on server controls which are not used in code behind (ID="Label1", ID="Translate1") are removed
• Think before you create a specialized block or webpart for eg. banners. This is easily created with TinyMCE
• Use extensions methods wisely, especially for PageData and PageReference
• No hard coded property names (PropertyName="MyProperty"),except for PageLink
• Use MultiView or similar instead of switching visibility on several placeholders
• No unfinished TODOs
• Unused code removed instead of commented out
• All static files should return 304 after the first request
• CSS and js are bundled and minified on Release build configuration
• PageTypeBuilderUI (Match properties with PTB) reports no mismatch
• When using NuGets, older versions of packages are removed
• The solution works as expected for anonymous visitors (developers are often logged in)
• An appropriate caching strategy is implemented
• No large code files exceeding 400 lines

Globalization

• Available languages are purposely limited
• System language is set (Norwegian for most of our sites)
• Language files exist only for available languages
• No hard coded text, use of language files only
• Generate canonical tag for fallback and replacement language
• Lang attribute is set for <html> tag
• Personalized UI language for editors and administrators are limited (e.g. remove Chinese, Japanese etc. in most cases)

Search

• Bookmarking search results (including all facets) is possible
• All searchable content has an appropriate preview
• An appropriate text is provided when search result contains no hits
• Certain pagetypes and pages are excludable from internal search result and bots by using "Exclude From Search" property. 
• Pagination variants are used instead of displaying all hits at once
• Ranking is preserved even when manipulating search result

Javascripts

• Console.log() is avoided in production
• The amount of inline scripts in markup is kept at a minimum
• The global namespace has not been cluttered
• Bundle and minify scripts are used in production

Security

• Specialized tools are secured just like edit and admin
• Clear text email addresses are obfuscated for anonymous users
• User input is validated server-side
• User input is HTML-encoded before rendering
• Querystrings and form values are HTML-encoded before rendering
• User input for scripts is encoded if needed

Development environment

• Tracking scripts (e.g. Google Analytics) are disabled
• VPP files and content used in development are clearly separated from real content
• Correct build actions ("None" vs "Content") are configured for all files
• Automatic login for developer user account is configured
• Editing the local host file is not required to run the project
• Switching between edit- and view mode works properly
• Performance is tuned after build by kickstarting the project and setting up "remove assemblies" in EPiServerFramework.config

Test environment

• Not accessible without logging in
• Not indexed by bots

Production environment

• Existing link juice is preserved when launching new site
• Language attribute for hosts in EPiServerFramwork.config is used (to avoid mysite.no/no/)
• Site does not respond to both www.mysite.no and mysite.no. Use redirect to only one of them
• robots.txt exists with customized content
• Logging is disabled or kept at a minimum
• No windows users or groups from development or test environment exists.
• Assemblies are build in Release configuration
• Debug compilation is disabled
• Routines for deployment are documented on internal Wiki
• SMTP settings are configured and tested
• Gzip compression is used
• OWASP top 10 are addressed

Summary

With this brief introduction, we hope to inspire and spark off some constructive debates around QA in EPiServer projects. Perhaps one day, we can join forces and create a collaborative QA system to benefit the entire community?

If you found this useful, have any questions or disagree with some of our views, we'd love to hear your feedback in the comments section.