Since we are asking the question, you would probably guess the answer to be ‘no’. Indeed it is :) Together with Dienst Justitiële Inrichtingen (the Dutch Custodial Institutions Agency, DJI, we reflect on an important and complex application migration project that we partnered on. So how does this work, and why did DJI opt to partner with Harmonizer for this project?
The application migration
DJI had been using Tridion as their intranet platform for many years. The information on their intranet websites had steadily grown over time, becoming an essential for their internal knowledge sharing and communication. Since Tridion was no longer supported by its vendor, DJI recently considered fit-for-purpose alternatives. In addition, Tridion is officially a Content Management System and not an intranet platform designed for collaboration and interaction, which has become key in the new way of working.
DJI selected Harmonics as their next social intranet platform. Everyone was excited about the choice and functionality of the new platform. The large challenge awaiting DJI was to transfer all valuable content from Tridion to Harmonics. The content consisted of all sorts of different types of information: news articles, users, FAQs, a knowledge bank, profile/contact information etcetera. DJI would use Harmonics, to post and distribute important messages, and to collaborate in teams across the organisation.
DJI knew one thing: they did not want to approach the task of migrating the information from Tridion to Harmonics manually. There were over 3,000 content items and more than 7,000 staff member profiles in Tridion. Apart from that, both systems operated quite differently. This means that it wasn’t as simple as copying and pasting the information from one system to the next - it needed to be transformed into the Harmonics structure as well.
For example: in Tridion, content lived in a folder structure. From the folder structure, information was featured on pages of the intranet. In Harmonics, information exists in groups and is further structured. The structure is maintained via tags and visually presented using widgets. This means that any approach, whether manual or automatic, needed to account for the translation of the old folder structure to the new structure based on groups and tags.
Another complicating factor was the internal cross referencing via hyperlinks. This structure also needed to remain intact as it came across from Tridion to Harmonics. No easy feat. In addition, since they were at it anyway, DJI wanted to clean up ‘dead’ links and irrelevant content.
Therefore DJI needed to make a difficult decision around who would be best placed to support this complicated and bespoke data transition process, for a reasonable fee. Via our partner Winkwaves, the company behind Harmonics, Harmonizer came into the picture.
The value of Harmonizer for data migrations
Why Harmonizer? At first it might seem like a stretch to use an integration platform to do a once-off data migration. However, it is not as far-fetched as it seems. At its core, Harmonizer specialises in a hyper flexible approach to extracting data from applications via APIs, transforming it in any thinkable way and then delivering it to the destination application in the best possible way. With an integration, this process is set once and repeated over and over again at the desired frequency and following the desired trigger.
A once-off migration project is in fact no different, with the exception of it running at a set frequency and following an automatic trigger. Everything else in terms of functionality delivers the same great value.
In terms of moving data from one application to the next there are a few ways to approach this.
Since every CMS or intranet platform has a database at the backend, one could leverage a database to database connection
Export content from the source application and ask the vendor of the destination application to build a custom import process that can handle the way the content is structured
Using APIs to extract and deliver data
An important factor in determining the right solution is whether you have an on premise instance of the application or cloud (native) instance. In addition, whether or not the ‘backend’ (e.g. database) of the application is directly accessible to you.
In a nutshell, commercial applications don’t always allow you to touch the backend / access the underlying database directly - and for cloud (native) applications, due to the way they operate, this becomes even harder, if not impossible. Often times this rules out option 1 altogether. If you would have direct access to the underlying database, option 1 would still be very expensive because of the increased complexity with understanding the intricacies of the data model and database.
Option 2 represents the the traditional way of migrating data, in the time where every software vendor was happy to build custom functionality on a one-off basis. However, nowadays every SaaS company needs to work at scale and nothing is developed for only a single customer anymore. Almost all software vendors will therefore say 'no' to option 2.
To understand the full picture with how data ‘lives’ in an application, you need to be across way more layers of the application than the database and data model alone. Therefore, the API is a good solution which provides functional and defined access to the associated application. Any other custom code solution that doesn’t leverage the API would therefore rely extensively on the vendor of the application and would almost by definition be expensive.
Why would you not leverage features like an API to facilitate complex data projects, if you can? And while you are at it, why wouldn't you opt for a modular platform, with lots of reusable code from one migration project to the next?
With Harmonizer, you can test the data migration as much as you want, provide feedback, tweak and test again. Or execute the migration in batches if that is desirable. Until you are exactly happy with the end result. Our object administration makes that we know exactly what we have migrated before and it also allows us to roll back the data migration or any part thereof - without storing any of your information permanently. The infinite customisation options make Harmonizer unique and a great value for money option.
The value for DJI
Given you’re reading this blog and came this far, it would be no surprise at this point that DJI selected Harmonizer for the migration from Tridion to Harmonics.
Harmonizer made sure that all 3,000+ content items and 7,000+ staff profiles were transferred to the optimal location, format, and using the best possible structure in Harmonics, so no manual effort was required at all. The old folder structure was automatically migrated and transformed to the required groups and tagging, based on a mapping set up by DJI.
As you may expect, there were a lot of internal links between different content items. Since Harmonizer keeps up with everything that in synchronized, the internal links could be rewritten, to insure that internal navigation between pages would still work. In addition, all pages were scanned for any remaining ‘dead ends’. DJI was provided with a list of all dead links, to resolve appropriately. Due to its modular nature, these type of transformations can be implemented using Harmonzier in a very efficient manner using exiting components.
At the end of the migration from Tridion to Harmonics, DJI was provided with a complete overview of all information that was migrated across. On this bases, it was calculated that Harmonizer reduced efforts by around ~90% compared to if the project was executed manually, saving 500 hours of manual labour.
DJI reported that one of the aspects they valued most of using Harmonizer, is how quickly the migration was able to be completed. This again was due to the modular nature of the platform - custom development was required at all for execution.
Furthermore, DJI really appreciated being able to make changes on-the-fly. For example, initially the approach to the FAQ in Harmonics was to create loose items for each question and answer. Late in the project, this approach was changed to building one list with dozens of questions and answers. DJI thought since the decision to change the FAQ approach was made so late, they would need to make the associated amendments manually. However, and much to their positive surprise, Harmonizer was able to accommodate this change in specifications with ease and very fast: within one hour.
DJI cited this flexibility to make changes and to roll back and forward in terms of migrated data (due to object administration via hash values) as the biggest advantages of using Harmonizer.
Now that the data migration project is complete, we are in further discussions with DJI how Harmonizer can support Harmonics on a more ongoing basis, via bespoke integrations. Think removing old or rarely accessed content according to certain business rules, and before doing so sending out notifications to content owners so they can interrupt the cleanup if need be. This means content stays relevant, which increases user engagement with a platform like Harmonics.
Another option that is on the table is to update manuals that reside in Harmonics automatically for DJI acronyms and other appropriate company messaging or language. And perhaps we also activate an automated notification service for staff to check their profile information in case it hasn't been updated for ~ a year.
As an aside, we haven’t just assisted DJI with these type of ‘once off migration’ projects. The flexible and modular design of the Harmonizer platform allows for high customisation in extracting the right data from any API, massaging it so it meets the requirements of the new system, and subsequently orchestrating the upload to the new system in the best possible manner via that system’s API.
As there are always multiple ways to approach a complex data migration project, the true value add of a service like ours is in understanding - conceptually and programatically - the finer details of the important business rules. Harmonizer can provide this service like no other platform can.
If this has peaked your interest, please don’t hesitate to reach out to us via our contact page!
Photo by Barth Bailey on Unsplash