WebCenter Custom Portal App - What happened to my changes? MDS runtime vs designtime (JDeveloper)

Have you ever been in a situation where you had created a new resource, such as a WebCenter Navigation, at runtime (RT) - through the WebCenter Portal Administration client - only to have the new resource disappear (from within the Admin client) upon a redeployment of the portal application?  If so, don’t be alarmed the new resource hasn’t been lost, only the “list” of resources that the Admin client uses to display the resources, has been updated (over written) from re-deployment.  Probably you are wondering why this is so?  In this blog I will attempt to make this clearer on how this (and a few other) scenarios are handled in MDS.

First of all, a brief background on MDS (Meta-Data-Services) and its role with WebCenter.  The MDS framework allows you to create customizable applications.  A customized application, specifically in this case a WebCenter Portal application, contains a base application (the base documents) and one or more layers containing customizations.  MDS stores the customizations in a metadata repository (usually file based for development, and database for staging environments) and retrieves them at run time to merge the customizations with the base metadata to reveal the customized application.  The Oracle documentation mentions that since the customizations are saved separately from the base, the customizations are upgrade safe; a new patch to base can be applied without breaking customizations. When a customized application is launched, the customization content is applied over the base application.

Indeed as the documentation states, the new customization (which in my example was a new Navigation), is safe, since it is still stored in the MDS repository.  However, what was not saved (upon a redeploy) is the resource list, which is stored in the generic-site-resources.xml file.  This file gets created automatically in JDeveloper whenever you create a WebCenter Portal application.  This following image shows where the generic-site-resources.xml file is located in the source path:

Notice that the generic-site-resources.xml file is located in the “file-based” MDS repository, and is “scoped” by an auto generated unique identifier (GUID).  The following is a code snippet that was taken from the generic-site-resources.xml from my example project.  Notice that it currently contains only one Navigation (Default Navigation) resource.  This Navigation resource is created OOTB for every Portal application.

Since I am going to create a new Navigation at run time (RT), I will leave this file as is, and just deploy the Portal application to demonstrate the “lost” portal resource use case scenario.  Before I do, I want to first explain a little bit about this (and the other possible) scenarios. 

The following is a matrix of 3 of the most probable use cases (Scenarios).  The most typical use case is Scenario 1, where the portal resources have been created in development via JDeveloper, then deployed, portal resource(s) updated/added at RT, then the portal application redeployed (where redeployment could be for any reason, such as a class update, .etc, not necessarily the “same” resource update).  I will talk more about Scenario 2 later, but from the matrix you can notice the subtle differences. 

Use cases	Resource Create Location	(RT) Update	Export (RT) then Import (DT	Re-Deployment of application
Scenario 1	DT	X		X
Scenario 2	RT	X	X	X
Scenario 3	RT	X		X

Scenario 3 is the focus for this example, since once the Portal application has been deployed; there will most likely be many RT updates/additions before any redeployment occurs. 

Now back to the example.  During the deployment phase for a typical Portal application, you must ensure that you pick a target MDS repository.  The WebCenter Admin guide outlines the possible use case scenarios, such as using ANT, WLST, and the Enterprise Manager console.  However, for this example, I am just going to deploy straight from JDeveloper.  From JDeveloper, after the initial packaging, the prompt for the target MDS appears:

For my example, I will choose the default settings.  However, it is important to note the name of the partition, since this name will be used later in my steps to retrieve the “lost” portal resource.  Note:  The auto generated partition name comes from the Portal application’s web deployment profile name.  For example, the partition name here is MDS_Prop_Test_Scenario3_webapp1_V2.0.  You can also choose to supply the name of the partition, instead of accepting the default one.  This name (in JDeveloper), will appear in the drop-down list for subsequent redeployments.

Once the Portal application has been successfully deployed, I can use Enterprise Manager to view the new partition in the MDS repository.  NOTE: you can also use WLST to perform this action:

Now that all the required pieces have been set, it is time to go the application and create a new Navigation resource.  However, before I do, I want to create a Metadata “label” to the new partition first.  To do this I will use one of the custom Metadata Services WLST commands.  For more information on these commands see the Oracle® Fusion Middleware WebLogic Scripting Tool Command Reference 11g Release documentation.  You can think of the Metadata label as a save point in the repository, which will be an invaluable tool for tracking down MDS related issues.  For my example, once I have connected (via WLST) to WebLogic, I will then execute the following command to create my desire label:

createMetadataLabel('MDS_Prop_Test_Scenario3_webapp1','WC_Spaces','label_1')

Now that the label has been successfully created, I can then proceed to perform a runtime update to the application.  Once the application has loaded in the browser, you must log in to enable the Administration link to appear.  Clicking on the link reveals the Admin console.  For my example, I will create a new Navigation, and then add (this step is optional and is not required to produce the issue) a new Link:

Now that the new Navigation has been added to the MDS repository, it is time to redeploy the application.  First close the browser, so that there is not any open session (this step is not mandatory, but a good practice).  However, before I perform the redeployment, I want to again create another Metadata label save point.  This will enable me to capture the new changes (Navigation) in the MDS between a set of (label) save points.  Again from WSLT, I execute the following command:

createMetadataLabel('MDS_Prop_Test_Scenario3_webapp1','WC_Spaces','label_2')

Once the second label has been created, I can then perform an export, which will then enable me to see the files that have been altered and/or added.  The following is the WLST command to perform the export:

exportMetadata('MDS_Prop_Test_Scenario3_webapp1','WC_Spaces','/home/oracle/mds_export','/**','%','',false,false,false,false,'label_1','label_2')

Notice that the export will produce the results in the mds_export directory (the third parameter in the export command), and that the search is retrieving all the files: (i.e. /**)

Notice that the file results contain an updated version of the generic-site-resources.xml.  Reviewing this file displays that the new Navigation entry to the navigation resource section:

This addition is good since I now know that my changes have been saved into MDS.  Next, I can perform the redeployment test.

Following the steps for deployment before, while ensuring that I choose the same partition, I then reload the application in the browser.  Once again I log into the app, I then proceed to click on the Administration link to get to the console.  Now when I select the Navigations link (from the Structure left pane), I can see that the new Navigation that I create a minute ago is no longer displayed:

What happened?  Are all of my changes gone?  This could be trouble if there were many updates since the last deployment.  Let’s check the current source of truth the MDS repository.  Since there have been no run time updates, I will not be able to capture the changes using MDS labels.  This is actually good, since my changes should still be in the repository.  However, I can still use the export command to retrieve the generic-site-resources.xml file.  To perform the export, I use the generic form of the export command:

exportMetadata('MDS_Prop_Test_Scenario3_webapp1','WC_Spaces','/home/oracle/mds_export','/**');

From the result set of files I can see a version of the generic-site-resources.xml.  Upon reviewing this file, I can see that the entry for the new Navigation in the navigation resource section is no longer present.  So what happened?  Basically, the generic-site-resources.xml, which is present in the JDeveloper project source, gets included in the MAR archive.  Then during the redeployment, this version of the generic-site-resources.xml file then OVER WRITES the version of the same file that was stored in the repository.  You can think of this as "last edit wins".  This process is also different than "seeded" customizations, such as composer based page edits, or page creations.

So how can I get my Navigation portal resource back into the runtime?  Currently, the only way is to add the portal resource reference into the generic-site-resources.xml, which is stored in the JDeveloper file based MDS.  This way during redeployment the changes from the run time are not lost.  One way to provide some sort of a procedure of ensuring these runtime changes are added back in is to enable the source code system (i.e. SVN, .etc) to DIFF the changes.  This would mean that the exported generic-site-resources.xml would basically be copied to and then replace the current version of source that JDeveloper is targeting.

 Then you can then merge the updates through a DIFF tool, such as WinMerge:

Of course all of this would not be necessary IF you would first EXPORT the new portal resource from runtime, then IMPORT the portal resource into JDeveloper, BEFORE you redeploy.  This Scenario is basically Scenario 2, which I mentioned previous before.  With Scenario 2 (and Scenario 1 to some extent), you are now dealing with all of the other files (under the siteresources/scopedMD/{GUID} directory) that get stored in the MDS repository.  This basically means that these files will also need to be added to the JDeveloper project (source-controlled) file system.  Currently, getting these files into JDeveloper (if the files did not originally start there), is either a manual (export/import) process or through some custom script based process.  Either way the files will need to stay in sync, otherwise there is a chance of lost changes upon redeployment.

In conclusion, the “lost” portal resource behavior (Scenario 3) affects all portal resources that can perform an export from the runtime (the only portal resource that is not affected is Pages).  Using the above mentioned strategy, you can use a combination of source code control and MDS to help minimize runtime changes during redeployment of a WebCenter Portal application.