Cernunnos Overview
- Arlo White
Introduction
Cernunnos is a scripting language developed by some of the JA-SIG developers and is the workhorse behind database export and import in uPortal. For more information on Cernunnos, visit the project website. Cernunnos is a bit like ant in that it's xml based and has somewhat similar syntax. There is some uP2 XML Import Export with Cernunnos that goes into more detail than this manual. The language makes it very easy to do things like run an SQL query and then do something with the results, e.g. create an xml file. Here is an exceprt from export-layout.crn:
<with-attribute key="Attributes.NODE" value="${newDoc(structures)}">
<append-node>
<username>${USER_NAME}</username>
</append-node>
<sql-query>
<sql>SELECT * FROM up_layout_struct WHERE user_id = ? AND layout_id = 1 AND chan_id IS NULL</sql>
<parameter value="${USER_ID}"/>
<subtasks>
<choose>
<when test="${jexl(TYPE != null and TYPE.startsWith('dlm:'))}">
<with-attribute key="DLM_NODEREF" value="${NAME}">
<with-attribute key="PARSED_NODE" value="${crn(lookup-dlm-noderef.crn)}">
<choose>
<when test="${jexl(PARSED_NODE == null or (PARSED_NODE[0] != null and PARSED_NODE[1] != null and PARSED_NODE[2] != null))}">
<append-node>
<dlm-node
struct-id="${STRUCT_ID}"
name="${groovy(PARSED_NODE == null ? NAME : PARSED_NODE[0] + ':' + PARSED_NODE[1])}"
fname="${groovy(PARSED_NODE == null ? null : PARSED_NODE[2])}"
next-struct-id="${NEXT_STRUCT_ID}"
child-struct-id="${CHLD_STRUCT_ID}"
type="${org.jasig.portal.io.FolderTypePhrase(${TYPE})}"
hidden="${org.jasig.portal.io.FlagYorNPhrase(${req(HIDDEN)})}"
immutable="${org.jasig.portal.io.FlagYorNPhrase(${req(IMMUTABLE)})}"
unremovable="${org.jasig.portal.io.FlagYorNPhrase(${req(UNREMOVABLE)})}"/>
</append-node>
Usage in uPortal
The uPortal crn-export and crn-import tasks hook into the cernunnos scripts found under uportal-impl/src/main/resources/org/jasig/portal/io. This directory may seem overwhelming, but the execution is actually very hiarchial and straightforward. If you are exporting, ant starts with export.crn, importing starts with import.crn. If you examine export.crn you see that if you called crn-export with -Dtype=all export runs through a number of types in export_internal.crn. Otherwise, it calls export_internal.crn with that type. If you look at export_internal.crn you can see the entry points into scripts such as export-layout.crn or export-user.crn. You don't need to know the Cernunnos language to understand what's going on; don't be afraid to delve into the scripts and follow the execution path.
Troubleshooting
Inevitably you will run into some kind of error from the export or import process. If you're lucky it will have a straightforward explanation of the problem. You will see a giant stack trace like this:
awhite@awhite:~/Code/EclipseWS/Poly_uPortal_3$ ant crn-import -Ddir=/home/awhite/Code/import/indus2/aw_layouts
[java] Base Import Directory=/home/awhite/Code/import/indus2/aw_layouts
[java] FILE_PATTERN=.*
[java] Import Layout: awaechte.layout
[java] (Location of error unknown)java.lang.RuntimeException: Unable to read the specified resource from the classpath: org/jasig/portal/io/import-layout.xsl
[java] Exception in thread "main" java.lang.RuntimeException: Unable to read the specified properties file: //properties/db/entities/import.properties
[java] at org.danann.cernunnos.core.PropertiesTask.perform(PropertiesTask.java:96)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.runtime.ScriptRunner$TaskDecorator.perform(ScriptRunner.java:415)
[java] at org.danann.cernunnos.runtime.ScriptRunner.run(ScriptRunner.java:267)
[java] at org.danann.cernunnos.runtime.ScriptRunner.run(ScriptRunner.java:204)
[java] at org.danann.cernunnos.runtime.ScriptRunner.run(ScriptRunner.java:156)
[java] at org.danann.cernunnos.runtime.Main.main(Main.java:59)
[java] Caused by: java.lang.RuntimeException: Unable to iterate files in the specified way.
[java] dir=/home/awhite/Code/import/indus2/aw_layouts
[java] includes=**/*.layout
[java] excludes=[Not Evaluated]
[java] at org.danann.cernunnos.io.FileIteratorTask.perform(FileIteratorTask.java:158)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.core.SetAttributeTask.perform(SetAttributeTask.java:66)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.core.WithTask.perform(WithTask.java:90)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.core.WithTask.perform(WithTask.java:90)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.core.PropertiesTask.perform(PropertiesTask.java:92)
[java] ... 6 more
[java] Caused by: java.lang.RuntimeException: Unable to invoke the specified script: classpath://org/jasig/portal/io/import-layout_v2-6.crn
[java] at org.danann.cernunnos.flow.CernunnosTask.perform(CernunnosTask.java:117)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.core.SetAttributeTask.perform(SetAttributeTask.java:66)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.flow.IfTask.perform(IfTask.java:62)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.io.FileIteratorTask.perform(FileIteratorTask.java:151)
[java] ... 22 more
[java] Caused by: org.danann.cernunnos.ManagedException: The Cernunnos Runtime encountered an error:
[java] Entity Name: <xslt>
[java] Source: /with/subtasks/sql-transaction/xslt
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:93)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.sql.TransactionTask.access$300(TransactionTask.java:40)
[java] at org.danann.cernunnos.sql.TransactionTask$PerformSubtasksTransactionCallback.doInTransactionWithoutResult(TransactionTask.java:141)
[java] at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
[java] at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:128)
[java] at org.danann.cernunnos.sql.TransactionTask.perform(TransactionTask.java:115)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:136)
[java] at org.danann.cernunnos.AbstractContainerTask.performSubtasks(AbstractContainerTask.java:111)
[java] at org.danann.cernunnos.core.WithTask.perform(WithTask.java:90)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] at org.danann.cernunnos.runtime.ScriptRunner$TaskDecorator.perform(ScriptRunner.java:415)
[java] at org.danann.cernunnos.runtime.ScriptRunner.run(ScriptRunner.java:267)
[java] at org.danann.cernunnos.flow.CernunnosTask.perform(CernunnosTask.java:113)
[java] ... 34 more
[java] Caused by: java.lang.RuntimeException: Unable to perform the requested transformation with XSL from
scriptEngine='classpath://org/jasig/portal/io/import-layout_v2-6.crn' and script='import-layout.xsl'
[java] at org.danann.cernunnos.xml.XslTransformTask.perform(XslTransformTask.java:185)
[java] at org.danann.cernunnos.runtime.RuntimeTaskDecorator.perform(RuntimeTaskDecorator.java:77)
[java] ... 49 more
[java] Caused by: java.lang.NullPointerException
[java] at org.danann.cernunnos.xml.XslTransformTask.perform(XslTransformTask.java:168)
[java] ... 50 more
The key to troubleshooting the stack trace is knowing that the last element is the most inward exception (opposite from typical Java stacktraces). There is nothing wrong with your import.properties file, this is simply the first thing cernunnos does because import.crn starts with:
<properties location="classpath://properties/db/entities/import.properties">
Look at the bottom of the stacktrace, here is your real problem. If you look at the upper Caused by's you can figure out where Cernunnos was. In this case, it was in the import-layout_v2-6.crn file, with the node hierarchy: /with/subtasks/sql-transaction/xslt
You can dig into that file to try to find the cause of the exception.
Sometimes Cernunnos will give you a straightforward error, such as:
[java] Caused by: java.lang.RuntimeException: Error looking up the specified member: ITS Staff
This one was actually because I hadn't setup my personContext.xml file, which defined this group. (Yes Cernnunos checks against UP_GROUP and PersonDirectory groups)
Sometimes when it has trouble looking up values from the database you might see something like:
[java] Caused by: org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [INSERT INTO up_ss_user_parm(param_val,
user_id, profile_id, ss_id, ss_type, param_name) values(?, ?, ?, ?, 2, ?)]; ORA-01400: cannot insert NULL into ("AWHITEUP3"."UP_SS_USER_PARM"."SS_ID")
[java] ; nested exception is java.sql.SQLException: ORA-01400: cannot insert NULL into ("AWHITEUP3"."UP_SS_USER_PARM"."SS_ID")
...
[java] Caused by: java.sql.SQLException: ORA-01400: cannot insert NULL into ("AWHITEUP3"."UP_SS_USER_PARM"."SS_ID")
Again, follow the execution path to find where it got the null value from.