References and render layers..

It can be time-consuming enough to set up your render layers: assigning objects, assigning material overrides...but you could also be in for a nasty surprise when you browse through your render layers and Maya spits out errors, like this one:

In this post, I'll tell you how to avoid these errors, and how to keep those render layer assignments.

Let me begin this post by creating a simple scenario that allows me to quickly demonstrate the fragility of working with reference files and render layers...

1) Create a couple of Maya scenes:

In the first scene:

• create a single poly sphere. Add it to a render layer, and call this: characterLayer.
• save the scene as character.ma

In the second scene:

• create a poly plane, a couple of materials, and two render layers.
• change the colours of the materials so that one is blue, and the other red.
• rename your render layers: redPerObject and bluePerLayer. You guessed it, this describes the material override method for each render layer.
• so, assign the poly plane to bluePerObject, and assign it with the blue material.
• assign the poly plane to redPerLayer and set the layer material override to use the red material.
• save the scene as set.ma

2) Reference your scenes and assign render layer overrides:

In a new Maya scene:

• reference in your two base scenes: characters.ma, and set.ma
• assign set:pPlane1 to the render layer: character:characterLayer
• assign character:pSphere1 to the render layers: set:redPerObject and set:bluePerLayer.
• select the render layer set:redPerObject, and assign the blue material to pPlane1 manually.
• save the scene as shot1.ma.

3) The problem:

At this point you can flip through the render layers without any fear of losing your overrides.

But watch what happens when you do this:

1. set set:redPerObject as the current render layer.
2. from the Reference Editor, unload and reload the reference file set.ma
3. flip through the render layers again....

Hang on....why is there an extra renderlayer called set:redPerObject1 ?

And if I make it the active render layer, I get the error:

// Error: Connection not made: 'set:pPlaneShape1.instObjGroups[0]' -> 'set:blinn1SG.dagSetMembers[-1]'.  Source node will not allow the connection.

Well...

In a nutshell, when the current render layer is from a referenced file you are trying to reload, Maya is unable to disconnect the render layer from the render layer manager...so the render layer stays in the scene instead of being unloaded.

When you reload your reference, Maya renames the incoming render layer with a slightly different name and thus cannot resolve the reference edits, which connect the objects from your reference file to other elements in the scene.

4) ...and if that wasn't bad enough:

Highlight the unwanted render layer: set:redPerObject, and delete it.

Re-assign the sphere to set:redPerObject1 and create a new new material override, and get the objects re-assigned to the appropriate render layers.

Now you've made the problem even worse.

Save the scene as shot2.ma, and re-open it.

Unfortunately, at this point, you have little chance of salvaging your material overrides, unless you manually delete the incorrect reference edits - which can be done if you look at the error messages in the Script Editor for clues as to which reference edits can't be evaluated, and should therefore be removed.

5) The solution:

Well, the solution is quite simple : make sure that you set the current render layer to the masterLayer before unloading and reloading a reference file.

If I go back to shot1.ma and want to update the reference file character.ma, there is one extra step:

1. set the current render layer to masterLayer
2. Reload the reference character.ma or set.ma
   

If you don't like the sound of that, then perhaps you'd like to download a script that does this for you.

1. download the script unloadReference.py
2. save it in your user/maya/20xx/scripts directory
3. add a couple of lines to your userSetup.mel file:
   
   python "import unloadReference.py";
   python "unloadReference.setup()";
   
   
4. Reload shot1.ma

Now, when you unload any referenced file, the script will use a callback to first set the active render layer to the masterLayer. Another callback will do the same thing before you reload your reference.

I realise this solution may come a little too late for some of you, as it's only a problem in versions of Maya up to and including Maya 2010 - and this is not a fix for scenes that already contain unresolvable reference edits - but I hope that the script I provided may save you some time, and sanity.

Cheers,

Owen

Comments

[Amorano]

Sorry, just to clarify, it is still the same behavior with references that have overrides applied on layers that then get de-referenced/unloaded.

Not 100% the same, but we see the same error behavior.

[Amorano]

Just an FYI, but this behavior is NOT fixed in 2012.

It is still a major issue. Granted, we have placed/placing the code in our startup routines for all users, but it is still broken (material overrides) unless you explicitly switch layers.

Cheers.

[jeroen]

is there any way I can make the fixRenderLayerOutAdjustmentErrors; execute every time when opening a scene?

the command fixes my problem but after saving and reopening the problem it back.

[mBort]

Hi, thx for the workaround tip.
By the way.... that scripts("unloadReference.py") does not work with Referenced Render Layers in your scene.

[masternull]

always saving scenes with the masterlayer applied seems to help...

[buy viagra]

I'm always wondering why the universe keeps creating spheres in every creation, been a supernova, galaxy, planet, star, it is always a sphere, maybe we should pay attention to the sacred geometry that the ancient cultures use.

[owen]

Kiernan, you're dead right, what I wrote was ambiguous.
So, the script unloadReference.py needs to be used in versions of Maya up to and including Maya 2010.
I edited the post already. Thanks for flagging it!

O

[Kiernan]

Awesome, thanks for the fix. When you mean "as it's only a problem up to Maya 2010". Does this mean Maya 2010 fixes this problem?