Version: Deadline 8.0
Bob is setting up a new Deadline farm at his company to support their 3ds Max workflow. After running some basic tests and pushing through a few successful renders, he asks a few artists to submit some jobs of their own.
After a few minutes, the artists report back that their jobs keep failing to render on any machine that’s not their own. Bob takes a look at the error reports, and discovers they all have a “Cannot load scene” error.
After a bit of investigating, Bob discovers that the error occurs because the scene files for these jobs reside on the artists’ workstations, so the render nodes aren’t able to find them at render time. It also explains why the renders work fine when an artist renders on their own machine.
One option to resolve this issue would be for the artists to save their scenes on his company’s shared file server, which all machines have access to. However, Bob’s company already has an established pipeline where artists keep their scenes on their machines. He needs a way to get these scene files to the render nodes, and Deadline already has a system to do this!
Submitting Auxiliary Files
Auxiliary files is a fancy term used to describe any files that are submitted with a job to Deadline. During submission, the auxiliary files are copied to the job’s folder in the Deadline Repository, and are then synced to the render nodes when the Deadline Slave picks up the job.
So in Bob’s case, all the artists needs to do is include the scene file as an auxiliary file. Most of the submission scripts that ship with Deadline already support this, so he shows them the “Submit Scene File With Job” option in the 3ds Max submitter, and asks them to enable the option and try submitting again.
This time, the jobs render successfully! This is because the scene file is synced to the render nodes, and they are able to load it at render time.
Be Aware of Auxiliary File Sizes
With a few succeful renders in the bank, Bob finishes rolling out Deadline, and for a time all is good. However after some time, the artists start to notice that some of their jobs take a long time to submit. Upon further inspection, Bob notices that some of their scene files are over a GB in size! He discovered this using the Auxiliary File Size column in the Job Panel in the Monitor.
Bob also realizes that his Deadline Repository server is starting to run out of disk space. When initially setting up Deadline, he had installed it on a separate machine, rather than use his company’s existing file server (which has much more available disk space). He needs to find a solution to this problem, and fast, or job submissions will start to fail!
Alternate Auxiliary File Location
If Bob could go back and start over, he would have set up the Repository on his company’s file server. This would solve the disk space issue, and it would also improve the submission time speed because the server is built to handle large files. In fact, Bob plans on migrating the Repository after their current production work is complete, but he needs a solution now!
It turns out that Deadline already has a solution, and it’s to specify an alternate location to store the auxiliary files. This will allow Bob to configure Deadline to store the auxiliary files on his company’s file server, while still keeping the Repository in on its current machine.
To do this, Bob opens the Deadline Monitor, enters Super User Mode from the Tools menu, and selects Tools -> Configure Repository Options. He selects the Job Settings page on the left and clicks on the Auxiliary Files tab. There is a check box conveniantly called “Store job auxiliary files in a different location”.
After enabling this option, he then adds the path to the “jobs” folder he created on his company’s file server. He only needs to enter in the Windows path (since they have a 3ds Max pipeline), but if he had a heterogeneous farm, he could enter in the path that would be used on each operating system.
Now when artists submit their jobs, their job information will be submitted to the Repository, but their auxiliary files will be copied to and synced from the company’s file server. Not only did this improve submission speed, but it also freed up disk space on the Repository machine. Now the company can finish their current production efficiently, while Bob plans the Repository migration after production ends.
Browsing Auxiliary Files
An artist approached Bob and asked him an interesting question. She had previously submitted a job to Deadline a week ago that finished rendering, and she was pretty happy with the results. However, she was asked to make some changes to the scene. After making the changes, she submitted the job again, but unfortunately the results didn’t look as good as the original.
She asked Bob if he could help her find the original scene file so she could make a few minor tweaks to it and render again. Luckily, the job was still in the Deadline farm, which means its auxiliary file was still in the alternate auxiliary file location! To find it, all she had to do was right-click on the job in the Monitor and select Auxiliary Files -> Explore.
She was able to copy that file back to her workstation, make her tweaks, and resubmit!
To Be Continued?
After productioned wrapped up, Bob was able to successfully migrate his entire Repository to the company’s file server by following the Repository Migration Documentation. Now, he’s been tasked with writing a custom Deadline plugin to support an in-house application that the company develops and uses.
It turns out this application will also require auxiliary files, which means Bob will have to use Deadline Command to submit these files with the jobs. He will also have to get the auxiliary file information in his plugin from Deadline’s API.
This is going to require a deeper dive, and a new blog entry...