Version: Deadline 8.0
We get an awful lot of questions from clients about how to harness the power of Cloud computing. How do I expand my render farm to the Cloud? Which Cloud provider should I use? How do I move all of my render farm to the Cloud?
I’ve been working on integrating Deadline with a bunch of the different Cloud service providers over the past few years and I’ve learned a lot of lessons about rendering on the Cloud. Hopefully, I can go over the basic concepts of Cloud rendering and answer some of these questions.
Provider-ing Some Insight
The first question you have to ask yourself when it comes to using the Cloud for a render farm is what Cloud service provider are you going to use?
There are many Cloud providers to choose from, and typically each one provides the necessary tools and infrastructure to set up a Cloud-hosted render farm. In fact, we’ve already had success setting up render farms on Amazon EC2, Google Compute Engine and Microsoft Azure. When it comes to choosing which one is right for you, we recommend taking some time to compare their features and pricing for yourself so you can make an informed decision.
If you’re just starting out, I think it’s safe to skip over most of the advanced features like the ones detailed in the section immediately below.
Here are a couple things about the providers that we often get interest on.
Google and Amazon both offer discounted instances. These instances are much cheaper than normal instances but they might be terminated at any time by the provider. Otherwise these instances act much like regular instances.
- Google has Preemptible Instances. They only run for 24 hours and can be terminated by Google prior to that.
- Amazon has Spot Instances. These instances work on a bidding system. Basically, when you start an instance, you also set the maximum price you’re willing to pay for it. If the “going rate” of that instance type is lower than your bid, then you’ll get your instance. You’ll then get charged the “going rate” of that instance for an hour. At the start of the next “instance hour” the bid price and “going rate” are compared again. If your bid is lower than the going rate then your instance will be terminated. This might happen if other people are willing to pay more for the instance type than you are. I recommend looking at the pricing history of spot instances before setting a bid price. If reliability is something that you’re looking for I might suggest bidding the On-Demand price, as typically, Spot Instances won’t get that high. You’ll both get a discount and get your instances.
Google, Amazon and Azure all offer both Linux and Windows instances. None of the providers provide Mac OS X instances.
Deadline also supports Openstack for a virtual private cloud solution.
Where’s the Repository?
I think the most important question you need to ask yourself before venturing into the Cloud is, how do I want this to look? Are you going to use the Cloud to supplement your existing render farm? Do you want a separate render farm in the Cloud to run alongside your local render farm? Or do you not have an existing render farm and want to create a brand new farm in the cloud? There are a lot of ways you can go about setting up your infrastructure on the Cloud.
Don’t think about the Cloud as a strange new platform. Think of it as a remote office. Many of the concepts and considerations you’d use when starting a remote office also apply to the Cloud, except nobody works there.
Firstly, figure out where you want the core infrastructure, the Repository and Database, of your Deadline render farm to live. You can use your existing Deadline infrastructure, that’s fine, and have your Cloud render nodes connect to it.
The other way to go is have your Repository and Database on the Cloud, and have your on premise render nodes connect to it.
Or you could do both. Have your Cloud render nodes connect to the infrastructure in the Cloud and keep your existing render farm the same as it currently is. You can submit the jobs you want to render locally to your local Deadline infrastructure and the jobs you want to be rendered on the Cloud can go to the Deadline infrastructure there. You might consider using this type of setup because of bandwidth and network latency. Having your Deadline Infrastructure and your render nodes all in the Cloud means that their communication will be much faster than going over the internet.
Check out the Quick Repository and Database Installation Guide for more information. We also have an Advanced Repository and Database Installation Guide that covers more advanced installation options.
Another option is to use a Deadline Proxy Server and have the nodes connect to your infrastructure through that. The Proxy Server is typically run in the same location as the Repository and Database. It serves as the connection point for your remote nodes. This is useful if you don’t have a VPN, but still require a secure connection (please see our documentation on Nginx about creating a webserver for SSL Security). You can even use the Proxy Server with a VPN to help with low bandwidth and/or high latency connections.
Here’s an example of using the Proxy Server with on premise infrastructure.
Here’s an example of using the Proxy Server with Cloud infrastructure.
Getting your assets to and from the Cloud is an important step in the rendering process. Assets are any files your jobs need to render. Assets are also any output your jobs create. Getting your files up to the Cloud and back down to your local network is a key part of Cloud rendering.
You can create a Virtual Private Network (VPN) between your local network and your Cloud network. The instances will be able to access your assets the same way the on premise nodes do.
Some companies do not allow for VPN use. In that case, you probably want to move all your required assets up to a file server that’s shared out to your nodes. You can manually move the files up and down, you can use a Dropbox, OneDrive type service, or you could get a little fancier with Aspera.
Another option is to use Cloud Storage. All of the providers offer Cloud Storage as a service, like Amazon S3 for example. Please consult your provider for more information, such as rates and availability.
If you decide not to use a VPN, you’ll probably need to use Deadline Path Mapping so that your nodes will be able to find the assets properly.
A License to Render
Licensing is a sometimes overlooked aspect of the Cloud. How do you license Deadline and all the software that you use? Here are a few ways of tackling this issue.
Much like assets, you can setup a Virtual Private Network between your on premise network and your Cloud network. With a VPN, your instances will be able to connect to your current license server exactly like your on premise machines do.
You can put a license server in the Cloud and have your instances get their licenses from there, however, I don’t recommend that as instances can be terminated (due to outages, etc.). Restarting your license server will become a problem as most licenses use the MAC Address of the machine they’re running on for authentication. Your restarted license server won’t have the same MAC Address as your old one, thus breaking your licensing.
Another option is to use Deadline Usage Based Licensing. You only pay for the render time you use, you can start as many nodes as you want (you don’t need a floating license for each node), and you don’t need a license server. You can use Usage Based Licensing to license Deadline itself along with a number of 3rd party rendering software. Check out the Thinkbox Store for pricing and availability.
Usage Based Licensing can also be used to supplement all of your existing licenses. What’s awesome about that is that all your on premise render nodes can use the permanent licenses you already have, while the Cloud render nodes only use what they need with Usage Based Licensing.
An Image is Worth a Thousand Renders
Once you have a Repository setup it’s time to make a render node. I like to do it as follows.
- Start a fresh instance using the tools provided by your Cloud provider.
- Install the Deadline Slave onto the instance. See the Quick Client Installation Guide or the Advanced Client Installation Guide for more information.
- Start the Deadline Slave on the instance and make sure it connects with the Repository and obtains a license.
- Configure the Deadline Slave to automatically launch when the Deadline Launcher starts up. Restart the instance and make sure the Slave starts and connects automatically.
- Take an image of this instance. That way, you’ll have a blank Deadline Slave image you can always use if you need to make another render node image with new software on it.
- Install the software you want for your rendering, and make sure the licensing works.
- Submit a job to Deadline to test the render.
- Image the instance.
If you need to make a new render node with new software on it, all you have to do is start the blank Deadline Slave image you initially made, install the new software, test it out, and then image it.
For Your Consideration
Here are a few details you can think about once you’ve got the basics down.
Sometimes providers have issues with their data centers, causing instances to be shutdown. How do we handle that situation?
Deadline Slave nodes usually don’t have data stored on them so adding them into auto scaling groups (all three providers have this feature) is a good way to maintain a constant (or sometimes varying) amount of instances. You may lose some output if the render nodes fails during a render. That’s much harder to avoid. Running shorter render tasks can help avoid this.
Losing a Repository or file storage can be much worse. I recommend taking regular back-ups.
All of the providers have security options in place. I highly recommend you take a look at their documentation and take steps to secure your infrastructure.
Getting different levels of monitoring setup on your infrastructure is another useful step to take. Monitoring will allow you to check the health of all the instances and subnetworks that are a part of your infrastructure. The providers will let you know if anything goes wrong or if there are bottlenecks in particular areas. You can also get detailed usage and billing reports from this feature. Consult your provider’s documentation about how to do this.
Conclusion: Just the Beginning
And that’s just a primer about how to get started with Deadline on the Cloud. There’s certainly a lot more to know, but that’s for a deeper dive in a future blog post.
If you have any questions about rendering on the Cloud or are interested in getting setup, feel free to contact Thinkbox Support.