Introduction:
SquidNet is an advanced network render management system used to reliably distribute, control and monitor render jobs on small, medium and large scale render farms. Jobs are managed and distributed evenly across all contributing CPUs ensuring that your farm’s performance is fully maximized. Using the latest networking technologies, SquidNet is designed, unlike most render management systems, to be easily installed on your farm with little or no configuration hassles.
SquidNet uses highly efficient and scalable load-balancing algorithms to effectively manage complex renderfarm work flows. Submit jobs using application templates that are tailored specifically for that application. Once your jobs are submitted, SquidNet distributes your request based on your jobs processing criteria (pool, specified nodes, etc…) without any further user intervention.
With SquidNet’s flexible design, adding additional features is never an issue. If there’s a feature that SquidNet doesn’t currently support, no problem! Just let us know and we’ll implement most features at no cost to you. In fact, most, if not all, of SquidNet’s advanced features come from ideas and recommendations from existing customers.
You can try out SquidNet for free! Just drop us a quick email and we’ll send you a free 15-day unlimited-use evaluation license.
2011 Feature Roadmap
In-app plugins: Submit job requests directly from within your applications.
New SquidNet User Interface: Support for WINDOWS, Linux and Mac.
SquidNet SDK: Embed SquidNet functionality within your existing applications or create your own.
SQL Database support: Connect to any SQL-based database engine for report generation and statistics.
Enhanced command line interface: Submit jobs from any shell script (Bat, Python, Perl, etc…)
Basic Render Management Features
Customizable Interface: Support for custom features and workflows via configurable application templates and plugins. No charge for feature customizations that benefit all SquidNet users. Cross-platform support: Install SquidNet on all Windows and x86-based Linux distributions. (Mac support coming soon!!) Unlimited render nodes: SquidNet can manage any size render farm. Choose from peer-to-peer (default) or client/master/slave configurations. Dynamic load balancing: SquidNet automatically determines in real time how to distribute jobs to partially-active and idle CPUs. Project-based workflow: All jobs are managed within project specific folders to help you better manage your customer scenes. Job Control: Start, stop, suspend, resume render jobs in real time. Job States: Job states include Queued, Processing, Interrupted, Suspended, Resumed, Error and Completed. Efficient Power Management: Automatically start offline nodes when jobs are queued. Automatically shutdown nodes when nodes have been idle for a while.
Installation:
Simple installer: Install SquidNet on your system using a single installer. No need to go thru any complicated installation procedures. One-step Network Installer: Upgrading your renderfarm to the latest version of SquidNet by simply selecting nodes to update and selecting “Update”. Automatic Node Detection: All nodes in SquidNet network automatically find each other. No need to go thru complicated configuration procedures. Self-contained: All SquidNet operational files are completely contained within the installation directory. No files are ever copied to any system folders thus ensuring complete isolation from your existing workflow.
Interface:
Single interface: All management and control functions are done from a single GUI-based interface console.
Multi-view: All control and monitoring functions are contained within built-in single windows called “views”. Different views exist for job monitoring, network monitoring, CPU system, workflow progress and many others.
Customizable Interface: Arrange all monitoring “views” into your own custom layout. Choose from up to 4 customer layers.
Command line interface: Submit render jobs using SquidNet’s command line interface.
Standalone Image And Sequence viewer: Use the snview application to view and play your animation sequencences.
Advanced Features:
Multi-application support: Supports all popular 3D modeling and rendering applications. H.264 video support: Automatically create H.264 videos from rendered frames. Theme-based interface: Pick from many different user interface themes. User Management: Assign user’s specific SquidNet accounts with admin-controlled permissions. Distributed management: Automatic load-balancing of all jobs across your renderfarm. Batch Rendering: Submit multiple scenes in a single batch job. Transfer Queue: Automatically transfer completed jobs to you customer’s site using SSH secured SCP/SFTP or standard FTP. Built-in image and sequence viewer: View completed images and frame sequences. Job Priority management: Assigned up to 25 different priorities to each job request. Job dependencies: Schedule the order that jobs are processed. Email and SMS: Email and SMS text messaging of job status. Application Path Manager: Manage your application’s installation paths from a single interface. Tile Rendering: Render single frames amongst multiple render nodes.
Where SquidNet came from?
SquidNet came into being because of the need to have a small inexpensive in-house rendering system. At the time (around 2004) most of the more popular rendering packages were too expensive for a small in-house network. Initially, SquidNet was just a few DOS batch scripts that copied Maya and 3DSMAX project files between mapped directories of a small 486/Pentium-based renderfarm network. From there, it became a Visual Basic (VB) program that replaced the existing DOS batch files. Today, SquidNet is built on top of a modern object-oriented framework using the latest in software development, multi-threading and networking technologies. Today, SquidNet posses a scalable and flexible architecture that easily outperforms the more complicated (and expensive) network management applications.
How SquidNet works?
The system is made up of two components: the SquidNet User Interface Console (UI) and the individual network render nodes . The UI console is the main interface into the system. It allows users to create, submit and monitor jobs within the SquidNet network. The render nodes are the workhorses of the network. They’re responsible for pulling job segments off the network queue and processing them using application specific programs (Maya, 3DMAX, LightWave, etc…) residing on the render hosts.
At a high level, the system as a whole behaves like this: users submit render job requests to the network queue via the SquidNet user interface console. From there, one or more render nodes monitor the network queue; pull job entries, process them using 3rd part rendering and compositing applications, then update the queue with the results of the request. All this is done seamlessly behind the scenes without any user intervention. What sets SquidNet apart from other distributed processing systems is that there’s no central point of failure. All render nodes function independent of each other; if any one node is unexpectedly removed from the network (due to user intervention or hardware failure), the rest of the network will continue to function as expected. All jobs are submitted to a virtual network queue that is accessible to any render node on the network (as long as it resides on the same network subnet). Within the queue, SquidNet job requests are indexed based on their priority level (0-24). Higher priority jobs (Level-0) are always processed first. If two or more jobs have the same priority level, the oldest job in the queue takes precedence. Render nodes continuously pull jobs off the queue until all jobs have been completely processed.
Licensing Information:
- SquidNet Deluxe is free and can be downloaded and installed without charge. The download includes 8 free core licenses.
- SquidNet Professional requires the purchase of a special license key that unlocks all advanced features. When purchased, you’ll be given (and emailed) a license key. Please safeguard license keys for future installations and upgrades. Purchased licenses have no expiration date and are good for all future updates and added features.
- SquidNet Core Floating Licenses are what enable the real processing power of SquidNet. By default, 8 core licenses are installed. Ideally, you should have one core license for every physical CPU core on your system.
- For any additional questions on licensing, please send us an email.
*** All licenses are guaranteed never to expire and will work on all future versions of SquidNet ***
Render farm
A render farm is a computer cluster built to render computer-generated imagery (CGI), typically for film and television visual effects, using off-line batch processing. This is different from a render wall, which is a networked, tiled display used for real-time rendering[1]. The rendering of images is a highly parallelizable activity, as each frame usually can be calculated independently of the others, with the main communication between processors being the upload of the initial source material, such as models and textures, and the download of the finished images.
Over the decades, advances in computer power would allow an image to take less time to render. However, the increased computation is instead used to meet demands to achieve state-of-the-art image quality. While simple images can be produced rapidly, more realistic and complicated higher-resolution images can now be produced in more reasonable amounts of time. The time spent producing images can be limited by production time-lines and deadlines, and the desire to create high-quality work drives the need for increased computing power, rather than simply wanting the same images created faster.
To manage large farms, one must introduce a queue manager that automatically distributes processes to the many processors. Each “process” could be the rendering of one full image, a few images, or even a sub-section (or tile) of an image. The software is typically a client–server package that facilitates communication between the processors and the queue manager, although some queues have no central manager. Some common features of queue managers are: re-prioritization of the queue, management of software licenses, and algorithms to best optimize throughput based on various types of hardware in the farm. Software licensing handled by a queue manager might involve dynamic allocation of licenses to available CPU’s or even cores within CPU’s. A tongue-in-cheek job title for systems engineers who work primarily in the maintenance and monitoring of a render farm is a render wrangler to further the “farm” theme. This job title can be seen in film credits.
Recent research has explored the feasibility of reprogramming modern video cards to do rendering in the card’s hardware.[2]
Server farm
A server farm or server cluster is a collection of computer servers usually maintained by an enterprise to accomplish server needs far beyond the capability of one machine. Server farms often have backup servers, which can take over the function of primary servers in the event of a primary server failure. Server farms are typically co-located with the network switches and/or routers which enable communication between the different parts of the cluster and the users of the cluster. The computers, routers, power supplies, and related electronics are typically mounted on 19-inch racks in a server room or data center.
Applications:
Server farms are commonly used for cluster computing. Many modern supercomputers comprise giant server farms of high-speed processors connected by either Gigabit Ethernet or custom interconnects such as Infiniband or Myrinet. Web hosting is a common use of a server farm; such a system is sometimes collectively referred to as a web farm. Other uses of server farms include scientific simulations (such as computational fluid dynamics) and the rendering of 3D computer generated imagery (also see render farm).
Server farms are increasingly being used instead of or in addition to mainframe computers by large enterprises, although server farms do not yet reach the same reliability levels as mainframes. Because of the sheer number of computers in large server farms, the failure of individual machines is a commonplace event, and the management of large server farms needs to take this into account, by providing support for redundancy, automatic failover, and rapid reconfiguration of the server cluster.
Performance:
The performance of the very largest server farms (thousands of processors and up) is typically limited by the performance of the data center’s cooling systems and the total electricity cost rather than by the performance of the processors.[1] A computer that runs 24/7 consumes (over its lifetime) electricity worth many times its initial purchase cost. For this reason, the critical design parameter for both large and continuous systems tends to be performance per watt, rather than cost of peak performance or (peak performance / (unit * initial cost)). Also, for high availability systems that must run 24/7 (unlike supercomputers that can be power-cycled to demand, and also tend to run at much higher utilizations), there is more attention placed on power saving features such as variable clock-speed and the ability to turn off both computer parts, processor parts, and entire computers (WoL and virtualization) according to demand without bringing down services.
