Return to site

Clusters For Mac

broken image


Clustal 2 comes in two flavors: the command-line version Clustal W and the graphical version Clustal X. Precompiled executables for Linux, Mac OS X and Windows (incl. XP and Vista) of the most recent version (currently 2.1) along with the source code are available for download here. Install VirtualBox for Mac using Homebrew. Run brew cask install virtualbox in your Terminal. VirtualBox lets you run virtual machines on your Mac (like running Windows inside macOS, except for a Kubernetes cluster.) Skip to step three if everything has worked to this point. To use Tanzu Kubernetes Grid, you download and run the Tanzu Kubernetes Grid CLI on a local system, known as the bootstrap environment. The bootstrap environment is the laptop, host, or server on which the initial bootstrapping of a management cluster is performed. This is Scimark Processors series extending to clusters for Mac systems based on x86-64bit nodes. It aims to make a reasonable benchmark for clusters, showing and comparing their computing power.

The following tables compare general and technical information for notable computer clustersoftware. This software can be grossly separated in four categories: Job scheduler, nodes management, nodes installation and integrated stack (all the above).

General information[edit]

SoftwareMaintainerCategoryDevelopment statusArchitectureOCSHigh-Performance/ High-Throughput ComputingLicensePlatforms supportedCostPaid support available
AcceleratorAltairJob Scheduleractively developedMaster/worker distributedHPC/HTCProprietaryLinux, WindowsCostYes
AmoebaNo active developmentMIT
Base One Foundation Component LibraryProprietary
DIETINRIA, SysFera, Open SourceAll in oneGridRPC, SPMD, Hierarchical and distributed architecture, CORBAHTC/HPCCeCILLUnix-like, Mac OS X, AIXFree
Enduro/XMavimax, Ltd.Job/Data Scheduleractively developedSOA GridHTC/HPC/HAGPLv2 or CommercialLinux, FreeBSD, MacOS, Solaris, AIXFree / CostYes
GangliaMonitoringactively developedBSDUnix, Linux, Windows NT/XP/2000/2003/2008, FreeBSD, NetBSD, OpenBSD, DragonflyBSD, Mac OS X, Solaris, AIX, IRIX, Tru64, HPUX.Free
Globus ToolkitGlobus Alliance, Argonne National LaboratoryJob/Data Scheduleractively developedSOA GridLinuxFree
Grid MPUniva (formerly United Devices)Job Schedulerno active developmentDistributed master/workerHTC/HPCProprietaryWindows, Linux, Mac OS X, SolarisCost
Apache MesosApacheactively developedApache license v2.0LinuxFreeYes
Moab Cluster SuiteAdaptive ComputingJob Scheduleractively developedHPCProprietaryLinux, Mac OS X, Windows, AIX, OSF/Tru-64, Solaris, HP-UX, IRIX, FreeBSD & other UNIX platformsCostYes
NetworkComputerRuntime Design Automationactively developedHTC/HPCProprietaryUnix-like, WindowsCost
OpenHPCOpenHPC projectall in oneactively developedHPCLinux (CentOS)FreeNo
OpenLavaTeraprocJob Scheduleractively developedMaster/Worker, multiple admin/submit nodesHTC/HPCGPLLinuxFreeYes
PBS ProAltairJob Scheduleractively developedMaster/worker distributed with fail-overHPC/HTCAGPL or ProprietaryLinux, WindowsFree or CostYes
Proxmox Virtual EnvironmentProxmox Server SolutionsCompleteactively developedOpen-source AGPLv3Linux, Windows, other operating systems are known to work and are community supportedFreeYes
Rocks Cluster DistributionOpen Source/NSF grantAll in oneactively developedHTC/HPCOpenSourceCentOSFree
Popular Power
ProActiveINRIA, ActiveEon, Open SourceAll in oneactively developedMaster/Worker, SPMD, Distributed Component Model, SkeletonsHTC/HPCGPLUnix-like, Windows, Mac OS XFree
RPyCTomer Filibaactively developedMIT License*nix/WindowsFree
SLURMSchedMDJob Scheduleractively developedHPC/HTCGPLLinux/*nixFreeYes
Spectrum LSFIBMJob Scheduleractively developedMaster node with failover/exec clients, multiple admin/submit nodes, Suite addOnsHPC/HTCProprietaryUnix, Linux, WindowsCost and Academic - model - Academic, Express, Standard, Advanced and SuitesYes
Oracle Grid EngineUnivaJob Scheduleractive Development moved to Univa Grid EngineMaster node/exec clients, multiple admin/submit nodesHPC/HTCProprietary*nix/WindowsCost
SynfiniWayFujitsuactively developedHPC/HTC?Unix, Linux, WindowsCost
TORQUE Resource ManagerAdaptive ComputingJob Scheduleractively developedProprietaryLinux, *nixCostYes
UniClusterUnivaAll in OneFunctionality and development moved to UniCloud (see above)FreeYes
UNICORE
Univa Grid EngineUnivaJob Scheduleractively developedMaster node/exec clients, multiple admin/submit nodesHPC/HTCProprietary*nix/WindowsCost
XgridApple Computer
SoftwareMaintainerCategoryDevelopment statusArchitectureHigh-Performance/ High-Throughput ComputingLicensePlatforms supportedCostPaid support available

Table explanation

  • Software: The name of the application that is described

Technical information[edit]

SoftwareImplementation LanguageAuthenticationEncryptionIntegrityGlobal File SystemGlobal File System + KerberosHeterogeneous/ Homogeneous exec nodeJobs priorityGroup priorityQueue typeSMP awareMax exec nodeMax job submittedCPU scavengingParallel jobJob checkpointing
Enduro/XC/C++OS AuthenticationGPG, AES-128, SHA1NoneAny cluster Posix FS (gfs, gpfs, ocfs, etc.)Any cluster Posix FS (gfs, gpfs, ocfs, etc.)HeterogeneousOS Nice levelOS Nice levelSOA Queues, FIFOYesOS LimitsOS LimitsYesYesNo
HTCondorC++GSI, SSL, Kerberos, Password, File System, Remote File System, Windows, Claim To Be, AnonymousNone, Triple DES, BLOWFISHNone, MD5None, NFS, AFSNot official, hack with ACL and NFS4HeterogeneousYesYesFair-share with some programmabilitybasic (hard separation into different node)tested ~10000?tested ~100000?YesMPI, OpenMP, PVMYes
PBS ProC/PythonOS Authentication, MungeAny, e.g., NFS, Lustre, GPFS, AFSLimited availabilityHeterogeneousYesYesFully configurableYestested ~50,000MillionsYesMPI, OpenMPYes
OpenLavaC/C++OS authenticationNoneNFSHeterogeneous LinuxYesYesConfigurableYesYes, supports preemption based on priorityYesYes
SlurmCMunge, None, KerberosHeterogeneousYesYesMultifactor Fair-shareyestested 120ktested 100kNoYesYes
Spectrum LSFC/C++Multiple - OS Authentication/KerberosOptionalOptionalAny - GPFS/Spectrum Scale, NFS, SMBAny - GPFS/Spectrum Scale, NFS, SMBHeterogeneous - HW and OS agnostic (AIX, Linux or Windows)Policy based - no queue to computenode bindingPolicy based - no queue to computegroup bindingBatch, interactive, checkpointing, parallel and combinationsyes and GPU aware (GPU License free)> 9.000 compute hots> 4 mio jobs a dayYes, supports preemption based on priority, supports checkpointing/resumeYes, fx parallel submissions for job collaboration over fx MPIYes, with support for user, kernel or library level checkpointing environments
TorqueCSSH, mungeNone, anyHeterogeneousYesYesProgrammableYestestedtestedYesYesYes
Univa Grid EngineCOS Authentication/Kerberos/Oauth2Certificate BasedIntegrityArbitrary, e.g. NFS, Lustre, HDFS, AFSAFSFully heterogeneousYes; automatically policy controlled (e.g. fair-share, deadline, resource dependent) or manualYes; can be dependent on user groups as well as projects and is governed by policiesBatch, interactive, checkpointing, parallel and combinationsYes, with core binding, GPU and Intel Xeon Phi supportcommercial deployments with many tens of thousands hosts>300K tested in commercial deploymentsYes; can suspend job on interactive usageYes, with support of arbitrary parallel environments such as OpenMPI, MPICH 1/2, MVAPICH 1/2, LAM, etc.Yes, with support for user, kernel or library level checkpointing environments
SoftwareImplementation LanguageAuthenticationEncryptionIntegrityGlobal File SystemGlobal File System + KerberosHeterogeneous/ Homogeneous exec nodeJobs priorityGroup priorityQueue typeSMP awareMax exec nodeMax job submittedCPU scavengingParallel jobJob checkpointing

Table Explanation

  • Software: The name of the application that is described
  • SMP aware:
    • basic: hard split into multiple virtual host
    • basic+: hard split into multiple virtual host with some minimal/incomplete communication between virtual host on the same computer
    • dynamic: split the resource of the computer (CPU/Ram) on demand

History and adoption[edit]

See also[edit]

Notes[edit]

External links[edit]

Retrieved from 'https://en.wikipedia.org/w/index.php?title=Comparison_of_cluster_software&oldid=955011470'

The challenge

We use a cluster of GNU/Linux-powered nodes behind a Cisco load balancer to serve some of our web sites. These nodes being quasi-identical (most config, filesystems, etc), we often need to apply the very same changes, via ssh, to each of them at once in our daily maintenance tasks.

ClusterSSH to the Rescue

ClusterSSH is a small Perl/TK utility that controls a number of xterm windows via a single graphical console window to allow commands to be interactively run on multiple servers over an ssh connection. This means that you type your shell commands once, and they simultaneously run on all of the servers ClusterSSH is connected to.

A perfect fit for our needs, but not quite limited to our higher end environment. Using ClusterSSH, you can control x number of computers via ssh, whether they are exact replicas of one another (cluster) or not. This can be especially useful in a computer lab environment, or with any large number of similar systems that you want to edit from the command line.

EG: You can easily run softwareupdate on OS X or yum/apt on Linux, or even download a file from a central server, on all the computers in your office at once and without the need for more elaborate or expensive graphical solutions.

For the record, there is another project on SourceForge that shares the same purpose and executable name (cssh), but that project has not seen a release since 2004, unlike ClusterSSH, which seems to be kept rather well up-to-date by its developer(s).

But wait, we're on OS X

For
Clusters

ClusterSSH is an X11-based application, which Apple has been providing a port for OS X for years. But if you are on an Intel-based Mac, you might have been wondering where to find the said port for your architecture, since it is not an install option on the DVD provided with your new machine. Well, it is there, but the installer package is in fact in a directory hidden from the Finder.

Clusters For Mac

To install Apple's X11 built for your MacIntel, simply do the following:

  • mount the DVD that came with your Mac
  • open a terminal window
  • type open /Volumes/[OSX_DVD]/System/Installation/Packages/X11User.pkg where [OSX_DVD] is the volume name of the install DVD
  • go through the newly launched installer
  • tada!

Maybe Apple thought their Universal Binary version was only half-baked (…), at least until the latest update they released on November 13, which was itself Intel-ready. Should you choose not to install the latter, you will be prompted to update X11 the next time you run Software Update.

The Perl install that comes bundled with OS X lacks one of the modules that ClusterSSH relies on to provide a small GUI window for you to type your distributed commands in: Tk. Depending on when you read this (2006-11-21), you might want to run a CPAN search for a newer version than the one I link to.

Installing Tk is quite easy if you are familiar with the terminal, and requires for you to have installed the Developer Tools that came bundled with OS X.

Dx Clusters For Mac

  • get the latest Perl/Tk module from CPAN
  • decompress the downloaded archive
  • important: open X11 (Applications » Utilities), and use its terminal for the next steps
  • cd to the Tk folder
  • perl Makefile.PL
  • make
  • make test
  • sudo make install
  • and you should be done

The make test step will procedurally try the miscellaneous X interface components that Perl/Tk provides, hence the need to run it in X11. All went fine in my context, but you can find more information on Steve Lidie‘s site at Lehigh University if needed: Perl/Tk and Mac OS X.

On to compiling ClusterSSH

Now that we have met ClusterSSH prerequisites (X11::Protocol is bundled with OS X), we can move on and compile the software for OSX.

  • get the latest version of the source from the official site.
  • decompress the downloaded archive
  • in a terminal window, cd to the clusterssh-* folder
  • ./configure
  • make
  • sudo make install

On many UNIX systems, you might now be able use the cssh command, but in OS X, you will probably need to make sure that your environment knows to look for software in /usr/local/bin by default, particularly in the terminal. To do so:

  • go back to the terminal
  • type env
  • see if /usr/local/bin is part of the list making up the PATH variable
  • if not type export PATH=$PATH:/usr/local/bin then exit.
Clusters mac

And that is essentially it…

Using ClusterSSH

Now that everything is installed, keep in mind that the purpose of this software is to run the same actions on a number of connected computers, which obviously has tremendous advantages, but also countless dangers.

Read every command before running them, and make sure you did not accidentally or temporarily shifted the window focus to one of the nodes, therefore processing your request one one of the hosts only.

Before you start using cssh, make sure to have a user with the same username and password combination on all of the targeted hosts, unless you opted to pass some of these parameters in the individual connection strings.

Clusters

ClusterSSH is an X11-based application, which Apple has been providing a port for OS X for years. But if you are on an Intel-based Mac, you might have been wondering where to find the said port for your architecture, since it is not an install option on the DVD provided with your new machine. Well, it is there, but the installer package is in fact in a directory hidden from the Finder.

To install Apple's X11 built for your MacIntel, simply do the following:

  • mount the DVD that came with your Mac
  • open a terminal window
  • type open /Volumes/[OSX_DVD]/System/Installation/Packages/X11User.pkg where [OSX_DVD] is the volume name of the install DVD
  • go through the newly launched installer
  • tada!

Maybe Apple thought their Universal Binary version was only half-baked (…), at least until the latest update they released on November 13, which was itself Intel-ready. Should you choose not to install the latter, you will be prompted to update X11 the next time you run Software Update.

The Perl install that comes bundled with OS X lacks one of the modules that ClusterSSH relies on to provide a small GUI window for you to type your distributed commands in: Tk. Depending on when you read this (2006-11-21), you might want to run a CPAN search for a newer version than the one I link to.

Installing Tk is quite easy if you are familiar with the terminal, and requires for you to have installed the Developer Tools that came bundled with OS X.

Dx Clusters For Mac

  • get the latest Perl/Tk module from CPAN
  • decompress the downloaded archive
  • important: open X11 (Applications » Utilities), and use its terminal for the next steps
  • cd to the Tk folder
  • perl Makefile.PL
  • make
  • make test
  • sudo make install
  • and you should be done

The make test step will procedurally try the miscellaneous X interface components that Perl/Tk provides, hence the need to run it in X11. All went fine in my context, but you can find more information on Steve Lidie‘s site at Lehigh University if needed: Perl/Tk and Mac OS X.

On to compiling ClusterSSH

Now that we have met ClusterSSH prerequisites (X11::Protocol is bundled with OS X), we can move on and compile the software for OSX.

  • get the latest version of the source from the official site.
  • decompress the downloaded archive
  • in a terminal window, cd to the clusterssh-* folder
  • ./configure
  • make
  • sudo make install

On many UNIX systems, you might now be able use the cssh command, but in OS X, you will probably need to make sure that your environment knows to look for software in /usr/local/bin by default, particularly in the terminal. To do so:

  • go back to the terminal
  • type env
  • see if /usr/local/bin is part of the list making up the PATH variable
  • if not type export PATH=$PATH:/usr/local/bin then exit.

And that is essentially it…

Using ClusterSSH

Now that everything is installed, keep in mind that the purpose of this software is to run the same actions on a number of connected computers, which obviously has tremendous advantages, but also countless dangers.

Read every command before running them, and make sure you did not accidentally or temporarily shifted the window focus to one of the nodes, therefore processing your request one one of the hosts only.

Before you start using cssh, make sure to have a user with the same username and password combination on all of the targeted hosts, unless you opted to pass some of these parameters in the individual connection strings.

Then, in an X11 terminal window, use the cssh command as illustrated below:

Clusters For Mac

cssh user@your.host.1 user@your.host.2 user@your.host.3 [..]

When all of the windows are launched, be sure to put t
he focus on the small window (see right) and type your commands. In most cases, this will mean starting by typing your password to the computers your are connecting to. And then, you're off to the races.

You can take some steps to make your time with ClusterSSH easier, such as using ssh keys to forgo having to type a password altogether (think about security though). You can also write a small shell script already containing the full list of nodes you are most often connecting to.

I have successfully tested ClusterSSH on Mac OS X 10.4.x (Tiger) on:

  • a 2006 MacBook (Core Duo)
  • a 2005 Mac Mini (G4)
  • and a 2003 PowerBook (G4)

Clusters Mac

For more information you can now use man /usr/local/man/man1/cssh.1 in the terminal to see the full cssh documentation, or see the online FAQ.

But please, do keep reminding yourself that the old 'rm -rf ~/* Game sniper ghost warrior 2. ' cliché is multiplied by the number of hosts you are connected to. ;)

Enjoy.

Update 1: a comment attached to this post led me to a similar tool that has its own advantages, and is well worth mentioning here: dsh.

Update 2: for those of us also running GNU/Linux, ClusterSSH is also available as a pre-compiled Debian package and Fedora RPM.





broken image