After much discussion of what goes on below the water line (in the provider/enabler space), hopefully this prototype release will refocus energy and get the creative juices flowing above it (with the users themselves). The idea is to follow Unix’s example (as explained below) by seamlessly blending cloud resources into existing working environments. See examples below for some ideas of what is possible – my favourite is taking a local file and pipelining it through two web based services (the first is a rudimentary spam filter and the second appends legal notices), but being able to interact with HTTP in general from the command line is quite cool. Support for true bidirectional pipes (eg via CONNECT rather than simplex POSTs) and remote instantiation of pipelines is… well… in the pipeline. As is dealing with content types (verbs and nouns having already been catered for), mapping arguments to the query string and a flexible authentication infrastructure (all of which will require some thought and creation of static interfaces).
Cloud User Shell (cush) is a multi-call executable, bringing cushy, RESTful cloud control to the command line.
Philosophy
It follows the wildly successful Unix philosophy:
This is the Unix cloud computing philosophy:
- Write programs components that do one thing and do it well.
- Write programs components to work together.
- Write programs components to handle text HTTP streams, because that is a universal interface.
RESTful
It is also RESTful, clearly delineating the 3 sides of the REST Triangle:
- Verbs (constrained) e.g. GET
- Nouns (unconstrained) e.g. URIs (http://samj.net/])
- Content Types (constrained, optional */*) e.g. HTML (text/html)
Installation
To install the python prototype cush.py
- Unix
- Change to a suitable directory (eg /usr/local/bin) and be sure you can write (eg sudo -s)
- If you have Subversion do svn export http://cush.googlecode.com/svn/trunk/src/python/cush.py so the keyword substitutions work
- Otherwise download http://cush.googlecode.com/svn/trunk/src/python/cush.py:
- curl -O http://cush.googlecode.com/svn/trunk/src/python/cush.py or
- wget http://cush.googlecode.com/svn/trunk/src/python/cush.py
- Make the file executable with chmod 755 cush.py
- cush.py install to create symlinks for the commands
- Windows
- Download http://cush.googlecode.com/svn/trunk/src/python/cush.py (right click, Save As)
- All
- cush.py upgrade periodically to stay up-to-date
Usage
Usage: cush [verb] [noun] or: [verb] [noun] eg: get http://samj.net
Cloud User SHell (cush) is a multi-call binary (like BusyBox) that combines
many useful cloud computing utilities into a single executable. Most people
will create a link to cush for each function they wish to use and cush will
act like whatever it was invoked as, but it can also be called directly and
passed the command as the first parameter.
Currently defined functions:
@, @@, delete, get, head, ls, post, propfind, put, rm
Examples
All of these work alongside your favourite shell, eg bash
Send files to/from the cloud and process them as if they were local:
$ echo "The quick brown fox jumps over the lazy dog." | put http://local/uploads/brown.txt $ get http://localhost/uploads/brown.txt The quick brown fox jumps over the lazy dog. $ get http://local/uploads/brown.txt | sed -e 's/brown/red/' | put http://local/uploads/red.txt
Even when they are:
$ get http://local/uploads/red.txt > /tmp/red.txt $ get file:///tmp/red.txt The quick red fox jumps over the lazy dog.
Dereference URIs with shorthand syntax (equivalent to GET command):
$ @ http://localhost/uploads/brown.txt The quick brown fox jumps over the lazy dog.
Manipulate data in the cloud by listing (ls) and removing (rm) URIs like local files:
$ ls http://localhost/uploads/colours/ brown.txt red.txt $ rm http://localhost/uploads/colours/red.txt $ ls http://localhost/uploads/colours/ brown.txt
Set up HttpPipelines (like Unix pipelines) for data processing:
The following shows an rfc822 email message being pipelined through two URIs for spam filtering (which adds a X-Spam-Flag: YES header if it finds ‘Viagra’) and a legal notice (which appends ‘This message is confidential.’).
$ cat rfc822-spam.txt |@ http://localhost/cgi-bin/spam-vapouriser.cgi |@ http://localhost/cgi-bin/legal-notifier.cgi X-Spam-Flag: YES Received: by 10.150.96.14 with HTTP; Tue, 5 Aug 2008 04:52:14 -0700 (PDT) Message-ID: <[email protected]> Date: Tue, 5 Aug 2008 13:52:14 +0200 From: "Sam Johnston" <[email protected]> To: "Sam Johnston" <[email protected]> Subject: Cloud Computing User Shell (cush) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Delivered-To: [email protected] Check out Viagra - it rocks. This message is confidential.
Obtain and manipulate structured data (eg titles of the items in a feed, weather in Sydney):
$ get http://feeds.feedburner.com/samj | xpath //rss/channel/item/title Found 24 nodes: -- NODE -- <title>Cloud standards: not so fast...</title> -- NODE -- <title>Cloud computing and Open Source software</title> -- NODE -- ... $ get http://weather.yahooapis.com/forecastrss?p=ASXX0112 | xpath /rss/channel/item/yweather:condition Found 1 nodes: -- NODE -- ...
Obtain information about URIs and technical debugging:
$ head http://samj.net/ content-length: 0 server: GFE/1.3 last-modified: Mon, 25 Aug 2008 22:04:36 GMT etag: "55ed4707-7cbc-4208-9cbc-cbc6105bc682" cache-control: max-age=0 private date: Thu, 28 Aug 2008 00:22:45 GMT content-type: text/html; charset=UTF-8