EwanSilver.com » Distributed Computing http://ewansilver.com/blog Ewan Silver Sat, 16 Aug 2008 22:09:49 +0000 http://wordpress.org/?v=2.9.1 en hourly 1 So you want to be enlightened? http://ewansilver.com/blog/2008/08/16/so-you-want-to-be-enlightened/ http://ewansilver.com/blog/2008/08/16/so-you-want-to-be-enlightened/#comments Sat, 16 Aug 2008 22:07:32 +0000 Ewan http://ewansilver.com/blog/2008/08/16/so-you-want-to-be-enlightened/ Availability Enlightenment

  • “And nobody expressed any concern that this might not be right?“
  • “Bill, you are building a system, not a collection of parts.“
]]> http://ewansilver.com/blog/2008/08/16/so-you-want-to-be-enlightened/feed/ 0 Reliability http://ewansilver.com/blog/2008/05/02/reliability/ http://ewansilver.com/blog/2008/05/02/reliability/#comments Fri, 02 May 2008 10:05:03 +0000 Ewan http://ewansilver.com/blog/2008/05/02/reliability/ Steve Vinoski talking about reliability (in particular my favourite: Erlang)

To achieve reliability, you have to accept the fact that failure will occur, Once you accept that, then other things fall into place: you need to be able to restart things quickly, and to do that, processes need to be cheap. If something fails, you don’t want it taking everything else with it, so you need to at least minimize, if not eliminate, sharing, which leads you to message passing. You also need monitoring capabilities that can detect failed processes and restart them

Amen to that!

Technorati Tags: ,

]]> http://ewansilver.com/blog/2008/05/02/reliability/feed/ 0 23 January 2008 http://ewansilver.com/blog/2008/01/23/23-january-2008/ http://ewansilver.com/blog/2008/01/23/23-january-2008/#comments Wed, 23 Jan 2008 07:55:33 +0000 Ewan http://ewansilver.com/blog/2008/01/23/23-january-2008/
  • Towards Robust Distributed Systems – Eric Brewer’s slidedeck that kicked off the whole CAP Theorem debate at PODC in 2000.

    • Technorati Tags: , ,

    ]]>     http://ewansilver.com/blog/2008/01/23/23-january-2008/feed/ 0     18 Jan 2008 http://ewansilver.com/blog/2008/01/18/18-jan-2008/ http://ewansilver.com/blog/2008/01/18/18-jan-2008/#comments Fri, 18 Jan 2008 14:49:00 +0000 Ewan http://ewansilver.com/blog/2008/01/18/18-jan-2008/
  • London Erlang Meeting – “How do you test large systems written in Erlang?” on Monday (I presume) Jan 21.
  • Map Reduce a major step back. Michael Stonebraker and David J. DeWitt let forth on why they dislike Google’s MapReduce infrastructure and how it is poor relation to standard relational database technology.
  • Relational Database Experts Jump The MapReduce Shark – a reply to Stonebraker and DeWitt that I pretty broadly agree with.
  • MapReducing 20 petabytes per day – Greg Linden commenting on the recent paper “MapReduce: Simplified Data Processing on Large Clusters” by Jeff Dean and Sanjay Ghemawat.

    • Technorati Tags: , , , , , , ,

    ]]>     http://ewansilver.com/blog/2008/01/18/18-jan-2008/feed/ 0     Eventual Consistency http://ewansilver.com/blog/2007/12/20/eventual-consistency/ http://ewansilver.com/blog/2007/12/20/eventual-consistency/#comments Thu, 20 Dec 2007 09:58:59 +0000 Ewan http://ewansilver.com/blog/2007/12/20/eventual-consistency/ Werner has a new piece on eventual consistency. Go read!!!!

    Technorati Tags: eventual consistency

    ]]>     http://ewansilver.com/blog/2007/12/20/eventual-consistency/feed/ 0     Amazon DB and Erlang http://ewansilver.com/blog/2007/12/15/amazon-db-and-erlang/ http://ewansilver.com/blog/2007/12/15/amazon-db-and-erlang/#comments Sat, 15 Dec 2007 22:48:09 +0000 Ewan http://ewansilver.com/blog/2007/12/15/amazon-db-and-erlang/ The world and it’s dog is jumping on the Amazon SimpleDB announcement. Interesting to note that apparently (big caveat there!) it is built on Erlang.

    I am keen to see how the DB heads respond to this. I also want to see how how long it takes for people to get their heads around the eventual consistency model it employs. The use of an eventual consistency model is often a critical factor in being able to scale any ultra large system but the concept scares many people to death (and thats only the people who realise what it means!)

    Technorati Tags: , SimpleDB, Erlang,

    ]]>     http://ewansilver.com/blog/2007/12/15/amazon-db-and-erlang/feed/ 0     July 23 2007 http://ewansilver.com/blog/2007/07/23/july-23-2007/ http://ewansilver.com/blog/2007/07/23/july-23-2007/#comments Mon, 23 Jul 2007 14:16:43 +0000 Ewan http://ewansilver.com/blog/2007/07/23/july-23-2007/
  • Interesting job spec from Vogels…
    My favourite line: “You have at least once tried to understand Paxos by reading the original paper.“. It all sounds like ancient greek to me!

    • Technorati Tags: ,

    ]]>     http://ewansilver.com/blog/2007/07/23/july-23-2007/feed/ 2     Thankgod for chaos! http://ewansilver.com/blog/2007/05/25/thankgod-for-chaos/ http://ewansilver.com/blog/2007/05/25/thankgod-for-chaos/#comments Fri, 25 May 2007 05:00:50 +0000 Ewan http://ewansilver.com/blog/2007/05/25/thankgod-for-chaos/ It’s always nice to get an Ebay perspective on things – especially when you totally agree with what is being said. For a long time I have felt that if we really want to embrace the power of large scale distributed systems we have to accept that we just do not know what is going on – life is non-determinstic, messy, chaotic and generally random. Most tech orientated people are used to thinking in a single linear fashion. First the processor does this, then it does that. Nice – but not the way the world works. Things actually happen at the same time. Just not neccasarily on your processor or in your address space. For many people dealing with that is a problem and something that should be somehow hidden away or, at the very least, forced into a 2PC harness. That might work in a small system but not when you reach internet scale systems. Forget all about a linear world and move to thinking about multiple agents doing multiple things at the same time.

    This is a thought process that is going to increasingly dominant the way we build systems.

    ]]>     http://ewansilver.com/blog/2007/05/25/thankgod-for-chaos/feed/ 1     Collaborative Filtering – Google style http://ewansilver.com/blog/2007/05/25/collaborative-filtering-google-style/ http://ewansilver.com/blog/2007/05/25/collaborative-filtering-google-style/#comments Fri, 25 May 2007 03:50:23 +0000 Ewan http://ewansilver.com/blog/2007/05/25/collaborative-filtering-google-style/ I have always had a soft spot for collaborative filtering so it is interesting to read Google’s take on things.

    Via Greg

    Technorati Tags:

    ]]>     http://ewansilver.com/blog/2007/05/25/collaborative-filtering-google-style/feed/ 0     The river is flowing…. http://ewansilver.com/blog/2007/01/03/the-river-is-flowing/ http://ewansilver.com/blog/2007/01/03/the-river-is-flowing/#comments Wed, 03 Jan 2007 16:31:09 +0000 Ewan http://ewansilver.com/blog/2007/01/03/the-river-is-flowing/ The Jini project was accepted into the Apache Incubator and is now available as a Podling called River.

    Great stuff!

    Technorati Tags: , , ,

    ]]>     http://ewansilver.com/blog/2007/01/03/the-river-is-flowing/feed/ 0