Re: [Hampshire] High availability database

2009/9/16 Chris Simmonds <chris@???>:
> Hugo Mills wrote:
>> On Wed, Sep 16, 2009 at 02:13:45PM +0100, Chris Simmonds wrote:
>>> Hi and thanks to everyone who replied. I'm busy researching some
>>> possibilities at the moment. However, just to clarify, the issue is high
>>> availability among the 50 or so nodes so that any node can go down and
>>> come back up again without impacting any of the others. There are no
>>> servers as such, each node is potentially both a server and a client so
>>> that if using a traditional database there needs to be some mechanism to
>>> elect a master server if the current master goes down. The trickiest
>>> case is if the network gets disrupted such that you get two separate
>>> segments for a while - each with their own master - which then get
>>> joined together again. On the other hand the amount of data is quite
>>> small - maybe a thousand rows in SQL terms. Actually it doesn't have to
>>> be SQL at all, I'm just using that as an example.
>>
>>    You also don't specify the robustness requirements: of your 50
>> servers, how many would you expect to be able to lose before data gets
>> lost? You talk of one at a time going down and coming back up, but is
>> it really a matter of a few vanishing at any one time, or could they
>> all go at the same time?
>>
>>    Can you give more details of why these machines are ephemeral? Are
>> they on unreliable network connections? People's desktops at work?
>> General purpose publically-accessible machines?
>>
>>    What problem are you trying to solve with this system?
>>
>>    Hugo.
>>
>
> Hi Hugo,
>
> The application is theoretical at the moment, but the requirement comes
> from a factory floor information system with multiple data entry points.
> The HA part is optional, but as it is the most interesting part that is
> the thing I have been concentrating on. However, I can see other
> applications where there is a group of peers sharing a common data
> store, for example a group of autonomous robots: if one trips up or gets
> lost the others can carry on without it. Loads of fun!
>
>

I think it is important to understand the application.
I think a possible solution to this problem could be Java Spaces.
Storing the same data on 50 PCs and keeping it in sync is a nightmere,
particularly if any of the 50 PCs could perform a write to that data.
Java Spaces can help considerably. It works on the pricipal that each
item of data has a globally unique key. The key is run through a
hashing function that results in a destination to store the data item.
So, given the unique key, any client can retrieve the data item. One
can configure it so that the destination is actually a "data
partition" so that the data is actually then stored on multiple PCs
for backup purposes.
The data store has no structure. It is the responsibility of the
client application to support any structure to the data.
So, as long as you can get each of the 50 PCs to create a globally
unique key for each item of data needing storing, you should be ok.

Underneath this, one should build a reliable and redundent network
architecture, so that one does not loose comms unless one of the PCs
is turned off.
The system can deal with that scenario.


Kind Regards

James