I've been thinking about load balancing lately and how to dynamically grow a cluster and its data set. I'd like to define "balance factor" as follows:

If the number of nodes in the cluster grows by a factor of x, at some future point when the data set has also grown by a factor of x the balance factor is the ratio of the smallest node to the largest node.

In my use case it is impractical for any node to know the size of all the nodes. Load balancing decisions must be made probabilistically on limited data. Today I happened across this blog post, which is very applicable to my scenario. It presents a very good solution for load balancing between a fixed number of bins, but when adding nodes without taking the system offline it is useful to look at more than 2 random points to maximize balance factor.

I did some experiments on different values for x and n (where n is the number of random points examined), and experimentally determined an approximate equation for balance factor b:

b = 1-(1/x)^(n-1)

(by approximate I mean that it is close enough for the range of values I care about, which is x between 1.1 and 10 and n between 2 and 25. It might be exactly right but I don't have time to do a proof before band practice.)

If you actually want to use this though, you need to determine n given x and the desired b. With some manipulation we get:

n = 1 + log(1-b)/log(1/x)

Let me know if you found this useful.

Amazing information,thank you for your ideas.after along time i have studied

ReplyDeletean interesting information's.we need more updates in your blog.

Hadoop Training in Chennai

Big data training in chennai

Big Data Training in Anna Nagar

JAVA Training in Chennai

Python Training in Chennai

Android Training in Chennai

hadoop training in Annanagar

big data training in chennai anna nagar

Big data training in annanagar

Great Article Artificial Intelligence Projects

DeleteProject Center in Chennai

JavaScript Training in Chennai

JavaScript Training in Chennai

Hello Admin!

ReplyDeleteThanks for the post. It was very interesting and meaningful. I really appreciate it! Keep updating stuffs like this. If you are looking for the Advertising Agency in Chennai | Printing in Chennai , Visit Inoventic Creative Agency Today..

More impressive blog!!! Thanks for shared with us.... waiting for you upcoming data.keep up!!

ReplyDeleteandroid training in chennai

android online training in chennai

android training in bangalore

android training in hyderabad

android Training in coimbatore

android training

android online training

Hey!

ReplyDeleteDigiPeek is the best SEO & Link Building Service Provider In The World. I have 7+ Years Experience To Build SEO, Backlinks & Improve Website Ranking.

If you need Profile Backlinks, Forum Backlinks, Dofollow Backlinks, Manual Backlinks, Trusted SEO Backlinks, Increase Domain Rating Then You Will Contact Me.

I am glad to help You!

Let's TRY!

It was amazing and so helpful. Thank you.

ReplyDeleteclassified sites in Dhaka