My Hadoop Cluster Configuration - So Far...




 Live Chat
 
Steven
Support Agent

Steven16:03
Hi, I'm Steven, how can I help you?

You16:04
...suppose I buy 4 supermicro 1Us.

Steven16:04
that would be great!
have you worked out a spec?

You16:04
...I know that I will need to buy additional equipment, such as the rack to mount them on...but what else would I need to complete the system?

Steven16:05
which server are you considering?

You16:05
Supermicro Short Depth 1U Xeon E-2200 / E-2100

Steven16:06
OK...uhm...if you choose a CPU, RAM and storage, this will be ready for you to install whatever OS you decide. Our servers come fully 
assembled with the power cord. Each machine has 2 LAN ports and a dedicated IPMI (remote management) port for a total of 3 ports. 
What will you be doing with these machines? ...are you sure that you don't want just a tower PC, with maybe 
Windows Home on it? To store recipees and such? Something heeaassy and siiimple to just play around with...so you can learn about computers.

You16:09
LOL....OMG...excuse me, sorry I forgot to tell you who I am. No, I am NOT an LCDPIINC. I am a data engineer 
working for a power company that is seriously considering the purchase and construction of a 4-node hadoop cluster, built into a 
roll-around cabinet. Each node would have a 16 TB hard drive hooked up to it. What other parts do I need so they work as a cluster? what 
kind of rails and rack equipment do I need? 

Steven16:18
Ahhh, OK. You're one of us LOL....Think of this as similar to starting with a case, then adding compatible components, but on a bigger 
scale. First, I think you will need a cabinet where the slots are compatible with the server blades. Since you live in a hot area of the 
country, it would be wise if the cabinet had additional cooling fans. Something like this: 
https://www.startech.com/en-us/server-management/2636cabinet.
Then, you will need some type of switch to connect the servers into a network, that is of rackmount type. It will need to be fast so as to
not slow down HDFS operations.

You16:18
NOW we are getting somewhere!

Steven16:18
:) ...Supermicro rackmount servers are designed to mount directly into the cabinet slots, so no rail kits are necessary. They mount from the 
front ears.

You16:40
ty ty ty :) :) :) thanx for all your help, I've been looking for someone like you for years, but all I could find using Google is
"People Ask: What Is Hadoop? Can I connect a cluster to a LAN? How many hard drives does an HDFS have?"

Steven16:48
LOL ...FYI...That cabinet is big and heavy. 33" x 29" x 6" and it weighs 62 pounds...does that matter to you?

You16:49
Oh no, it should be rugged to hold all the nodes and they will be hefty as well...weight doesn't concern me

Steven16:55
we could put in a small SSD instead of hte NVME if you want to save a little $

You16:56
yes, the SSD only needs to hold the OS, hdfs and mapreduce java software, and other ancillary software to connect to the other nodes.

Steven16:56
we could put in a Samsung 860 EVO. is 250GB enough?

You16:57
Oh yes, I plan to put redhat on each node, it will fit in a fraction of that size. It only needs about 10GB or so

Steven16:58
let me get some pricing together...

You16:59
That would be great, that is what I will take to the bank when I apply for the loan

Steven17:06
if you give me your email, name and address I can send you an official estimate. We could do 1 x 512F-350 chassis, E-2224 CPU, 16GB of RAM 
and a 250G Samsung EVO shipped to Arizona for $779 each.

You17:18
...this is great, I've come farther in an hour and a half than I have in 3 years on this... ty ty :) :)

Steven17:19
good to hear, we are not always the least expensive, but we pride ourselves on customer service.

You17:20
just let me know I'll give you a very high recommendation :)

Steven17:20
tell a friend!, that is the best way to grow our business

You17:22
consider it done ! :) :) OK well, I'll gather up all the information, and talk to the bank...maybe I'll get an early christmas 
present! I have well over 1500 movies I want to load into my new data lake, and the complete series of well over 2 dozen tv shows. 
Plus all my music, sentimental files...everything will fit into it easily!

Steven17:23
Wow, that is a lot of movies....you're taking the right direction because the point of using a DFS is that if a hard drive is in danger 
of failing, hadoop will quickly move the data off it to some other part of the system, then it will tell you to replace the drive

You17:23
:) :) :) :) You have a great evening!

Steven17:23
You too, and good luck with the bank