Dream has Too Much Money – Dream Server Build

When dream needed a server that could store unlimited gameplay footage with a matching budget he knew there was only one place to go yo and we’re going to be building it in true dream fashion that’s right we’re going to be speed running the creation of a 1.2 petabyte hard drive storage

Server with a 200 terabyte all nvme ssd server all we need is a little bit of help all right i think we’re going for a world record here guys kyogsi has sponsored the ssd so that’s taken care of 45 drives practically built the server for us and you guys are all here

To witness it starting with the big face reveal you ready okay it’s more of a faceplate but look it’s a really cool one it has dream on Let’s go jakey oh jesus oh server to build your voice oh this is a lot of drives to shuck because the servers for dream some of the hardware did end up getting sent to us but the hard drives were not one of those things meaning that yes my friends dream actually did

Shell out for 60 the full 60 20 terabyte hard drives that’s going to be a total of 1.2 petabytes of raw storage which should work out to somewhere in the neighborhood of one petabyte of usable storage when we’re done here i think we’re going to play

This one a little bit safer i’ll put some hot spare so it might be more like 900 okay sure that’s fair enough i don’t want to get a tech support call realistically from like wherever the crap dream is from the server hardware itself was provided by 45 drives and

This is their turbo tier server it’s got a 26 core xeon gold 512 gigs of ddr4 memory which we’ll be using as a zfs read cache we’ve got four lsi hbas which are allowing us to hook up all of these freaking drives and we actually got them to send a dual port

25 gigabit per second network card which um sorry 45 drives we’re actually going to steal from this server and put in the ssd server and there is a reason for this the fancy switch hey i need more drives jake hold on shut up world record the fancy switch that ubiquity sent happens

To have two 25 gig ports so i’m thinking we put them together aggregate 50 gig to the storage server wow it’s probably excessive but we’re gonna do it anyways honestly though outside of our own server room he is getting the finest storage setup out of probably

Anyone else on youtube or twitch by the way jake can i just shout you out for putting these in exactly the correct orientation for me to just am i slot them right in are you being sarcastic are you not trying to no no i was actually no it’s like the most ergonomic

Thing ever i just grabbed them and put them in okay i’ll be sure to do it the other way now oh you piece of crap why did i say anything i was not trying at all it just happened to be how they were getting pulled out that’s it are we done 1.2 petabytes

You know it’s not that fun when it’s this easy what i mean it’s i’m now i’m still into it we haven’t found out if it works yet so we’re replacing this dual 25 gigabit nick with this dual 10 gigabit nick and the reason we’re doing that is that i

Mean they’re hard drives realistically if we manage to get you know what 20 gigabit trunks so two gigabytes a second out of it that’ll be fine i mean it’ll just it’ll do a lot more than that but you’re just using this for archives so yes well yes and no i think it’s possible

That they could end up editing partially off of it if they were working on a project it’ll have plenty for that called back from you know a previous video or something along those lines come on jake what what was the ergonomic rationale behind putting this server here i’m not saying

You’re wrong i’m just saying i don’t see how you’re right are you complaining because you’re short no i’m complaining because tall people don’t consider my short person needs that’s what i’m complaining about right right very important distinction unlike the petabyte of hard drive based storage this one actually

Is going to have a ton of room for expansion and that is thanks to high density drives like the cm6 from kyogre these puppies are available in up to what 15.3 terabytes of top this is a 30 terabyte what shut up it’s not not

Only is it a 30 terabyte ssd it’s a 30 terabyte tlc ssd it’s not even qlc these are crazy kiosks just sent these for dream where’s mine well they sent 15 for dream and i was like yeah we’ll just give them like seven oh really yeah it’s

Not like he paid for them yeah that still means he’s gonna end up with 120 terabytes of usable space with two parity drives and a hot spare wrap your brain around it right this is the state of the art i mean i think they’re looking at what 22 to 26

In the next year and those aren’t even conventional those are shingles exactly this has 50 more capacity than this in a fraction of the size mind you it’s probably quite literally 20 times the cost not to mention this is pcie gen 4 these will do almost 7 gigabytes a

Second read on a single drive and that’s why we’re keeping some yes you should okay so we’re gonna do a four node actually wait the what oh my oh my god oh my god how do you do that you just literally holy

Oh my god oh my god oh my god did i put it back are you kidding me dude what’s the actual is wrong with you oh my god oh oh i want to throw up i really oh do you think it still works no did you hear the

Noise in me well it’s a pretty it’s a pretty bouncy floor okay the video’s done i’m leaving goodbye it might still work okay lunch says it still works [Laughter] you know it might actually still work here’s the thing i’ve actually talked to seagate engineers about the kind of drop

Testing that they do on these things there’s a solid chance that thing still works what happened what nothing you talking no no i was just saying that uh you’re a fine young man and i’m really proud of the way you’ve turned out i’m not proud of the way you’ve turned out

This is pretty gorgeous this is the new version of the one we have right yeah that’s specifically designed for epic milan so that’s a 32 core 75 43p so single single processor it boosts to 3.8 gigahertz i think so not quite as good as the 75 f3 yeah but that’s pretty

Freaking incredible this thing is gonna rip are you putting the ram in are you gonna talk about the ram at all i’m just gonna put it in really we’re speed running okay well it’s gonna be 512 gigs of ram nope what 128 128 that’s it they’re eight gig dimms these are from

The one million dollar pc that we didn’t use right microns sent us literally like flats 96. flats of these eight gig ecc 3200 modules they’re fast yep they’re exactly what you want just not the capacity that you want but for this for you it’s perfect because we’re not using ram caching

Since they’re nvme drives you don’t need that for zfs but he’s gonna get 128 gigs anyway so we just want a nice fast cpu which for epic high speed memory is really important and then we’re basically not going to hit this at all yeah i mean stuff will

Go through it but that’s it that’s pretty much it 46 47 48 49 50 51 52 53 54 55 36 57 58 59 that’s it shut up it does not there’s only 59 yeah because it starts at zero oh hold on there might be 60 actually

Hold on it starts at zero doesn’t it it might start at zero might be okay that it does are you screen recording there’s the moment yeah i’m not screen recording but it does start at zero there is 60. that doesn’t necessarily mean one of them isn’t completely buggered okay

Cooler time yeah wow this is a beefy cooler for a two years yeah the nice thing about a 2u is you get double the cooler capacity which means your fans get to spin a lot slower yeah which is going to be good for at your house

Yeah one user like we’re using a similar machine to this at my place right yeah okay just a less cool one oh really so dream’s going to have a better server than me yeah i’m feeling pretty inadequate right now i love these tool-less sleds man when i built the

First nvme wanik it took frickin forever and i didn’t even put all the screws in oh these are so much easier to deal i can’t believe that every one of these is 30 freaking terabytes i just installed 60 terabytes of ssd storage hold on here comes number 90

90. like freaking what i mean we do have more we can put more they’re dense yeah they’re i was going to grab one of the cd6s which are like the not enterprise ones um they are substantially heavier these things are dense as here david catch

I don’t know i know hold on a second no i want to i want to open it what no yeah no yeah chill jesus just relax what’s the worst that can happen i lose the screws and we can’t put it back together it’s got a multi pcb design oh my god

Of course it does oh my god these caps though that’s gonna be the difference these are their actual enterprise capacity they’re willing to say it’s enterprise this is for power loss protection these capacitors ensure that if any data is sitting in a cache or in a dram cache

For example it can be flushed to nand before the drive actually powers down even in the event of a sudden power loss to the server that’s freaking cool and those are huge look at the little holes wait they have cooling for the caps that’s amazing wow i’m so happy right now that’s so

Much nan how much nand is that man hold on we gotta find more nand we’re going on a we’re going on a voyage oh there’s a clip there’s a clip here no hold on hold on oh my god this is so cool oh i never get a chance to play with stuff

Like this god damn it there’s another layer hold on so they got thermal pads between oh my god jake i think it’s three layers jake no stop please stop hold on please please relay i can’t do this i think it’s three legs on how do we get this layer my

Heart there it goes holy balls it’s got four dram cash dies what about that controller that thing is huge hold on here it goes here it goes here it goes oh ain’t wait there’s more gas well that’s the thing is the higher capacity you go for your nan flash the

More dram cash you need and if you’re gonna have 30 freaking terabytes demand you’re going to need a lot of dram eight gigabit one g by eight so it’s one gig so it’d be eight gigs i guess eight gigs of flippin ram this ssd has as much as most people’s

Gaming machines there’s a new update i brought out the unify switch we were kind of originally thinking we might build a whole rack and send it to him but then apparently his like installers are doing a bunch of crap so we’re just gonna give him the switch yeah this

Switch has two 25 gig ports that we’re going to aggregate together to be 50 gig for nvme server totally probably overkill for what he’s doing and then we’re going to do two 10 gigs aggregated together for 20 gig to the hard drives then all these other ports are 10 gig to

The rest of the house so that means if for whatever reason they expanded and had like a whole team of editors there they could have five people concurrently hitting this boy for 50 gigabits a second and this boy for 20 gigabits a second that is a lot of throughput for

Game play and if you really wanted you could switch this to a 4×10 card and then you could have 40 gig each editor will get its own 10 gigabits it’s pretty sick it’s a nice little switch ah you were gonna show me something cool there’s a new view yeah hey look so they

Added this new port view so now instead of having to edit a port one by one you can go hey i wanna edit that one that one that one that one and maybe this one over here oh that’s cool and you can set bam i only want this now security-minded

Folks might put mac address like limits so only that one mac address can connect to that port it’s a nice security thing you can also see what device is connected to a specific port a lot easier so i can just click on it so i

Plug the bmc into that one port and then i just click on that there’s the ip here’s a cool feature for ubiquity it would be really nice to be able to take this go yes that’s the device that i want and just set a rule that only that mac address

Can connect to that port just with one click here yeah if you just go like a limit yeah lock mac address lock that would be cool i bet they’ll do it they’re probably gonna watch this and they’re probably going to do it yeah they might do it hey guess what it works

Now i got a vga cable because i thought the ipmi was broken turns out they screwed up i love you gigabyte but they sent a server that doesn’t support milan chips no it’s a roam server it’s a roam server do we have any roam chips to we do so we

Had a 75 or 70 402 p which is a 24 core still going to be plenty for this yeah for the cut down drive configuration that we put in here anyway yeah okay it’s gonna be great okay so now it’ll actually be great yeah we’re in truenast now on both of them

We’re gonna be using trinas core as much as i like scale core is a little more production ready yeah and we want this to just work and him not to have to screw with it and this is actually shockingly user friendly to use these days back in the early frenzy oh boy you

Had to kind of be nerdy enough that you could have basically done it yourself in linux anyway now man this gui is so pretty okay linem ti which one do you want to start with i want to start with the one that definitely has a working drive in this slot right here

I will admit there are 60 functioning drives in the server presently [Laughter] i will not admit that i don’t think or that didn’t make sense i will not admit that i don’t think i think that drive will die faster than the other ones that’s nice but that wasn’t the bet

I’ll buy you lunch whatever yes why would you do that it’s fine it’s fine don’t touch it yet it’s still spinning i’m letting it still spinning it’s letting it spin wiggle wobbling barely i can’t believe you bought a petabyte i can’t believe you threw that hard drive

Can we just cut back to that for a second holy crap i can’t believe you did that okay so we want some pretty chunk-tastic raid zenis i’m figuring maybe not 15-drive v-dabs like we do i was thinking to do 10-drive z2s or i mean for like performance apparently the

Optimal is six drives but that’s a lot of capacity loss yeah we’d have to do z1s at that point and i really don’t want to do that so raid z2 is kind of like raid 6 where you can lose two of your drives before you actually lose any data but it means that

Two drives out of whatever the v dev size you have are lost in terms of capacity so raid z1 sets aside only one of those drives for parity data but that means you can only sustain one loss before your next loss is going to cost you data so for dream

You’re thinking 10 well hold on i just realized we want to have some hot spares we don’t want ten why don’t we do this why don’t we do four hot spares one for each bank and then and then we can do seven drive v dams upgrade z2 we’re gonna lose a lot of

Capacity why don’t we go 14 drive v devs then raid z3 so that would allow three drives lost per one of these rows sure with a spare okay so if we do raid z3s 14 drives we get 800 terabytes usable i’m about to blow your mind how

Much is each of these drives worth uh like 600 six six hundred that’s like six hundred bucks six hundred dollars okay let’s do some quick math real quick here okay six hundred twelve hundred twenty four hundred okay five thousand ten thousand ten thousand dollars worth of drives is sitting doing

Nothing there will be no data actually stored on them okay they’re just holding data just in case these ones fail or standing there ready to go in case these fail 10 grand that level of redundancy costs in a server like this look at all this

Big data big data you’re going to set it to be an smb share we’re gonna go into advance i’m gonna do z standard compression because it’s really great record size of one mag because it’s all footage bam now we can store stuff big data so it works out to be 740 tera tibi

Bytes usable after formatting that’s a big data all right let’s do the nvme we’re going to do it pretty similar to the petabyte server where we have a hot spare but we’re just doing one and it’s a raid z2 which means we have two disks

Of parity that is so overkill it is with a hot spirit do you want to get a phone call hold on the problem with ssds hear me out the problem with ssds they’re not like hard drives where they kind of staggered fail usually when you have

Like 20 ssds and they’re all from the same batch they’ll just start failing at the same time because they’re so much more dependent on like right endurance it’s better to be safer he’s got 100 tv bites that’s so much that’s that’s more than we have i know but this is like 90 terabytes

Of ssd storage just not being used well it’s being used jake i think it’s fine raid z1 and a hot spare i know their enterprise drives he’s barely going to touch these compared to the workloads they’re designed for you’re you know i’m looking around i see wendell like through the camera

Hey there mr wilson it’s me linus the menace well jake and i are having a bit of a debate right now we’ve got seven of these 30 terabyte keyoxia drives and i’m not going to tell you who is in which position so that you’re unbiased but there are two potential configurations

For them in in true nas here um are you willing to weigh in on on which one you think is proper one of us wants to do a raid z2 with a hot spare and one of us wants to do a raid z1 with a hot spare i would probably do everything honestly

A little bit more it’s a little bit more dangerous um but those drives are so fast and if you’re on top of it it’s probably okay but what if they’re not on top of it if you’re not on top of it then you gotta go do you really think it’s likely that

We’re gonna hit these drives hard enough that we should be concerned about their lifespan if you get a hot spare on the flash side i would probably just make it raise d2 instead of having a hot spare only because flash wears differently than mechanical hard drives now okay so

What about a raid z2 and a hot spare oh my god jake okay all right let’s just go raid z2 then with seven drives we need to put some footage on this thing nice it’s going through my workstation so it’s not going to be that fast wow even

Through your workstation it’s doing 400 plus megabytes a second you know what i should go jump onto mine and hit it at the same time and see if that even goes down i bet it won’t the bottleneck is going to be your your station not this machine okay well i’m still copying at

400 to 450 megabytes a second how about you okay you ready for me to press go oh you haven’t pressed go yet okay i already did yeah that’s what i figured yeah the one thing to consider is because we’re copying like a folder full of a bunch of little files windows will

Never really get there because it’s like starting stopping starting zapping starting stopping but that’s still super awesome this is really important just because we have redundancy okay so we have that extra couple of drives that can fail doesn’t mean that we have backup so if this is where they’re going to

Store their active projects then if something were to go wrong let’s say uh a power supply blue i mean theoretically those are also redundant so you could just you could just yank one in the middle of operation but let’s say it blew in such a catastrophic way that it lit on fire

Okay the server is dead but somehow the fire doesn’t spread outside the steel chassis and light the other server on fire we need a backup that backup is gonna go automatically from this server to this one unfortunately eating up some of our capacity but not that much it’s gonna be

Incremental so but each time it backs up it’s it’s keeping what’s already on there and just adding the new stuff yes and it will cycle automatically so we can set it so it keeps like a daily backup uh for two weeks and it automatically deletes the old ones when

It makes the new ones and doesn’t store a bunch on here and then as soon as they delete something from here that will eventually fade out of those daily or weekly backups until the only copy that exists is the archival copy which like us dream has said is not

Super essential it’s a nice to have not a need to have because you can always go re-download it off of youtube if it comes to that that’s your off-site backup essentially thanks youtube okay i started the backup it’s backing up from this server over to this server and it’s running at

400 megabytes a second i mean that’s a lot of data you could probably get it to go faster by disabling like secure ciphers on ssh but right now we just have it in the default configuration just to show that it works and since by default this is

Only going to be a daily backup it’s got 24 hours to move whatever is on here that’s plenty enough if you guys enjoyed this video check out kyoccia’s products they’ve got everything from enterprise to consumer grade ssd products we’re gonna have them linked down below also major shout out to gigabyte who sent

Over the wrong server but they did send a server so kudos for that micron for sending over the ram amd for providing the cpu that we couldn’t use thanks gigabyte um and also 45 drives for sending over this epic storinator that happens to color match the green tabs on

Oh wow and with the dream front plate they can do like literally anything custom you want yeah it looks amazing if you guys enjoyed this video go check out uh i don’t know one of the other times jake and i worked on a server thing he’ll find something petabyte of flash yeah

Oh yeah we need to get back going we gotta finish that but jake is too busy helping me with my house he moved in early he’s an in-demand kind of boy