SSD speeds are nothing short of miraculous in my mind. I come from the old days of striping 16 HDDs together (at a minimum number) to get 1GB/s throughput. Depending on the chassis, that was 2 8-drive enclosures in the "desktop" version or the large 4RU enclosures with redundant PSUs and fans loud enough to overpower arena rock concerts. Now, we can get 5+GB/s throughput from a tiny stick that can be used externally via a single cable for data&power that is absolutely silent. I edit 4K+ video as well, and now can edit directly from the same device the camera recorded to during production. I'm skipping over the parts of still making backups, but there's no more multi-hour copy from source media to edit media during a DIT step. I've spent many a shoot as a DIT wishing the 1s&0s would travel across devices much faster while everyone else on the production has already left, so this is much appreciated by me. Oh, and those 16 device units only came close to 4TB around the time of me finally dropping spinning rust.
The first enclosure I ever dealt with was a 7-bay RAID-0 that could just barely handle AVR75 encoding from Avid. Just barely to the point that only video was saved to the array. The audio throughput would put it over the top, so audio was saved to a separate external drive.
Using SSD feels like a well deserved power up from those days.
The latency of modern NVMe is what really blows my mind (as low as 20~30 uS). NVMe is about an order of magnitude quicker than SAS and SATA.
This is why I always recommend developers try using SQLite on top of NVMe storage. The performance is incredible. I don't think you would see query times anywhere near 20uS with a hosted SQL solution, even if it's on the same machine using named pipes or other IPC mechanism.
Meanwhile a job recently told me they are on IBM AS400 boxes “because Postgress and other sql databases can’t keep up with the number of transactions we have”… for a company that has a few thousand inserts per day…
Obviously not true that they’d overwhelm modern databases but feels like that place has had the same opinions since the 1960s.
Then there's optane that got ~10us with. The newest controllers and nand is inching closer with randoms but optane is still the most miraculous ssd tech that's normally obtainable
But they've retired it??
Eventually we'll have machines with unified memory+storage. You'll certainly have to take a bit of a performance hit in certain scenarios but also think about the load time improvements. If you store video game files in the same format they'd be needed at runtime you could be at the main menu in under a second.
At a minimum, we should be able to get everything to DRAM speeds. Beyond that you start to run into certain limitations. Achieving L1 latency is physically impossible if the storage element is more than a few inches away from the CPU.
Most new motherboards do already have the highest throughput M.2 connector very near the CPU.
The most recent desktop I built has it situated directly below the standard formfactor x16 PCI slot.
I think part of the reason why it's so close is also for signal integrity reasons.
Far more so than latency.
Also: routing PCIe lanes is a pain. Being able to take 4 pairs and terminate them makes routing everything else simple
The separation into RAM and external storage (floppy disks, magnetic tapes, hard drives and later SSD etc) is the sole consequence of technology not being advanced enough at the time to store all of the data in memory.
Virtual memory subsystems in operating systems of the last 40+ years pretty much do exactly that – they essentially emulate infinite RAM that spills over onto the external storage that backs it up.
Prosumer grade laptops are already easily available, and in 2-3 years there will be ones with 256-512 Gb as well, so… it is not entirely incoceivable that in 10-20 years (maybe more, maybe less) the Optane style memory is going to make a comeback and laptops/desktops will come with just memory, and the separation into RAM and external storage will finally cease to exist.
P.S. RAM has become so cheap and has reached such large capacity that the current generation of young engineers don't event know what a swap is, and why they might want to configure it.
I have a feeling (and it's just a feeling) that many SoC-style chips of the future will abandon Von Neumann Architecture entirely.
It's not that much of a stretch to imagine ultra dense wafers that can have compute, storage, and memory all in one SoC.
First, unify compute and memory. Then, later, unify those two with persistent storage so that we have something like RAM = VRAM = Storage.
I don't think this is around the corner, but certainly possible in about 12 years.
I am also of the opinion that we are heading towards the convergence, although it is not very clear yet what the designs are going to converge on.
Pretty much every modern CPU is a hybrid design (either modified Harvard or von Neumann), and then there is SoC, as you have rightfully pointed out, which is usually modified Harvard, with heterogenuous computing, integrated SIMT (GPU), DSP's and various accelerators (e.g. NPU) all connected via high-speed interconnects. Apple has added unified memory, and there have rumours that with the advent M5 they are going to change how the memory chips are packaged (added to the SoC), which might (or might not) lay a path for the unification of RAM and storage in the future. It is going to be an interesting time.
Optane is no longer available :(
It's not really the SSDs themselves that are incredibly fast (they still are somewhat), it's mostly the RAM cache and clever tricks to make TLC feel like SLC.
Most (cheap) SSDs their performance goes off a cliff once you hit the boundary of these tricks.
[flagged]
It can be good to know that SSDs are fast until you exhaust the cache by copying gigs of files around.
It doesn’t hurt to be aware of the limitations even if for the common case things are much better.
We're talking about devices capable of >2GB/s throughput, and acquiring footage <.5GB/s. No caching issues, but I'm not buying el cheapo SSDs either. These are all rated and approved by camera makers. It wasn't brought up because it's not an issue. For people that are wondering why camera makers approve or not particular recording media, this is part of the answer. But nobody was asking that particular question and instead the reply tried to rain on my parade.
Not sure what warranted such an aggressive response.
I grew up in the 90s, on 56kb modems and PCs that rumbled and whined when you booted them up. I was at the tail end of using floppies.
I never said I didn't love the speed of SSDs, and when they just started to become mainstream it was an upgrade I did for everyone around me. I made my comment in part because you mentioned dumping 4K into the SSD and/or editing it. It can be a nasty surprise if you're doing something live, and suddenly your throughout plummets, everything starts to stutter and you have no idea why.
It wasn’t. IMHO the best feature and only real reason I hang around HN is exactly because I get to interact with others who know more.
The magic is in knowing how the trick works.
A bit further down the comment chain, martinald has a great succinct explanation:
https://news.ycombinator.com/item?id=44538489
> once you hit the boundary of these trick
Tell me more. When do I hit the boundary? What is perf before/after said boundary? What are the tricks?
Tell me something actionable. Educate me.
Your tone is quite odd here. I'm having difficulty parsing your intention, but I'm going to assume you're being genuine because why not.
For the RAM cache, you hit the boundaries when you exhaust the RAM cache. It performs faster, but is smaller and once full, data has to be off/loaded at the rate of the slower backing NAND. It might not be RAM, either, sometimes faster SLC NAND is used for the cache.
It's not really possible to describe it much more concretely than that beyond what you've already been told, performance falls off a cliff when that happens. How long "it" takes, what the level of performance is before and after, it all depends on the device.
There are many more tricks that SSD manufacturers use, but caching is the only one I know of related to speed so I'll leave the rest in the capable hands of Google.
Two of the main ones actually aren't really DRAM related but how full the drive is.
For most (all?) SSD drives they need a good 20% of the drive free for garbage collection and wear levelling. Going over this means it can't do this "asynchronously" and instead has to do it as things are written, which really impacts speed.
Then on top of that on cheaper flash like TLC and QLC the drive can go much faster by having free space "pretend" to be SLC and write it in a very "inefficient" size wise but fast method (think a bit like striped RAID0, but instead of data reliability issues you get with that it only works when you have extra space available). Once it hits a certain threshold it can't pretend anymore as it uses too much space to write in an inefficient fast way and has to write it in the proper format.
These things are additive too so on cheaper flash things get very very slow. Learnt this the hard way some years ago when it would barely write out at 50% of HDD speeds.
> cheaper flash like TLC and QLC the drive can go much faster by having free space "pretend" to be SLC
I'm afraid I don't understand how exactly this makes it faster. In my head, storing fewer bits per write operation should decrease write bandwidth.
Of course we observe the opposite all the time, with SLC flash being the fastest of all.
Does it take significantly more time to store the electrical charge for any given 1-4 bits with the precision required when using M/T/QLC encoding?
Tone's hard to read in text, but I read the tone as "eagerly excited to learn more". I am also interested and appreciate your comment here.
From a lawyer, 'tell me something actionable' would mean 'cross the line and say something upon which I can sue you', so I sympathise with anyone finding it unsettling.
In context it's more likely a translation issue, perhaps from modern translation picking up a challenging tone and inadvertently turning it to 'challenged by a lawyer or cop' mode to maintain the tone.
My tone is a combination of genuine curiosity and moderate annoyance at a dismissive but unhelpful comment.
RootsComment: SSD speed is miraculous! Jorvis: well ackshually is just RAM and tricks that run out Me: your comment provides zero value
I am annoyed by well ackshually comments. I’d love to learn more about SDD performance. How is the ram filled? How bad is perf when you cache miss? What’s worse case perf? What usage patterns are good or bad? So many interesting questions.
Look at this Kioxia Excera drive[0]. It plummets from 6800Mb/s (850MB/s) all the way to 1600Mb/s (200MB/s).
Its not really a well ackshually comment, there's real pitfalls. Especially when doing 4K. RAW 4K is 12Gb/s and would fill 450GB within 5 minutes. ProRes 4444XQ within 10 minutes. ProRes4444 in 40 minutes.
Martinald his comment is right too. By being very inefficient and treating TLC (or even QLC) as single level and only writing one bit to a cell, much higher performance can be extracted. But once you hit the 80% full threshold, the drive starts to repack the last/least used data into multiple bits per cell.
A RAM cache and SLC cache can both speed access times up, act as a write buffer and mask the shuffling of bits, but there is a limit.
Lastly, its kind of ironic to see me portrayed as jaded when someone else is the one pouring out vitriol all over the thread. Ah well.
[0]https://tweakers.net/reviews/12310/wd-sn5000-4tb-ssd-hoe-sne...
Right? I’m comparing my direct experience of enduring the pain of slower than Christmas HDDs to the incredible speeds of SSDs, and get a well actually it’s not SSDs that are fast blah blah. Look dude, I don’t care about your magic smoke that you’re so smart you know how the smoke is made. I just care that I can transfer data at blisteringly fast speeds. I couldn’t care less about QLC, SLC, or TLC because reading/writing at >2GB/s is all the tender loving care I need. Don’t rain on my parade because you’re jaded.
I haven’t had a spinning platter in my dev machine since I think 2008 or 2009. Even back then an SSD was the single biggest upgrade I’d seen the first 3D accelerator cards in the late 90s. (Oh god I’m old).
More recently we saw SSDs get added to video game consoles and load times are about 4x faster. And that’s with code/data optimized for a spinning plate not an SSD.
I know they aren’t actually magic. But they might as well be! I’d love to hear details on what weird conditions reduce their performance by 10x. That’d be cool and fun to know. Alas.
Toms Hardware usually includes a "Sustained Write Performance and Cache Recovery" test
The test measures the write cache speed and the time to the fall to the native NAND write speed. There are usually irregularities in the sustained write speeds as well.
https://www.tomshardware.com/reviews/wd-black-sn850x-ssd-rev...
The other test I've seen is based on writing and using up free space, SSD performance can drop off as the free space fills up and garbage collection efficiency goes down. I think this impacts random writes particularly
In the enterprise space, drives tend to keep more over provisioned NAND free to maintain more consistent performance. Very early on the SSD timeline, it was advisable to only allocate 80% of consumer drives if you were using them outside of desktops and expected the workload to fill them.
This hits home even more since I started restoring some vintage Macs.
For the ones new enough to get an SSD upgrade, it's night and day the difference (even a Power Mac G4 can feel fresh and fast just swapping out the drive). For older Macs like PowerBooks and classic Macs, there are so many SD/CF card to IDE/SCSI/etc. adapters now, they also get a significant boost.
But part of the nostalgia of sitting there listening to the rumble of the little hard drive is gone.
> But part of the nostalgia of sitting there listening to the rumble of the little hard drive is gone.
I remember this being a key troubleshooting step. Listen/feel for the hum of the hard drive OR the telltale click clack, grinding, etc that foretold doom.
Thank the gawds we no longer have to worry about the click of death
Now it's just a silent glitch of death.
I've just finished CF swapping a PowerBook 1400cs/117. It's a base model with 12MB RAM, so there are other bottlenecks, but OS 8.1 takes about 90 seconds from power to desktop and that's pretty good for a low-end machine with a fairly heavy OS.
Somehow the 750MB HDD from 1996 is still working, but I admit that the crunch and rumble of HDDs is a nostalgia I'm happy to leave in the past.
My 1.67 PowerBook G4 screams with a 256GB mSATA SSD-IDE adapter. Until you start compiling code or web surfing, it still feels like a pretty modern machine. I kind of wish I didn't try the same upgrade on a iBook G3, though...
>I kind of wish I didn't try the same upgrade on a iBook G3, though...
Oh god. Those were the worst things ever to upgrade the hard drive. Just reading this gave me a nightmare flashback to having to keep track of all the different screws. This is why my vintage G3 machine is a Pismo instead of an iBook.
Yeah this machine will probably never be the same. It does have an SSD now! But also a CD drive that isn't latching properly and the entire palmrest clicks the mouse button.
It doesn't help that I'm not a great laptop repair tech as is, but wow are those iBooks terrible. The AlBook was fine, and the Unibody MacBooks just a few years later had the HDD next to the battery under a tool-less latch.
I just picked up a 1.5GHz Powerbook G4 12-inch in mint condition. RAM is maxed out but I've been putting off the SSD-IDE upgrade because of how intrusive it is and many screws are involved.
I had a 2011 MBP that I kept running by replacing the HDD with an SSD, and then removed the DVD-ROM drive with a second SSD. The second SSD had throughput limits because it was designed for shiny round disc, so it had a lower ability chip. I had that until the 3rd GPU replacement died, and eventually switched to second gen butterfly keyboard. The only reason it was tolerable was because of the SSDs, oh and the RAM upgrades
Did you ever have the GPU issue? My sister had a 2011, I had to desolder a resistor (or maybe two?) on it to bypass the dGPU since it was causing it to boot loop. But now it's still running and pretty happily for some basic needs!
Yes, that's why it was on the 3rd repair. Apple knew they had issues and replaced it for me before by replacing the entire main board. Twice. The last time I took it in, they would no longer replace for free and wanted $800 for the repair. That was half the cost of modern laptop, so I chose no. I was unaware of being able to disable the GPU like that. I still have it on a shelf, but honestly, I don't see trying to do the hack now but might have considered back then.
I posted how I did it on my blog: https://www.jeffgeerling.com/blog/2017/fixing-2011-macbook-p...
Might still be worth doing for someone into older computers, I've considered putting a few of my old computers on the free pile at a VCF!
hmm, it is a Friday with nothing planned for the weekend...
While these are geared toward retrocomputing, there are things that attempt to simulate the sound based on activity LEDs: https://www.serdashop.com/HDDClicker
> For older Macs like PowerBooks and classic Macs, there are so many SD/CF card to IDE/SCSI/etc.
Would those be bandwidth limited by the adapter/card or CPU? Can you get throughput higher than say, a cheap 2.5" SSD over Sata 3/4?
You are limited at first by the IDE/SCSI interface, so below SATA speeds.
Oh I must have misread the comment initially as PCIE/SCSI
You should try now-discontinued Intel Optane especially p5800x. I got my OS running on them and they are incredible.
I'm running 12 of them for ZFS cache/log/special, and they are fast/tough enough to make a large array on a slow link feel fast. I shake my fist at Intel and Micron for taking away one of the best memory technologies to ever exist.
The endurance on those drivers is amazing.
I have (stupidly) used a too small Samsung EVO drive as a caching drive, and that is probably the first computer part that I've worn out (bar a mouse & keyboard).
Just a few more years until we get MRAM as viable storage technology. And affordable fusion, and hovercars.
Totally. I spent a lot of time 15-20 years ago building out large email systems.
I recently bought a $17 SSD for my son’s middle school project that was speced to deliver like 3x what I needed in those days. From a storage perspective, I was probably spending $50 GB/mo all-in to deploy a multi million dollar storage solution. TBH… you’d probably smoke that system with used laptops today.
In a similar vein, I had Western Digital Raptors striped in my gaming PC in the mid 2000s. I remember just how amazed I was after moving to SSDs.
> I come from the old days of striping 16 HDDs together (at a minimum number) to get 1GB/s throughput
Woah, how long would that last before you'd start having to replace the drives?
If you're interested in some hard data, Backblaze publishes their HD failure numbers[1]. These disks are storage optimized, not performance optimized like the parent comment, but they have a pretty large collection of various hard drives, and it's pretty interesting to see how reliability can vary dramatically across brand and model.
---
1. https://www.backblaze.com/cloud-storage/resources/hard-drive...
The Backblaze reports are impressive. It would have been very handy to know which models to buy. They break it down to capacity of the same family of drives so a 2TB might be sound, but the 4TB might be more flaky. That information is very useful when it comes time to think about upgrading capacity in the arrays. Having someone go through these battles and then give away the data learned would just be dumb to not take advantage of their generosity.
Many years ago I felt like I dodged a bullet splurging for the 4TB Seagates instead of the 3TB Seagates I needed.
Can confirm. My 3TB Seagate was the only disk so far (knocking on wood) that died in a way that lost some data. Still managed to make a copy with dd_rescue, but there was a big region that just returned read errors and I ended up with a bunch of corrupt files. Thankfully nothing too important...
Depending on the HDD vendor/model. We had hot spares and cold spares. On one build, we had a bad batch of drives. We built the array on a Friday, and left it for burn-in running over the weekend. On Monday, we came in to a bunch of alarms and >50% failure rate. At least they died during the burn-in so no data loss, but it was an extreme example. That was across multiple 16-bay rack mount chassis. It was an infamous case though, we were not alone.
More typically, you'd have a drive die much less frequently, but it was something you absolutely had to be prepared for. With RAID-6 and a hot spare, you could be okay with a single drive failure. Theoretically, you could lose two, but it would be a very nervy day getting the array to rebuild without issue.
I asked because I did a makeshift NAS for myself with three 4tb ironwolf, but they died before the third year. I didn't investigate much, but it was most likely because of power outages and a lack of a nobreak PSU at that time. It's still quite a bit of work to maintain physical hard drives and the probability of failure as I understand tend to increase the more units the array has because of inverse probability (not the likelihood of one of them failing but the likelihood of none of them failing after a period of time, which is cumulative)
Any electronic gear that you care about must be connected to a UPS. HDDs are very susceptible to power issues. Good UPS are also line conditioners so you get a clean sine wave rather than whatever comes straight from the mains. If you've never seen it, connect a meter to an outlet in your home and what how much fluctuations you get throughout the day. Most people think about spikes/surges, while forgetting that dips and under-volting is damaging as well. Most equipment have a range of acceptable voltage, but you'd be amazed at the number of times mains will dip below that range. Obviously location will have an affect on quality of service, but I hear my UPSes kick in multiple times a week to cover a dip if only for a couple of seconds.
The fun thing about storage pools is that they can lull you into thinking they are set it and forget it. You have to monitor SMART messages. Most drives will give you a heads up if you know where to look. Having the fortitude to have a hot spare instead of just adding it to the storage pool goes a long way from losing data.
I run 24x RAID at home. I’m replacing disks 2-3 times per year.
Are your drives under heavy load or primarily just spinning waiting for use? Are they dying unsuspectedly, or are you watching the SMART messages and being prepared when it happens?
They’re idle most of the time. Poweted on 24/7 though, and maybe a few hundred megabytes written every day, plus a few dozen gigabytes now and then. Mostly long-term storage. SMART has too much noise; I wait for zfs to kick it out of the pool before changing. With triple redundancy, never got close to data loss.
To be clear, I should have said replacing 2-3 disks per year.
So, now someone can strip several SSDs to gain more performance as well for other purposes than video editing, right?
[dead]