What's new

messed up external SSDs drives + file systems + and MacOS and Windows interchange?

M

Michael Lindsay

Hi and thank you for any thoughts!

My question about small consumer external SSD (2tb and 4tb) drives + file systems + and MacOS and Windows interchange.

Quick background:
I have a little studio that has some post and Cine cameras (shooting RAW… RED/ARRI/Phantom)
My post runs off a 100TB+ 10gig SAN
My final backup is LTO-7 LTFS
However, till a project is over, there are copies of Rushes (and other assets) on portable consumer SSD drives (T7 and T9 for example). These are important from my workflow pov
Because our DIT stations are Mac and most of our post is PC i evaluated Exfat and found it acceptable. It seemed to work fine for a few years. Then I moved to Samsung t7 and t9 4tb drives .

Note never had an issue with exfat till I moved to Samsung t7 and t9! So I initialed assumed this the issue.
Now I am looking at 4xdrives with write speeds of less than 1MB/s (read is about 300+). If I reformat I get them back to predicted speeds! So I am assuming my issue is an exfat corruption?

I am going to move to making all external SSDs files system a journaled system. Basically either NTFS or AFPS and installing Paragon (workstations) or Tuxera (macLaptops).

Which route has the least impact on the speed of the drives and which is the most reliable?

Thank you!!

Michael
 
i center on HFS, avoid APFS, use Paragon on both OSX and W10/11 machines, no speed diffrence comapreing NTFS -vs- HFS/Paragon on the windoze machines
on set the HFS drives are easier for the DiT's to repair should it be needed, or shift work to a MBP that does not have Paragon installed should someone dump a diet coke over the main machine
 
Avoid APFS because not everyone has invested in upgrading windows machines to the newer format Paragon.
HFS is fine. NTFS is better. NTFS on mac works perfect. NTFS - in the event of corruption - recovers (R-studio and similar) much better that HFS.

Do not use EXFAT! There is some compatibility issues with the auto-repair on exfat because apple cheaped out and did not pay microsoft (owner of exfat) for the full licence. What it comes down to - if the drive is disconnected abruptly - then the next time you connect it to the system - it will run auto-repair as it should on an exfat drive. The problem that I have found and also replicated - if you autorepair a drive that was formated on a mac on a windows - then it is possible it will get corrupt or broken in some sort of way. The other way around is also true. Keep in mind that certain factors have to come together (writing is a big one) for this to happen - but still - this is messed up and this file-system should be avoided at all times if possible.
 
Hi Peteris.. just checking you mean HFS+?

PS I am def dumping exfat. It is more practical for me to use a mac file system (HFS+) and have the software layer on one or two PCs at work as I have quite a few MAc Laptops. However any significant drop in access (read speed) on the PC would make me want to switch around.... for me significant would be over 10%
 
+1 on do not use ExFat. Every time we have mounting issues or slow speeds it's ExFat. HFS+ or NTFS for shuttle/client drives, use Pargon for compatibility on Mac and Windows. ExFat was a good idea but in practice is the least reliable. I would never use it in a mission critical workflow. We use ZFS for working server volumes.
 
Michael Lindsay said:
Note never had an issue with exfat till I moved to Samsung t7 and t9! So I initialed assumed this the issue.
Now I am looking at 4xdrives with write speeds of less than 1MB/s (read is about 300+). If I reformat I get them back to predicted speeds! So I am assuming my issue is an exfat corruption?
Click to expand...
I had a similar thing happen to my t7 but i can't remember which file system it was formatted to (but i'm 85% sure it was exfat), and as i was troubleshooting i read online that it happens when you fill the drive to 100%.

after re-formatting (exfat) i had no issues. hope it helps.
 
@ Michael In regards to USB C SSDs such as the Samsung T7 series another issue to consider is USB C SSD Drives do not support "TRIM" on Mac OS. TRIM is typically only supported on internal drives and Thunderbolt devices.

Based on my experience when the T7 SSD is near full capacity the performance will greatly degrade to the point it will be unusable. Repeatedly deleting large files from the T7 and then rewriting new files also aggravates the lack of TRIM.

The solution is to format the SSD to gain back the performance.

As a general protocol don't rely on USB C SSDs as the sole depository of mission critical media when operating on Mac OS.
 
Tony Sal said:
In regards to USB C SSDs such as the Samsung T7 series another issue to consider is USB C SSD Drives do not support "TRIM" on Mac OS. TRIM is typically only supported on internal drives and Thunderbolt devices.

Based on my experience when the T7 SSD is near full capacity the performance will greatly degrade to the point it will be unusable. Repeatedly deleting large files from the T7 and then rewriting new files also aggravates the lack of TRIM.
Click to expand...
I have about 20 of the 4TB Samsung T7 Shield SSDs and have seen this problem on MacOS several times. The specifics are more nuanced.

MacOS clearly issues TRIM commands to the Samsung T7 and similar USB-C drives each time they are mounted -- regardless of whether "trimforce" is enabled or disabled. This can be seen by running the below command from terminal, then mounting the drive. Based on my testing, if you have a Samsung T7 on a Mac and if you periodically unmount and remount it (especially after huge numbers of writes) the problem is much less likely. This is because TRIM happens for sure upon every mount operation.

log stream --predicate 'senderImagePath CONTAINS "apfs" AND eventMessage CONTAINS "spaceman"' --info

Study of the filesystem internals indicates APFS may issue trim commands at other times besides at mount, but so far, I haven't seen that.

Despite these auto-issued TRIM commands by APFS, it's true that the Samsung T7 SSD can enter a persistent severely degraded write performance condition, whereby the writing data rate is about 2 megabytes per second.

The internal SSD controller on the device apparently enters a pathological steady state, requiring heavy garbage collection (GC) and causing write amplification on essentially every write.

I've used Samsung Magician to update the firmware on several of my T7 drives that encountered this. My last firmware update was from version FX171P2Q_3602_01070403 to version FX172P2Q_3603_02000200. However, I believe there are reports of this problem happening on that version as well. It's unclear if it helps. Updating the firmware is difficult because Samsung Magician (on MacOS) requires a kernel extension, which means on current MacOS you must first boot into Recovery Mode and lower system security, reboot, then the Samsung kernel extension installation requires at least two more reboots.

I think it can happen on both Windows and Mac, but you tend to hear about it a lot on Mac. The only solution is backing up all data then doing a low-level erase and reformat. On a 4TB T7, the low-level erase can take a couple of hours.

# First this command (disk6 is an example. Do diskutil list to see the disk names)
diskutil secureErase 0 /dev/disk6

# Then when it finishes, this command
diskutil eraseDisk APFS "4TB_NF50" /dev/disk6
 
Further study and testing with several different T7 SSDs shows the design is constrained by an inexpensive, low-performance internal controller. The internal design uses a low-performance Samsung “Pablo” controller and is a DRAM‑less NVMe SSD behind an ASMedia ASM2362 USB‑to‑NVMe bridge chip. After a certain pattern of write/erase operations, it can enter a pathological garbage-collection state in which write performance persistently degrades to about 0.1% of normal speed. This also happens on the latest firmware. It is not a TRIM issue but a design issue. The only solution is as stated above.

For Mac, I'd recommend a Thunderbolt SSD with hardware SLC write cache and a higher-performance controller. One example is the Oyen Digital U34 Bolt. Internally, it uses the high-performance Phison PS5018‑E18 controller and a significant RAM cache. My team has about 10 of those. At 4TB it is about $600 vs $500 for the T7, but it is (a) much faster and (b) impervious to the write performance issues of the Samsung T7 or similar lower-end USB-C SSDs. https://www.amazon.com/dp/B0CV62T1BW
Meaningful investigative work on disk I/O requires a professional test tool like diglloydTools DiskTester (which is no longer developed but is still available), plus an understanding of available platform-specific monitoring tools. On Windows, there are many extensive monitoring tools, such as PerfMon and Windows Performance Analyzer (both free), plus lots of third-party tools. On Mac, there is only XCode Instruments, which is really designed for app developers, but it's free.

diglloydTools DiskTester is now available in a platform-independent command-line Java version, but the website is very difficult to navigate. https://diglloydtools.com/disktester.html

Post-production workflow specialists are probably better off using the free "fio" I/O stress utility. On Mac it's available through the Homebrew package manager: https://brew.sh/

I think on Windows 11 there is a similar package manager named Winget, but I've never used it.

fio tutorial on Linux (should be similar on Mac and Windows):
 
You must log in or register to reply here.
Back
Top