Tag Archives: ZFS

ZFS, NFS, ESXi and slow performance

Posted on by
Reply

On a previous post I mentioned putting the ZIL on a SLOG. Let me get into why this was done. There is a known problem with NFS as it relates to synchronous writes, especially with ESXi. I knew of this problem when I set out to use NFS for ESXi datastores. ESXi uses synchronous writes when writing data to a NFS datastore. ESXi sets the O_SYNC flag when writing data.

O_SYNC – The file is opened for synchronous I/O.  Any write(2)s on the resulting file descriptor will block the calling process until the data has been physically written to the underlying hardware.

So what’s happening here is that ESXi request an acknowledgement that the data it sent is actually being written to stable storage before it makes another write. Normally the ZIL (ZFS Intent Log) is stored on same storage pool where the data is being written if you do not have it on a SLOG (Separate Intent Log) device. The ZIL holds synchronous data written to it then flushes it out as a transactional write. Without the ZIL on a SLOG I got about 8MB/s writes. After I enabled the ZIL on a SLOG, which is on a SSD, I now get about 30MB/s which is a significant improvement. Keeping in mind this is on a 3 drive RAIDZ using 7200rpm disks. So you have a few choices in my opinion if you’re going to be using ESXi with NFS on ZFS:

  1. Put your ZIL on a seperate device that has a fast write speed like a SSD or some type of Flash RAM device.
  2. Use iSCSI.
  3. Don’t use ZFS.
  4. Disable sync writes. (I’d only do this if you don’t give a hoot about the data that’s being written)

Now you may need to experiment with your setup as there could be other factors creating your problems or perhaps another solution. If you go with option 1 I’d recommendan SSD with a fast write speed using SLC NAND chips.

Sources:
http://pthree.org/2012/12/06/zfs-administration-part-iii-the-zfs-intent-log/
http://forums.freenas.org/threads/sync-writes-or-why-is-my-esxi-nfs-so-slow-and-why-is-iscsi-faster.12506/
https://blogs.oracle.com/realneel/entry/the_zfs_intent_log
http://constantin.glez.de/blog/2010/07/solaris-zfs-synchronous-writes-and-zil-explained

Custom NAS build using FreeNAS

Posted on by
3

So I embarked on the journey of building a NAS for my home lab. The purpose of this storage solution was to run VM’s and store backups from a few PC’s in my home. The build consisted of the following:

  • 3 x 2TB Seagate Barracuda 7200RPM HDD’s
  • Corsair CX430 Power Supply (80 plus)
  • NZXT Gamma Mid tower case
  • MSI B75MA-G43 Motherboard
  • Intel Celeron G1610
  • 8GB (2 x 4GB) Corsair Vengence DDR3
  • 4GB (2 x 2GB) G.Skill Ripjaws
  • Intel 313 20GB SSD
  • Crucial v4 64GB SSD
  • ADATA 8GB USB3 Thumbdrive
  • Syba PEX24033 PCI-e Dual port Gigabit ethernet

I decided to use FreeNAS 9.1.0 on this build. The FreeNAS interface is pretty good and there is a strong community behind it. FreeNAS can be ran from a USB drive which is how my NAS is currently ran. The 3 2TB HDD’s make up the main storage pool using ZFS in a RAIDZ array. A RAIDZ array is similar to RAID5. The Intel 313 20GB SSD is used for the ZIL, otherwise know as the ZFS Intent Log which when put on a separate device is known as the SLOG or separate intent log. The ZIL is primarily write intensive so its important to have a device that accommodates a lot of writes and is fast enough to receive those writes. The second SSD is used for L2ARC which is a read cache between the RAM (ARC) and your storage pool. There’s quite a bit of knowledge out there about these and I implore you to read up about them and decide if its right for you.

When this build is up and running it uses about 50-60 watts which met my needs for a decent storage location that didn’t use a lot of power. The Intel Celeron G1610 helped a lot with this. This Celeron G1610 was released this year and is based on the Ivy Bridge architecture. It was using this or an Atom processor and I chose the Celeron so it would give the NAS some uumph. The performance of this processor compared to the value of its price is awesome.

My first mistake with this build was only using 4GB of ram. ZFS will disable pre-fetching unless you have more than 4GB. Use more than 4GB! My second mistake was only have 1 on-board NIC. The dual port card was an extra purchase and helped when using it with ESXi and multiple datastores. If you are just using your NAS for basic storage you may be able to get away with just 1 NIC.

Now I have to vent about the power supply. I’ll be on my 3rd Corsair CX430 power supply in about 2 weeks. I have to applaud Corsair for their easy warranty returns but having 2 of these go bad just sucks. If this 3rd one goes bad I’ll probably return it for another brand or perhaps take a closer look at the other parts in my build to make sure nothing else could be causing the issue.

FreeNAS System