12

I've been tasked with selecting two new switches intended to be our new iSCSI SAN backbone.

Currently, we are using two Dell N4064 improvised for the task since that's what we had when the need arrived, but now we are going to replace them with two smaller, 24 port 10Gbit Ethernet switches.

I would ask if any 10Gbit Ethernet will perform adequately as an iSCSI backbone or if there are any specs I should be looking for or avoiding in my selection.

Thank you in advance for your input.

ÜM Mineiro
  • 129
  • 4
  • Clarification - this is a stack of 2x switches dedicated for redundant iscsi connections, and not for general traffic of which some is iscsi and some is "frontside" traffic ? – Criggie Feb 21 '20 at 00:02
  • 1
    Power consumption (aka heat generated) can become an issue. That's true for any device in your server room, of course. But if your boss thinks that he's found a great deal on *old* powerful (read: power hungry) switches, tell him that he might need to upgrade the AC as well. – Klaws Feb 21 '20 at 14:51
  • 1
    Criggie. This are to be used as two independent, non stacked, switches. They will be used just for the iSCSI traffic. – ÜM Mineiro Feb 21 '20 at 17:21
  • Latency and bandwith are most important. Make sure there is no backplane bottleneck and the switches work cut-through. You also may want to consider DAC cables. And I would use rather more than less ports, if you can avoid upstream links it's a big plus. – eckes Feb 22 '20 at 01:30
  • Did any answer help you? If so, you should accept the answer so that the question doesn't keep popping up forever, looking for an answer. Alternatively, you can post and accept your own answer. – Ron Maupin Dec 17 '20 at 15:40

4 Answers4

11

In a word: deep buffers, preferably per-port buffers

iSCSI does not tolerate frame drops. Delays are sort of ok, but a switch dropping a frame will create all manner of issues.

(Finding buffer size -- and type -- in manufacturer specs can be a challenge. They aren't always provided, and what is provided isn't always true.)

Ricky
  • 31,438
  • 2
  • 43
  • 84
  • I would actually argue the switch should do cut-through forwarding for low latency and avoid store+foreward and therefore should not buffer. – eckes Feb 22 '20 at 01:27
  • 3
    If two or more machines try to send a frame at the same time, without a buffer, it gets dropped. Storage is about *reliability*, not low latency. In a SAN, it always takes measurable time to get the block you want; even an SSD is not instant. – Ricky Feb 22 '20 at 04:00
8

Make sure the switch is non-blocking. That is, the switch backplane can support all ports at line rate with no oversubscription. For a 24-port switch, the minimum backplane bandwidth is 24 x 10G, or 240Gb.

Zac67
  • 81,287
  • 3
  • 67
  • 131
Ron Trunk
  • 66,852
  • 5
  • 65
  • 126
4

Be sure to understand what your iSCSI implementation on the end systems has to say about LACP (Port Aggregation) and/or if it mandates path isolation, to have two completely independent network paths between any initiator and any target.

If either is true, then MLAG (Multichassis Link Aggregation) capabilities probably won't be required.

Marc 'netztier' Luethi
  • 8,654
  • 1
  • 13
  • 30
3

In complement to Ron's answer, you need to look for Jumbo Frame. That is not mandatory, but it will significantly improve the bandwidth.

A good point too is the latency. It should ideally be below the µs.

You may also go for a full SFP+ switch, because depending on the equipment, you will have a mix of interfaces.

Bruno Mairlot
  • 131
  • 2
  • 3
    "jumbo frames ... significantly improve the bandwidth" - that largely depends on your hardware. With decent offloading or hardware initiators the difference between standard and jumbo frames may be hardly noticable. – Zac67 Feb 20 '20 at 15:04
  • I agree, in a scenario where I switched on jumbo frames it did only improve a few percents because the drivers have been doing receive aggregating. That's at least a feature of the Intel server adapters. – eckes Feb 22 '20 at 04:08
  • We actually tested this on VMware ESXi, and jumbo frames added single percentage points in performance, very similar to this article (http://www.boche.net/blog/2011/01/24/jumbo-frames-comparison-testing-with-ip-storage-and-vmotion/), so we decided to just to stick with 1500 due to the configuration demand that jumbo frames requires in larger environments. – Stuggi Feb 25 '20 at 11:44