HP EVA array with Microsoft Hyper-V R2

Problembeschreibung:
===================
Windows 2008 R214 Node Cluster
HP EVA 6000
DualPort FC pro Blade.

(Server) meldet das Verbindung zum Storage nicht mehr verfügbar war.

Cluster Shared Volume 'Volume6' ('Cluster Disk 7') is no longer accessible from this cluster node because of error 'ERROR_TIMEOUT(1460)'. Please troubleshoot this node's connectivity to the storage device and network connectivity.
Cluster Shared Volume 'Volume4' ('Cluster Disk 5') is no longer available on this node because of 'STATUS_CONNECTION_DISCONNECTED(c000020c)'. All I/O will temporarily be queued until a path to the volume is reestablished.
Cluster Shared Volume 'Volume6' ('Cluster Disk 7') is no longer available on this node because of 'STATUS_CONNECTION_DISCONNECTED(c000020c)'. All I/O will temporarily be queued until a path to the volume is reestablished.

Scope Agreement:
================
Ziel der Anfrage ist, Ihnen bei der Problemanalyse und Problembehebung zu helfen, sollte es nicht ein Drittanbieter Problem oder ‚by design‘ Folge sein.

- Das Problem von gestern war identisch, der Node Server hat die Verbindung zum Storage verloren und das IO wurde durch Netzwerk an andere Nodes geroutet und dann auf die Disk geschrieben.
- Sie verwenden den multipath Treiber von HP.

Multipath in Windows Server 2008 R2
When setting up the storage environment, be sure to obtain the latest Multipath I/O (MPIO) drivers and management software and install them on each server that accesses the EVA. At the release of this white paper, the current version of the HP MPIO Full Featured DSM for EVA4x00/6x00/8x00 families of Disk Arrays (EVA MPIO DSM) is 4.00.00, which does not yet support Cluster Shared Volumes. If Cluster Shared Volumes are used, the built-in Microsoft Windows MPIO drivers must be used. To use the Microsoft MPIO drivers and tool, enable the Multipath I/O feature as explained in

Link HP MPIO Full Featured DSM for EVA4x00/6x00/8x00 families of Disk Arrays 4.01a: http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&cc=us&prodTypeId=18964&prodSeriesId=421492&swItem=co-84060-1&prodNameId=421494&swEnvOID=4064&swLang=13&taskId=135&mode=3

---Weitere Informationen---

HP MPIO Full Featured DSM for HP EVA now supports Cluster Shared Volumes


In an HP Storage Works EVA Best Practices document published in February 2010, we were advised not to use the HP MPIO DSM with Cluster Shared Volumes, but as an alternative to use Microsoft Native MPIO only. This was unfortunate because the HP MPIO Full Featured DSM allows for better visibility and control of the multiple paths between Hyper-V R2 host and its storage.

During a storage change in one of our datacenters, we noticed that not all nodes were able to see all CSV disks. The Hyper-V R2 cluster we were working on has eight cluster nodes. On another occasion, one of my cluster MVP colleagues, Edwin van Mierlo, had explained to me that in a Hyper-V R2 cluster with CSV, the number of SCSI registrations can add up very quickly.
In our case we work with an 8-node Hyper-V R2 cluster with HP BL490c blades and HP EVA6400 storage array with 20 LUNs.
Each CSV LUN counts as one SCSI Reservation. The coordinator node makes a persistent reservation for each LUN it owns in the cluster. Because all other nodes in the cluster must be able to read/write to that same LUN (CSV with Direct I/O), they must register all possible paths to that LUN, dependent on one of the following two registration methods:
Storage that require a registration per path
Storage that require a registration per Initiator
The software on the host will send a registration over all of the paths. So the Storage will receive a registration REGISTER_ IGNORE_EXISTING_KEY on all paths. It is then up to the storage to do the correct thing. http://www.freepatentsonline.com/EP1246060.html
In a two node cluster this would mean:
Example: 2 node cluster, 2 Initiators per node, 4 paths per Initiator, 1 Disk (CSV) :
For Storage "class" A, per path registration, you get:
16 registrations (4 paths x 2 initiators x 2 nodes x 1 disk)
1 reservation (from 1 node in the cluster that owns the disk)
For Storage "class" B, per Initiator registration, you get:
4 registrations (2 nodes x 2 initiators)
1 reservation (from 1 node in the cluster that owns the disk)
In an 8-node cluster with 20 CSV disks, the number of registrations go up quite significantly:
For Storage "class" A, per path registration, you get:
1280 registrations (4 paths x 2 initiators x 8 nodes & 20 disks)
20 reservations (from all nodes owning an online disk)
For Storage "class" B, per Initiator registration, you get:
16 registrations (8 nodes with 2 initiators)
20 reservations (from all nodes owning an online disk)
I am not sure what method HP uses for HP EVA, but I hope it is “class” B. If anyone knows, please let me know!
While searching for other cases with CSV problems in HP EVA environments I struck on this important piece of news:
The author of the blog refers to a newly released version of HP MPIO Full Featured DSM v4.01 which fully supports Cluster Shared Volumes with Windows Server 2008 R2
See the original blog: http://blogs.technet.com/b/hugofe/archive/2010/06/18/hp-dsm-support-with-csv.aspx
Download the software here.
Download the release notes here.
Important parts:
“if there is a stray reservation on a disk device, the device becomes inaccessible to the host systems connected to it. This is because the strong reservation type (persistent reservations) supports load balancing in clusters. You can use the persistent reservations clear utility that comes with HP MPIO EVA DSM package to remove the stray reservations.”
“Note: For Microsoft Windows Server 2008 and Windows Server 2008 R2 Failover cluster to work properly, set the array host mode on the HP CommandView EVA to ‘Microsoft Windows LH’ or set custom host mode to “00000004198009A8”.
In the CommandView we use (v9.2.1) we don’t have the Microsoft Windows LH host mode option.
If you issue the hpprutil devices command you can see this:

You can clear the reservation with hpprutil clear devices

We finally seem to have some light at the end of the tunnel about Hyper-V R2 clusters and mysteriously disappearing cluster shared volumes.

---Weitere Informationen---

HP DSM now supports CSV

If you see Clusters with CSV on EVA Devices that sporadically loose disks, you might want to look at the DSM that until the June did not support CSV.
From HP:
On 4rth June, 2010, we released HP MPIO FF DSM v 4.01 which supports CSV with Windows Server 2008 R2. Attached is the release notes and the link for download is the following:
http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?lang=en&cc=us&taskId=101&prodClassId=-1&contentType=SupportManual&docIndexId=64180&prodTypeId=18964&prodSeriesId=421492
I’m attaching this release note and install guide below is the link for download
http://h20000.www2.hp.com/bizsupport/TechSupport/CoreRedirect.jsp?redirectReason=DocIndexPDF&prodSeriesId=421492&targetPage=http%3A%2F%2Fbizsupport2.austin.hp.com%2Fbc%2Fdocs%2Fsupport%2FSupportManual%2Fc02221077%2Fc02221077.pdf