Return to Home --> Modules --> Fibre Channel Layers

 

FC-4 Layer

 

FC-4 maps upper level protocols to FC-2. The correspondent FC-2 mapping of the FC-4 Information Unit (IU) is the Fibre Channel Sequence. FC-4 module is composed of one FC-4 Controller module,  one or more SCSI over Fibre Channel (SCSI_FCP) modules, and Fibre Channel Service modules - Extended Link Services (FC-ELS), Generic Service (FC-GS), and Switch Internal Link Service (SW-ILS).  Note N_Port has no SW-ILS module and F/E_Port has no FCP modules. Strictly speaking, FC-ELS should be part of the FC-2 Layer. SimSANs just merges it into the FC-4 Layer - if you happen to look at the source codes of some HBA drivers you will find such kind of implementation is natural. In SimSANs, each FC-2 Layer (FC_Port) must have an FC-4 Layer associated with it so you can consider FC-4 Layer as FC_Port driver.

 

Mouse over the boxes to view specific modules.

 

Click to view "FCP Module" Click to view "FC-4 Controller Module" Click to view "FC-ELS Module" - Fibre Channel Extended Link Service Click to view "FC-GS Module" - Fibre Channel Generic Service Click to view "SW-ILS Module" - Switch Internal Link Service Click to view "Host/CHIP CPU Module" Click to view "Switch Transport Module" Click to view "FC-2 Module"

FC-4 Layer

Click here to see its VML version (requiring IE)

 

Extended Link Services : FC-ELS

FC-FS defines many Extended Link Services but SimSANs only implements some of those most important ones, especially those related to the N_Port Device Discovery: FLOGI - Fabric Login, PLOGI - N_Port Login, PLOUT - N_Port Logout, ABTX - Abort Exchange, and RSCN - Registered State Change Notification.

 

FLOGI enables the requested N_Port to get its Address_ID assigned by the attached F_Port, and create a communication channel with that F_Port by setting or exchanging the operational parameters: R_A_TOV, E_D_TOV, and BB_Credit, etc. The How-To section shows where to setup these parameters before running a SimSANs application. FLOGI Request Sequence uses F_Port WKA (0xFFFFFE) as the destination address (D_ID). Upon the completion of FLOGI, the requesting N_Port resets link. Following this link reset, operational parameters (e.g., BB_Credit) take effect.

 

PLOGI enables the requested N_Port to create a communication channel (EE_Pair) with the remote N_Port it wants to communicate with by setting and exchanging operational parameters: Class-3 Concurrent Sequence and REC_TOV value for FCP IU, etc. Before PLOGI, each N_Port has its own default REC_TOV, which is E_D_TOV plus 1000ms, e.g., if E_D_TOV is 2000ms then default REC_TOV is 3000ms. The larger REC_TOV will be setup as the agreed timer value for both N_Port during the PLOGI procedure. In FC-FS, PLOGI actually does not support the exchange of REC_TOV value instead RTV (Read Time Out) FC-ELS enables an FC_Port to read E_D_TOV value from another FC_Port. For simplicity I merge this function into the PLOGI.

 

RSCN is issued by Fabric Controller (WKA 0xFFFFFD) to notify all the registered N_Ports when an event occurs. In SimSANs, RSCN usually works with SW_RSCN to detect failed N_Ports or discover newly registered N_Ports within the Fabric. See "FCP N_Port Device Discovery" below. Note FC-FS allows an N_Port to request RSCN but SimSANs does not enable this feature.

 

PLOUT is used to tear down the communication channel between an N_Port login pair. In SimSANs implicit PLOUT is performed by a registered N_Port when it detects a logged-in N_Port fails - EE_Pair is removed and relevant resources are released at this moment.

 

ABTX is originated by an N_Port requesting the remote N_Port to abort an Exchange. In SimSANs, this procedure usually happens when a SCSI IO (at the SCSI Initiator side)  is experiencing time-out due to the occurrence of Frame loss caused by link failure while the SCSI Target has no idea about that and still is performing normal IO operations. In this case, ABTX is originated by the SCSI Initiator requesting the SCSI Target to remove the ESB (Exchange State Block) and other related resources pertain to the time-out IO. SCSI Initiator may re-schedule this IO later.

 

Note in SimSANs, PLOGI and ABTX are always originated by SCSI Initiator Port, usually the N_Port in a Host or in a CHIP (if the N_Port in a CHIP initiates SCSI IOs it must be participating in a remote mirroring session). Besides, all of the above five ELSs use Class-3 Frame.

 

Generic Service : FC-GS

FC-FS defines many Generic Services while SimSANs only implements the Name Server - GS_Type 0xFC (Directory Server) with GS_Subtype 0x02. Three Name Server commands are implemented: RFF_ID - Register FC-4 Features by Port Identifier, GFF_ID - Get FC-4 Features by Port Identifier, and GID_FT - Get Port Identifiers by FC-4 Type. Similar to FC-ELS, these three commands are related to N_Port Device Discovery.  Common Transport IU (CT_IU) is introduced for this service.

 

RFF_ID is sent out to the attached Switch by an N_Port after it completes FLOGI. The port Address_ID as well as its FC-4 Feature (in SimSANs it is either FCP_Initiator or FCP_Target) are registered into the Domain Name Server Database during this procedure.  The Switch then notify other registered N_Ports (by issuing RSCN FC-ELS) or other inter-connected Switches (by issuing SW_RSCN SW-ILS) so that interested N_Ports can be able to PLOGI with this newly registered port and create communication channel with it.  Class-3 Frame is used for this command and Directory Server WKA (0xFFFFFC) is the D_ID of the Request CT_IU.

 

GFF_ID is requested either by an N_Port or a Domain Controller (through an E_Port) to get an N_Port's FC-4 Feature by its Port Identifier.  For the N_Port originating this command, Class-3 Frame is used and the Request CT_IU uses Directory Server WKA (0xFFFFFC) as the D_ID. For the E_Port originating this command, Class-F Frame is used and the Request CT_IU uses Domain Controller (0xFFFC00+Domain_ID) as the D_ID - please refer to FC-SW-2 for more about Distributed Server Addressing. 

 

GID_FT is requested either by an N_Port or a Domain Controller (through an E_Port) to get an N_Port's Identifier by its FC-4 Type. In SimSANs, only one FC-4 Type is supported: SCSI_FCP (0x08) since we're focusing on SCSI over Fibre Channel while other FC-4 mappings are not in our interests.  Like GFF_ID, for the N_Port originating this command, Class-3 Frame is used and the Request CT_IU uses Directory Server WKA (0xFFFFFC) as the D_ID. For the E_Port originating this command, Class-F Frame is used and the Request CT_IU uses Domain Controller (0xFFFC00+Domain_ID) as the D_ID - please refer to FC-SW-2 for more about Distributed Server Addressing. 

 

FCP N_Port Device Discovery: (1) When an FCP Initiator N_Port completes FLOGI and RFF_ID, it issues a GID_FT to the Name Server (usually located in the attached Switch) to get a list of N_Port Identifiers that support SCSI_FCP. For each Port Identifier returned in the Accept CT_IU for that GID_FT, a GFF_ID is issued to the Name Server to get the FCP Feature for that N_Port device. If the device is determined to be an FCP Target, the FCP Initiator performs PLOGI with the Target. SCSI IO operations can be performed between the FCP Initiator / Target N_Port pair if the PLOGI succeeds. (2) Similarly, GFF_ID and GID_FT are issued between adjacent E_Ports by their Domain Controllers so that the Name Servers in the adjacent Switches (or Domains) can exchange and synchronize their Name Server Databases. Note the synchronization procedure normally happens when an E_Port is up and attached to a current Switch fabric. (3) Besides, newly registered N_Port Identifier will be flushed to other FC_Ports (RSCN to N_Ports and SW_RSCN to E_Ports). An FCP Initiator receiving RSCN will issue GFF_ID to get the FC-4 Feature for that new device and PLOGI will be performed if FCP Target is determined. An E_Port receiving SW_RSCN causes its Domain Controller to request GFF_ID to get the FC-4 Feature for that new device in order to update its Name Server Database.

 

Switch Internal Link Service : SW-ILS

SW-ILS module is used to configure Switch Fabric, and build and maintain routing table. Several important commands are implemented in the SimSANs: ELP - Exchange Link Parameters, HLO - FSPF Hello, LSU - FSPF Link State Update, LSA - FSPF Link State Acknowledgement, and SW_RSCN - Inter-Switch RSCN.  All of these SW-ILS are issued between adjacent E_Ports and use Fabric Controller WKA (0xFFFFFD) as the S_ID and D_ID for both request and reply sequences. 

 

ELP create communication channel between adjacent E_Port by setting or exchanging the operational parameters: R_A_TOV, E_D_TOV, BB_Credit, and Port_Name etc., whose procedure is similar to FLOGI between an N_Port / F_Port pair as stated in FC-ELS. Note both E_Ports may simultaneously originate ELP to each other. If this occurs, Port_Name needs to be compared. If the Port_Name in the received ELP is lower than this port's name, the port sends SW_RJT. If the Port_Name in the received ELP is greater than this port's name, the port sends SW_ACC. Upon the completion of ELP, the port issued successful ELP resets the link. Following this link reset, operational parameters (e.g., BB_Credit) take effect.

 

HLO establishes "two-way" communication between E_Ports. LSU and LSA are used to synchronize and maintain the topology databases of the adjacent Switches. Upon the completion of topology database synchronization, "adjacency" is achieved, which means the inter-switch link (ISL) is able to delivery and forward Class-3 Frames. Link State Record (LSR) and Link Descriptor (LD) are encapsulated into the payload of LSU and LSA for the Switches to perform path selection and build and maintain routing tables. 

 

SW_RSCN is used to distribute RSCNs between Switches. It is similar to RSCN except that SW_RSCN uses SW-ILS as the transport while RSCN uses FC-ELS as its transport. SimSANs implements SW_RSCN as part of the N_Port discovery procedure - see above "FCP N_Port Device Discovery".  Besides N_Port discovery, The distribution of SW_RSCN also enables the Switches to detect an N_Port failure and update the Name Server Databases by removing the entry to the failed N_Port. 

 

SCSI over Fibre Channel : SCSI_FCP

SCSI_FCP (please refer to FCP-2) creates a mapping between Upper SCSI Layer and  Lower Fibre Channel Protocol Layer. Each IO operation is mapped to a Fibre Channel Exchange through SCSI_FCP. Five FCP IUs are implemented in the this module: FCP_CMND, FCP_XFER_RDY, FCP_DATA, FCP_RSP, and FCP_CONF. Each IU corresponds to a Fibre Channel Sequence. The picture below shows the mapping mechanism used by SimSANs for SCSI IO operations, where Data_Out refers to the operations (e.g., SCSI_WRITE, MODE_SELECT, etc.) that write data from Initiator to Target, and Data_In refers to the operations (e.g., SCSI_READ, MODE_SENSE, RPT_LUNS, etc.) that read data from Target to Initiator.  Note the picture shows how single-DATA_ACTION IO is executed. For multi-DATA_ACTION IO, multiple FCP_DATA are executed for DATA_IN operation and multiple FCP_XFER_RDY/FCP_DATA  are executed for DATA_OUT operation.

 

SCSI FCP Mapping

Click here to see its VML version (requiring IE)

 

One of the most important FCP features that should be implemented is the FCP error detection and recovery (FCP_EDR), which is introduced by FCP-2. A simplified version of FCP_EDR is designed and deployed in SimSANs - SubIO Error Detection and Recovery (SIO_EDR), which is not implemented as a function of FCP module, instead, implemented as a part of SCSI Layer.  Moving FCP_EDR from FCP module to SCSI module greatly reduces the complexity to simulate the EDR mechanism while still maintains simulation accuracy at the same time.  SCSI Module tells more about SIO_EDR topic.

 

Each FCP is associated with one EE_Pair that has created a Login relationship between local N_Port and a remote N_Port.  You can consider the FCP Pair as a Process Login (PLGI) pair. SimSANs does not implement an explicit PLGI, instead, an implicit PLGI is performed once the PLOGI is completed between an N_Port pair.  Note EE_Pair.0 has no associated FCP module since it is reserved for the operations originated by non-FCP modules. Similar to ESB, an FCP IOSB is created and maintained for each open FCP IO.

 

FC-4 Controller

This module is a transportation junction of SCSI Commands (between SCSI and FC-4), FC-4 IUs (between FC-2 and FC-4), and Switch Controlling messages (between Switch Control Centre and FC-4). Separate FC-4 Controller is designed for N_Port (in Host or CHIP) and E/F_Port (in Switch). For N_Port, the FC-4 Controller is a software driver, consuming CPU (Host CPU or CHIP CPU) resources. Four timing elements are designed: FCP_IU_SEND and FCP_IU_RECV for FCP modules, and FC4_IU_SEND and FC4_IU_RECV for non-FCP modules (FC-ELS and FC-GS). For E_Port/F_Port, the FC-4 Controller is a hardware processor,  actually consuming FC-2 Controller resources.  Please refer to Timing Elements about their definitions and configurations. 

 

 

This page was last updated 2003.10.15