Host

Host Module

Host module is composed of one or more (usually two) N_Ports, one or more N_Port driver (FC-4) associated with each port, Host Peripheral Bus Controller (HPC), CPU, SCSI, and IO Generator. In SimSANs, each Host must connect with Switch(es) through the N_Port <--> F_Port link(s). FC-FS allows the N_Port <--> N_Port point-to-point connection but SimSANs does not support this feature, which means that Host cannot has direct connection(s) with Devices (e.g., Disk Arrays). Note the SCSI module in here is similar to the Application Client defined in SAM-2 (the Logical Unit Device Server and Task Manager are within LUN Controller module).

Mouse over the boxes to view specific modules.

Host Module

Click here to see its VML version (requiring IE)

N_Port

N_Port module plays the role of Fibre Channel HBA, containing FC-1 and FC-2 layers. Detailed module structures about FC-1 and FC-2 can be found in Fibre Channel Layers.

Mouse over the boxes to view specific modules.

N_Port Module

Click here to see its VML version (requiring IE)

N_Port Driver

Each N_Port has a driver associated with it. In fact the driver is just FC-4 Layer module.

Host Peripheral Bus Controller

HPC creates a dedicated data channel between HBA and its driver (Note: the driver is a software module loaded into memory) so that the data transfer between them does not cost CPU resources. An example of HPC is PCI bus controller. Both HPC and CPU must contend to gain the privilege of accessing the memory. A memory bus controller should exist to handle the contention but SimSANs does not explicitly implement such a module, instead, user can implicitly add this feature by customizing the CPU timing elements.

CPU

SCSI and FC-4 need to consume the CPU resources. Nine timing elements are defined: four of them (mentioned in FC-4 Controller) for FC-4 modules and the rest five for SCSI module, which are: CRE_IT_PATH, RMV_IT_PATH, INI_SCSI_IO, INI_SCSI_SIO, COM_SCSI_IO. Please refer to Timing Elements about their definitions and configurations.

SCSI Layer

SCSI Layer is designed according to SAM-2 standard. For Host the SCSI module is taking the role of Initiator. Several SCSI commands has been implemented: REPORT LUNS, MODE SENSE, and INQUIRY from SPC-2, and READ and WRITE from SBC-2. Besides SCSI commands initiation, this module also performs the following tasks: Initiator-Target Path creation and removal, LUN Masking, and IO Error Detection and Recovery, etc. The concept of "SubIO" is introduced for simplifying the implementation of SCSI module. For each open SCSI IO, an SCSI_IOSB is created and maintained for recording and monitoring the IO's operational state.

Initiator-Target Path is associated with a login N_Port pair, one is Initiator N_Port and the other is Target N_Port. Following a successful PLOGI with the Target, the Initiator starts the procedure of Initiator-Target Path creation by performing the following steps: (1) issues REPORT LUNS command to the Target LUN0 to get a list of logical units (LUNs); (2) issues MODE SENSE command to each of the reported LUN to get its NUM_OF_BLOCKS and BLOCK_LENGTH parameters (through the block descriptor defined in Mode parameters of SPC-2); (3) issues INQUIRY command to each reported LUN to obtain the logic unit's WWN (World Wide Name). Acquiring LUN WWN enables the SCSI Layer to identify possible redundant paths to a logic unit. SimSANs introduces the concept of "LUN PATH" that identifies an Initiator-Target Path through which a LUN WWN is accessible. For a given LUN, redundant paths exist if multiple LUN PATHs are identified for that LUN. At any one time, only one LUN PATH is active for IO operations. Initiator-Target Path should be removed when PLOUT is performed, in SimSANs, which usually indicates either Initiator or Target N_Port fails.

LUN Masking enables the N_Port to mask any LUNs that this N_Port should has no access to. LUN Masking is one of the two security mechanisms (the other is WWN zoning mentioned in Switch Control Centre) implemented in the SimSANs. Before running a SimSANs application, user should setup the LUN masking parameters. How-To section tells more about LUN masking configuration. Besides providing security, LUN masking also enables the SCSI Layer to distribute its IOs among multiple HBAs to achieve load balancing. For example, suppose a Host has two HBAs installed and two LUNs (LUN1 and LUN2) are identified accessible through both HBAs. Usually without LUN masking IO operations to both LUN1 and LUN2 will travel through one of the HBAs, but by setting up LUN masking, we can let, say, LUN1 identified by one HBA and LUN2 identified by the other HBA thus the IO traffic to both LUNs are distributed among HBAs.

SubIO does not appear in either Fibre Channel standards or SCSI standards, instead it is introduced and used only in the SimSANs. SubIO is defined as a segmentation of a SCSI IO. Each SubIO is just a single-DATA_ACTION IO, to which the standard SCSI protocol is applied, i.e., a SCSI CDB (Command Descriptor Block) is created for each SubIO not for the parent SCSI IO. SimSANs allows a maximum Sequence size of 64KB, which means that for a given SCSI IO with request size large than 64KB, multiple SubIOs are presented. Let me show you an example to make the SubIO concept clearer:

SubIO Example:

Suppose a SCSI WRITE IO is generated by IO Generator with request size of 96KB. Without implementing SubIO, this SCSI IO operation should execute multi-DATA_ACTION as follows: [1] Initiator transmits SEND_SCSI_CMD (FCP_CMND) to Target; [2] Target transmits RECV_DATA_OUT_REQ (FCP_XFER_RDY) telling Initiator that it is ready to receive the 1st 64KB data (remember maximum Sequence size is 64K); [3] Initiator then performs the 1st 64KB DATA_OUT Action (FCP_DATA); [4] Once Target is confirmed to get this data it continues requesting the next 32KB data by transmitting the second RECV_DATA_OUT_REQ (FCP_XFER_RDY) to Initiator; [5] Initiator performs the 2nd 32KB DATA_OUT Action (FCP_DATA); [6] Target completes receiving data and transmits SEND_CMD_COMP (FCP_RSP) to conclude the whole IO operation because it has just finished the last data action. By implementing SubIO, this SCSI WRITE IO is fulfilled by executing two single-DATA_ACTION WRITE IOs, one with a request size of 64KB and the other with a request size of 32KB. Both IOs are independently executed and concluded with separate SEND_CMD_COMP (FCP_RSP).

The SCSI_FCP shows the diagram about how the single-DATA_ACTION works. Of all the five SCSI commands used by SimSANs, REPORT LUNS, MODE_SENSE, and INQUIRY do not request an IO size larger than 64KB (Note SPC-2 does not limit the request sizes to 64KB for these commands), so they themselves are single-DATA_ACTION IOs, which means that even without the implementation of SubIO these commands are executed and concluded with one data action. The rest two commands READ and WRITE use SubIO to execute read and write operations with request sizes larger than 64KB. For each active SubIO, a SubIOSB is maintained and allocated under its parent - the SCSI_IOSB (mentioned before). In SimSANs, SIMPLE task attribute is used to be associated with each CDB. Other task attributes are not available - future version may consider HEAD OF QUEUE and ORDERED attributes.

Error Detection and Discovery is designed on a per-SubIO basis, which I name it SIO_EDR - SubIO Error Detection and Recovery. The SIO_EDR is a simplified version of FCP IU Error Detection and Recovery (FCPIU_EDR - please refer to FCP-2). Instead of performing error detection and recovery for each FCP IU, the SIO_EDR operates on SubIO level by using REC_TOV timer (PLOGI talks about REC_TOV setup). SIO_EDR works very simple: [1] When an SubIO is started its SubIOSB is allocated and REC_TOV timer starts; [2] If the SubIO is not concluded when the timer expires, that SubIo is re-scheduled and its SubIOSB is reset and the timer restarts; [3] Parameter MAX_SEQ_TRY (refer to System Configuration) is setup to define the maximum attempts of re-scheduling a SubIO. If MAX_SEQ_TRY is exceeded, the SubIO is aborted. The following diagram shows where to start and stop REC_TOV timer for SCSI Initiator and Target (SCSI Target is discussed in LUN Controller).

REC_TOV Timer

Click here to see its VML version (requiring IE)

IO and SubIO Re-scheduling and Abortion:

SCSI Initiator re-schedules or aborts IOs to response to the following two events:

[1] Initiator-Target Path Unavailable: This happens when N_Port (either Initiator or Target) fails. In this situation, the IOs (and their SubIOs) that have been opened (but not completed) on the failed path will be re-scheduled onto other available paths. If a re-scheduled IO cannot find available LUN PATHs, that IO is aborted.

[2] SubIO Time-out: Usually this happens when a Frame is missing or encountering errors during its transmission through the Switch Fabric. In this situation, Initiator re-schedules this SubIO. If MAX_SIO_TRY is exceeded, that SubIO is aborted. Note if any of its SubIOs is aborted, the parent SCSI IO is aborted even other SubIOs are completed.

IO Generator

IO Generator is used to generate SCSI READ and WRITE IO streams. Each stream represents a certain IO workload pattern, which is the combination of the following parameters: R/W Ratio, Request Size, LUN Block Address, and IO Gap (interval between consecutive IOs). User is able to take use of 20 pre-defined distribution functions (refer to OMNeT++ User Manual) to generate the random numbers for each of the four parameters. Besides those pre-defined functions, SimSANs also add another four functions for user to choose: const_size, seq_block, const_gap, and self_sim. "const_size" generates constant request size for each IO within an IO stream. "seq_block" generates sequential block IOs. "const_gap" generates IOs in a constant frequency. "self_sim" generates self-similar IO traffic using Mersenne Twister random number generator. User can setup as many IO streams as possible to generate complicate synthetic workload. How-To and Samples have more details about how to configure IO generation.

This page was last updated 2003.10.15