FCP Adapter/Device Interface

Kernel Extensions and Device Support Programming Concepts

FCP Adapter/Device Interface

The AIX FCP adapter device driver does not contain the ddread and ddwrite entry points, but does contain the ddconfig, ddopen, ddclose, dddump, and ddioctl entry points.

Therefore, the adapter device driver's entry in the kernel devsw table contains only those entries plus an additional ddstrategy entry point. This ddstrategy routine is the path that the FCP device driver uses to pass commands to the device driver. Access to these entry points is possible through the following kernel services:

The FCP adapter is accessed by the device driver through the /dev/fscsi# special files, where # indicates ascending numbers 0,1, 2, and so on. The adapter is designed so that multiple devices on the same adapter can be accessed at the same time.

For additional information on spanned and gathered write commands, see Understanding the Execution of Initiator I/O Requests.

scsi_buf Structure

The I/O requests made from the FCP device driver to the FCP adapter device driver are completed through the use of the scsi_buf structure, which is defined in the /usr/include/sys/scsi_buf.h header file. This structure, which is similar to the buf structure in other drivers, is passed between the two FCP subsystem drivers through the strategy routine. The following is a brief description of the fields contained in the scsi_buf structure:

Reserved fields should be set to a value of 0, except where noted.
The bufstruct field contains a copy of the standard buf buffer structure that documents the I/O request. Included in this structure, for example, are the buffer address, byte count, and transfer direction. The b_work field in the buf structure is reserved for use by the FCP adapter device driver. The current definition of the buf structure is in the /usr/include/sys/buf.h include file.
The bp field points to the original buffer structure received by the FCP Device Driver from the caller, if any. This can be a chain of entries in the case of spanned transfers (FCP commands that transfer data from or to more than one system-memory buffer). A null pointer indicates a nonspanned transfer. The null value specifically tells the FCP adapter device driver that all the information needed to perform the DMA data transfer is contained in the bufstruct fields of the scsi_buf structure.
The scsi_command field, defined as a scsi_cmd structure, contains, for example, the SCSI command length, SCSI command, and a flag variable:
1. The scsi_length field is the number of bytes in the actual SCSI command. This is normally 6,10,12, or 16 (decimal).
2. The FCP_flags field contains the following bit flags:
  
  SC_NODISC
  
  Do not allow the target to disconnect during this command.
  
  SC_ASYNC
  
  Do not allow the adapter to negotiate for synchronous transfer to the FCP device.
  
  During normal use, the SC_NODISC bit should not be set. Setting this bit allows a device executing commands to monopolize the FCP transport layer. Sometimes it is desirable for a particular device to maintain control of the transport layer once it has successfully arbitrated for it; for instance, when this is the only device on the FCP transport layer or the only device that will be in use. For performance reasons, it may not be desirable to go through FCP selections again to save FCP transport layer overhead on each command.
  
  Also during normal use, the SC_ASYNC bit must not be set. It should be set only in cases where a previous command to the device ended in an unexpected FCP transport free condition. This condition is noted as SCSI_TRANSPORT_FAULT in the adapter_status field of the scsi_cmd structure. Since other errors may also result in the SCSI_TRANSPORT_FAULT flag being set, the SC_ASYNC bit should only be set on the last retry of the failed command.
3. The scsi_cdb structure contains the physical SCSI command block. The 6 to 16 bytes of a single SCSI command are stored in consecutive bytes, with the op code identified individually. The scsi_cdb structure contains the following fields:
  1. The scsi_op_code field specifies the standard FCP op code for this command.
  2. The scsi_bytes field contains the remaining command-unique bytes of the FCP command block. The actual number of bytes depends on the value in the scsi_op_code field.
The timeout_value field specifies the time-out limit (in seconds) to be used for completion of this command. A time-out value of 0 means no time-out is applied to this I/O request.
The status_validity field contains an output parameter that can have one of the following bit flags as a value:

SC_SCSI_ERROR

The scsi_status field is valid.

SC_ADAPTER_ERROR

The adapter_status field is valid.
The scsi_status field in the scsi_buf structure is an output parameter that provides valid FCP command completion status when its status_validity bit is nonzero. The scsi_buf.bufstruct.b_error field should be set to EIO anytime the scsi_status field is valid. Typical status values include:

SC_GOOD_STATUS

The target successfully completed the command.

SC_CHECK_CONDITION

The target is reporting an error, exception, or other conditions.

SC_BUSY_STATUS

The target is currently transporting and cannot accept a command now.

SC_RESERVATION_CONFLICT

The target is reserved by another initiator and cannot be accessed.

SC_COMMAND_TERMINATED

The target terminated this command after receiving a terminate I/O process message from the FCP adapter.

SC_QUEUE_FULL

The target's command queue is full, so this command is returned.

SC_ACA_ACTIVE

The FCP device has an ACA (auto contingent allegiance) condition that requires a Clear ACA to request to clear it.
The adapter_status field is an output parameter that is valid when its status_validity bit is nonzero. The scsi_buf.bufstruct.b_erro field should be set to EIO anytime the adapter_status field is valid. This field contains generic FCP adapter card status. It is intentionally general in coverage so that it can report error status from any typical FCP adapter.
If an error is detected during execution of a FCP command, and the error prevented the FCP command from actually being sent to the FCP transport layer by the adapter, then the error should be processed or recovered, or both, by the FCP adapter device driver.

If it is recovered successfully by the FCP adapter device driver, the error is logged, as appropriate, but is not reflected in the adapter_status byte. If the error cannot be recovered by the FCP adapter device driver, the appropriate adapter_status bit is set and the scsi_buf structure is returned to the FCP device driver for further processing.

If an error is detected after the command was actually sent to the FCP device, then it should be processed or recovered, or both, by the FCP device driver.

For error logging, the FCP adapter device driver logs FCP transport layer and adapter-related conditions, while the FCP device driver logs FCP device-related errors. In the following description, a capital letter (A) after the error name indicates that the FCP adapter device driver handles error logging. A capital letter (H) indicates that the FCP device driver handles error logging.

Some of the following error conditions indicate a FCP device failure. Others are FCP transport layer or adapter-related.

SCSI_HOST_IO_BUS_ERR (A)

The system I/O transport layer generated or detected an error during a DMA or Programmed I/O (PIO) transfer.

SCSI_TRANSPORT_FAULT (H)

The FCP transport protocol or hardware was unsuccessful.

SCSI_CMD_TIMEOUT (H)

The command timed out before completion.

SCSI_NO_DEVICE_RESPONSE (H)

The target device did not respond to selection phase.

SCSI_ADAPTER_HDW_FAILURE (A)

The adapter indicated an onboard hardware failure.

SCSI_ADAPTER_SFW_FAILURE (A)

The adapter indicated microcode failure.

SCSI_FUSE_OR_TERMINAL_PWR (A)

The adapter indicated a blown terminator fuse or bad termination.

SCSI_TRANSPORT_RESET (A)

The adapter indicated the FCP transport layer has been reset.

SCSI_WW_NAME_CHANGE (A)

The adapter indicated the device at this SCSI ID has a new FCS world wide name.
The add_status field contains additional device status. For FCP devices, this field contains the FCP Response code returned.
When the FCP device driver queues multiple transactions to a device, the adap_q_status field indicates whether or not the FCP adapter driver has cleared its queue for this device after an error has occurred. The flag of SC_DID_NOT CLEAR_Q indicates that the FCP adapter driver has not cleared its queue for this device and that it is in a halted state (so none of the pending queued transactions are sent to the device).
The q_tag_msg field indicates if the FCP adapter can attempt to queue this transaction to the device. This information causes the FCP adapter to fill in the Queue Tag Message Code of the queue tag message for a FCP command. The following values are valid for this field:

SC_NO_Q

Specifies that the FCP adapter does not send a queue tag message for this command, and so the device does not allow more than one FCP command on its command queue. This value must be used for all commands sent to FCP devices that do not support command tag queuing.

SC_SIMPLE_Q

Specifies placing this command in the device's command queue. The device determines the order that it executes commands in its queue. The SCSI-2 specification calls this value the Simple Queue Tag Message.

SC_HEAD_OF_Q

Specifies placing this command first in the device's command queue. This command does not preempt an active command at the device, but it is executed before all other commands in the command queue. The SCSI-2 specification calls this value the Head of Queue Tag Message.

SC_ORDERED_Q

Specifies placing this command in the device's command queue. The device processes these commands in the order that they are received. The SCSI-2 specification calls this value the Ordered Queue Tag Message.

SC_ACA_Q

Specifies placing this command in the device's command queue, when the device has an ACA (auto contingent allegiance) condition. The SCSI-3 Architecture Model calls this value the ACA task attribute.

Note: Commands with the value of SC_NO_Q for the q_tag_msg field (except for request sense commands) should not be queued to a device whose queue contains a command with another value for q_tag_msg. If commands with the SC_NO_Q value (except for request sense) are sent to the device, then the FCP device driver must make sure that no active commands are using different values for q_tag_ms. Similarly, the FCP device driver must also make sure that a command with a q_tag_msg value of SC_ORDERED_Q, SC_HEAD_Q, or SC_SIMPLE_Q is not sent to a device that has a command with the q_tag_msg field of SC_NO_Q.
The flags field contains bit flags sent from the FCP device driver to the FCP adapter device driver. The following flags are defined:

SC_RESUME

When set, means the FCP adapter device driver should resume transaction queuing for this ID/LUN. Error recovery is complete after a SCIOLHALT operation, check condition, or severe FCP transport error. This flag is used to restart the FCP adapter device driver following a reported error.

SC_DELAY_CMD

When set, means the FCP adapter device driver should delay sending this command (following a FCP reset or BDR to this device) by at least the number of seconds specified to the FCP adapter device driver in its configuration information. For FCP devices that do not require this function, this flag should not be set.

SC_Q_CLR

When set, means the FCP adapter driver should clear its transaction queue for this ID/LUN. The transaction containing this flag setting does not require an actual FCP command in the scsi_buf because it is flushed back to the FCP device driver with the rest of the transactions for this ID/LUN. However, this transaction must have the SCSI ID field (scsi_buf.scsi_id) and the LUN field (scsi_buf.lun_id) filled in with the device's SCSI ID and logical unit number (LUN). This flag is valid only during error recovery of a check condition or command terminated at a command tag queuing device when the SC_DID_NOT_CLR_Q flag is set in the scsi_buf.adap_q_status field.

SC_Q_RESUME

When set, means that the FCP adapter driver should resume its halted transaction queue for this ID/LUN. The transaction containing this flag setting does not require an actual FCP command to be sent to the FCP adapter driver. However, this transaction must have the SCSI ID field (scsi_buf.scsi_id) and the LUN field (scsi_buf.lun_id) filled in with the device's SCSI ID and logical unit number (LUN). If the transaction containing this flag setting is the first issued by the FCP device driver after it receives an error (indicating that the adapter driver's queue is halted), then the SC_RESUME flag must be set also.

SC_CLEAR_ACA

When set, means the SCSI adapter driver should issue a Clear ACA task management request for this ID/LUN. This flag should be used in conjunction with either the SC_Q_CLEAR or SC_Q_RESUME flags to clear or resume the SCSI adapter driver's queue for this device. If neither of these flags is used, then this transaction is treated as if the SC_Q_RESUME flag is also set. The transaction containing the SC_CLEAR_ACA flag setting does not require an actual SCSI command in the sc_buf. If this transaction contains a SCSI command then it will be processed depending on whether SC_Q_CLR or SC_Q_RESUME is set. This transaction must have the SCSI ID field (scsi_buf.scsi_id) and the LUN field (scsi_buf.lun_id) filled in with the device's SCSI ID and logical unit number (LUN). This flag is valid only during error recovery of a check condition or command terminated at a command tag queuing.
The dev_flags field contains additional values sent from the FCP device driver to the FCP adapter device driver. The following values are defined:

FC_CLASS1

When set, this tells the SCSI adapter driver that it should issue this request as a Fibre Channel Class 1 request. If the SCSI adapter driver does not support this class, then it will fail the scsi_buf with an error of EINVAL. If no Fibre Channel Class is specified in the scsi_buf then the SCSI adapter will default to a Fibre Channel Class.

FC_CLASS2

When set, this tells the SCSI adapter driver that it should issue this request as a Fibre Channel Class 2 request. If the SCSI adapter driver does not support this class, then it will fail the scsi_buf with an error of EINVAL. If no Fibre Channel Class is specified in the scsi_buf then the SCSI adapter will default to a Fibre Channel Class.

FC_CLASS3

When set, this tells the SCSI adapter driver that it should issue this request as a Fibre Channel Class 3 request. If the SCSI adapter driver does not support this class, then it will fail the scsi_buf with an error of EINVAL. If no Fibre Channel Class is specified in the scsi_buf then the SCSI adapter will default to a Fibre Channel Class.

FC_CLASS4

When set, this tells the SCSI adapter driver that it should issue this request as a Fibre Channel Class 4 request. If the SCSI adapter driver does not support this class, then it will fail the scsi_buf with an error of EINVAL. If no Fibre Channel Class is specified in the scsi_buf then the SCSI adapter will default to a Fibre Channel Class.
The add_work field is reserved for use by the FCP adapter device driver.
The adap_set_flags field contains an output parameter that can have one of the following bit flags as a value:

SC_AUTOSENSE_DATA_VALID

Autosense data was placed in the autosense buffer referenced by the autosense_buffer_ptr field.
The autosense_length field contains the length in bytes of the SCSI device driver's sense buffer, which is referenced via the autosense_buffer_ptr field. For FCP devices this field must be non-zero, otherwise the autosense data will be lost.
The autosense_buffer_ptr field contains the address of the SCSI devices driver's autosense buffer for this command. For FCP devices this field must be non-NULL, otherwise the autosense data will be lost.
The dev_burst_len field contains the burst size if this write operation in bytes. This should only be set by the FCP device driver if it has negotiated with the device and it allows burst of write data without transfer readys. For most operations, this should be set to 0.
The scsi_id field contains the 64-bit SCSI ID for this device. This field must be set for FCP devices.
The lun_id field contains the 64-bit lun ID for this device. This field must be set for FCP devices.

Adapter/Device Driver Intercommunication

In a typical request to the device driver, a call is first made to the device driver's strategy routine, which takes care of any necessary queuing. The device driver's strategy routine then calls the device driver's start routine, which fills in the scsi_buf structure and calls the adapter driver's strategy routine through the devstrat kernel service.

The adapter driver's strategy routine validates all of the information contained in the scsi_buf structure and also performs any necessary queuing of the transaction request. If no queuing is necessary, the adapter driver's start subroutine is called.

When an interrupt occurs, the FCP adapter interrupt routine fills in the status_validity field and the appropriate scsi_status or adapter_status field of the scsi_buf structure. The bufstruct.b_resid field is also filled in with the value of nontransferred bytes. The adapter driver's interrupt routine then passes this newly filled in scsi_buf structure to the iodone kernel service which then signals the FCP device driver's iodone subroutine. The adapter driver's start routine is also called from the interrupt routine to process any additional transactions on the queue.

The device driver's iodone routine should then process all of the applicable fields in the queued scsi_buf structure for any errors and attempt error recovery if necessary. The device driver should then dequeue the scsi_buf structure and then pass a pointer to the structure back to the iodone kernel service so that it can notify the originator of the request.

Related Information

Specific information about the FCP adapter device driver routines is available in FCP Adapter Device Driver Routines.

An introduction to the FCP subsystem is provided by Programming FCP Device Drivers.