An Efficient Way of iSCSI based Remote Storage Service for Mobie Device using Intermediate Target
Daegeun Kim
Internet Computing Lab.
Korea University,
An-Am Dong, Sung-Buk Gu, Seoul, 136-701, Korea
+82-2-927-3797
vicroot@
ilab.korea.ac.kr
Soo-Mok Jung
Internet Computing Lab.
Korea University
Myong-Soon Park
Internet Computing Lab.
Korea University
myongsp@
ilab.korea.ac.kr
Abstract
As mobile device prevails, requests about various services using mobile device have increased. Specially, requests of application services that need large data space such as multimedia, game, database [1] greatly increased. However, mobile appliance has difficulty in applying various services like wire environment because the storage capacity of the one is not enough. Therefore, researches [5] which provide remote storage service for mobile appliance using iSCSI are being progressed to overcome storage space limitation of mobile appliance.
But, when iSCSI is applied for Mobile appliance, iSCSI I/O performance drops rapidly if iSCSI client moves from server to far away position. It is because iSCSI has structural quality to be very sensitive at delay time.
In this paper, we introduce intermediate target server and localize iSCSI target to improve the shortcoming that iSCSI performance drops sharply according as latency increases when mobile appliance recedes from storage server.
Keywords
iSCSI, SCSI, Mobile Device, Remote Storage, Block Level I/O
1. Introduction
The explosive growth of mobile appliance market has made a lot of demands in mobile-related services. Effort to apply wired network environment services which need large amount of storage space such as multimedia, databases to mobile appliance with wireless network environment was performed. However, mobile appliance should be small and light to support mobility, so that it uses small size flash memory instead of hard-disk having large data space. In the case of PDA, it usually has memory space of 32 ~ 64M and cell-phone or smart phone permit smaller memory space. Therefore, there was difficult in saving multimedia data such as mpg, mp3, etc. and installing large size software such as database engine. Limited storage space of Mobile appliance has been a barrier for applying various services in wired environment. As the result, necessity of remote storage service for mobile appliance rose to overcome limited storage space of Mobile device and store large amount of data or provide various application service [5] [8] [9].
Remote storage service is classified by two, one is file-level I/O and the other is block-level I/O. File-level I/O service is provided by communication between server and client file systems. In the case of file I/O, because client’s I/O request is passed to device via server file system, data share is possible between other clients through locking mechanism which server file system offers, but file system overhead drops I/O performance.
However, in block-level service, client’s I/O request is directly passed to storage device without server file system in the form of block I/O command. So, it is not able to provide data sharing service by itself, but I/O performance is better than the one of file-base I/O service [9].
The bandwidth of wireless network for mobility, which mobile device use, is usually lower than the one of wired network. If the purpose of remote storage service is to extend individual storage space for mobile appliance without data sharing, block-level I/O service is more suitable. iSCSI is a standard protocol that transports SCSI command which is a representative block I/O through TCP/IP network. However, iSCSI has a problem that I/O performance drops sharply if network latency increases between iSCSI initiator and target.
In this paper, when iSCSI-based remote storage service applies to mobile appliance, we improve the problem that iSCSI performance is fallen rapidly according as mobile client recedes from storage server and network latency increases. This paper comprises as the follow. In chapter III, we present SCSI architecture model and iSCSI basic operation to describe the reason that iSCSI performance drops when distance between iSCSI initiator and target is long. In chapter IV, we explain a research related with improving iSCSI performance using client’s local cache. In chapter V, we are going to explain the proposed way to improve iSCSI performance using intermediate server, and we talk about system architecture and algorithm in chapter VI. In chapter VII, we analyze the result of simulation with ns2. And at last we conclude it and describe our future works.
2. Background
In this chapter, we describe SCSI architecture model-3 and iSCSI basic operation process to explain the relation of iSCSI Initiator-Target's distance and performance, and then analyze iSCSI performance test result by different delay time.
3.1 SAM-3 ( SCSI Architecture Model - 3 )
SAM-3 is a draft for SCSI standard leaded by T10, a Technical Committee of Accredited Standards Committee INCITS (InterNational Committee for Information Technology Standards). SAM-3 specifies the SCSI Architecture Model. The purpose of the architecture is to provide a common basis for the coordination of SCSI standards and to specify those aspects of SCSI I/O system behavior that are independent of a particular technology and common to all implementations[7].
3.1.1 Linked SCSI Command Processing
Serial SCSI Commands are processed sequentially in SAM-3[7]. Only after previous command processing is completed, next command is processed. This sequential processing mechanism cause that SCSI protocol is sensitive to network latency.
Figure 1 shows the sequence in which two serial SCSI commands are processed between SCSI initiator and target. SCSI initiator has a waiting time from sending created command to target until receiving the result. If the distance of initiator and target is longer, initiator’s waiting time is larger because electronic wave delay of the network is grown. Even if the change of waiting time about one SCSI command is small, subsequent all commands will be delayed by previous response latency because SCSI command process is sequential. If there are n serial SCSI commands to be processed, time lag of n x dt (time lag of response delay per one SCSI command) occur until last command is processed.
Figure 1. Linked SCSI command process
As a result, when the distance between SCSI Initiator and Target is long in some degree, response time of a SCSI command is delayed and link utilization of SCSI protocol dropped causing degradation of I/O performance. Even in the case of iSCSI which embody SCSI protocol over TCP/IP network, described feature of SCSI makes iSCSI performance limited by the distance of initiator and target, although iSCSI has advantage that distance of Initiator and Target can become overlong than SCSI cable or fiber channel.
3.2 iSCSI
The iSCSI (Internet Small Computer System Interface) is an emerging standard storage protocol that can transfer SCSI command over IP network[5]. Since the iSCSI protocol can make client access the SCSI I/O devices of server host over an IP Network, client can use the storage of another host transparently without the need to pass server host's file system[6].
In iSCSI layer which is on top of TCP layer, common SCSI commands and data are encapsulated in the form of iSCSI PDU(Protocol Data Unit). The iSCSI PDU is sent to TCP layer for the IP network transport. Through the procedure, client who wants to use storage of the remote host can use it. Because the encapsulation and the decapsulation of SCSI I/O commands over TCP/IP enable storage user to access remote storage device of the remote host directly [3].
Likewise, if we build remote storage system for mobile appliance using iSCSI protocol, mobile clients can use the storage of server host directly like their own local storage. It enables mobile appliance to overcome the limitation of storage capacity as well as to adapt various application services of wired environment in need of mass scale data. Different from the traditional remote storage system that based on file unit I/O, iSCSI protocol provides block unit I/O. So it can make more efficient transmission throughput than the traditional remote storage system like CIFS and NFS does.
Another characteristic of iSCSI protocol is that it operates on standard and commonly used network component like Ethernet. Since iSCSI protocol directly can be plugged in Ethernet environment, it is easier to manage than another storage data transmission protocol as Fibre Channel. Moreover, iSCSI can reduce costs to build storage system due to the use of infra network without additional adjustment. The iSCSI protocol
was defined as the standard SCSI transmission protocol lately by IETF and a lot of related researches are being processed with the development of Gigabit Ethernet technology.
3.2.1 iSCSI Basic Operation
Figure 2 shows exchanged control and data packets’ sequence in Read and Write Operation of iSCSI protocol [10].
Figure 2. iSCSI basic operation
Like figure 2, iSCSI exchanges control packets (SCSI Command, Ready to Transmit, SCSI Response) and data packet (SCSI Data) to process one R/W operation. Three control packets and one data packet are used in write operation and spend 2 x RTT. It takes 2 control packets and 1 x RTT in case of to read operation. Because control packet including header information is no more than 48 byte, bandwidth waste becomes serious when initiator recedes a little from target. Therefore, distance of iSCSI initiator and target, and network latency influence iSCSI I/O performance.
3.2.2 ISCSI performance change by network latency
Figure 3 show storage system constructed in University of Minnesota for performance experiment of iSCSI based storage service in different network environment [6]. They analyzed the performance change and feature of iSCSI when storage is accessed in Gigbit Ethernet(LAN) and through Campus Network(WAN).
Figure 3. iSCSI performance experiment environment
Table 1. Change of delay by network and data size
Read(ms)Configuration / 1KB / 4KB / 16KB / 64KB
GigE LAN iSCSI / 9.57 / 9.7 / 10.91 / 15.82
Campus LAN iSCSI / 11.37 / 12.45 / 14.33 / 24.2
Write(ms)
Configuration / 1KB / 4KB / 16KB / 64KB
GigE LAN iSCSI / 10.08 / 10.35 / 11.24 / 15.0
Campus LAN iSCSI / 12.8 / 12.5 / 14.62 / 22.98
Through the experiment result in Table 1, we can know that the latency by network distance and bandwidth between Initiator and Target influences iSCSI performance. ISCSI shows similar performance in small Gigabit Ethernet environment such as when disk is directly accessed through fiber channel, but shows low performance in Campus LAN (WAN) environment in which network traffic is liable to variation and latency is big. That is why delay time affects iSCSI's I/O performance a lot. Specially, in case data size is large, transmission delay influenced by network bandwidth is much bigger than other latency, and in case data size is small, propagation delay influenced by distance of physical network is main part of iSCSI latency. In the case of mobile appliance, which usually run I/O by small size block, delay by the distance of physical network that packet move is more important factor degrading iSCSi performance than the delay by bandwidth.
3. Related Work
The way using cache to improve disk I/O performance has been already studied already for a long time. Usually, researches, that use part of memory space as buffer cache to improve performance of local disk, were most cases. However, researches that use local or network cache to improve iSCSI's performance as iSCSI has been a standard recently and products supporting iSCSI appear.
3.1 iCache
iCache is a research to improve iSCSI performance using local cache of client system. Initiator’s system has specific cache space for iSCSI data and iSCSI block data is cached to minimize network block I/O. Therefore, iSCSI does not send I/O request through network every time when disk I/O happens, instead read cached block or send blocks cached in LogDisk at once to server for improving iSCSI performance. iCache's buffer space consists of two hierarchical caches which are NVRAM and LogDisk. Data is stored sequentially in NVRAM and when enough data is gathered, iCache process moves data from NVRAM to LogDisk. Blocks which are frequently accessed, are kept in NVRAM of which access speed is fast and iCache stores less accessed data in logDisk. Also, destage operation is achieved by kernel thread called as LogDestage. In Destage Operation, NVRAM’s data is moved to LogDisk or LogDisk's data is sent to remote storage using iSCSI protocol.Cache techniques used in iCache are based on DCD technology [11] proposed to improve Disk I/O performance.
Figure 4. iCache architecture
It is difficult to apply iCache for mobile device which has lacking memory because iCache need additional memory and hard-disk space to embody local cache which consists of NVRAM and LogDisk.
4. Proposed Idea
In this chapter, we introduce the way localizing iSCSI target with intermediate server called intermediate target to solve the problem that iSCSI performance drops when iSCSI mobile client recedes from storage server. iSCSI response time for read operation is minimized with nearest intermediate target which prefetches read block for client to use or response beforehand packet to iSCSI initiator for written block.
In proposed architecture, we suppose following 3 to simplify the problem.
- Suppose distributed storage server environment. iSCSI intermediate target server spread over wide areas. mobile client is connected with nearest iSCSI intermediate target through iSNS (Internet Storage Name Server) [3]
- As transmission distance increases, propagation delay of physical media and sum of the queing delay of intermediate routers increases.
- ISCSI latency includes propagation delay by distance, queuing delay and transmission delay by bandwidth, and ignores processing delay of end nodes.
5.1. iSCSI Intermediate Target
In iSCSI based remote storage service for mobile node, when mobile node serviced in A area moves to C area, distance between iSCSI initiator and target is prolonged and packet transfer time increases. In chapter 3, we showed SCSI command is processed sequentially in SAM-3 [7] and iSCSI basic operation needs exchange of several control packets to process one command. Because small size control packet of iSCSI protocol influence iSCSI response time, if the distance between server and client is long, the link utilization drops sharply and iSCSI performance becomes low.