Chapter 5 Grid Network Services and Implications for Network Service Design Joe Mambr etti, Bill St. Arnaud, Tom DeFanti, Maxine Brown, and Kees Neggers 5.1 INTRODUCTION The initial chapters of this book introduce Grid attributes and discuss how those attributes are inherent in Grid architectural design. Those chapters describe the bene- fits of designing multiple resources within a Grid services framework as addressable modules to allow for versatile functionality. This approach can provide for both a suite of directly usable capabilities and also options for customization so that infras- tructure resources can be accessed and adjusted to match the precise requirements of applications and services. These chapters also note that until recently this multifaceted flexibility has not been extended to Grid networks. However, new methods and architectural standards are being created that are beginning to integrate network services into Grid environments and to allow for more versatility among network services. Chapter 3 explains that the SOA used for general Grid resources are also being used to abstract network capabilities from underlying infrastructure. This architecture can be expressed in Grid Networks: Enabling Grids with Advanced Communication Technology Franco Travostino, Joe Mambretti, Gigi Karmous-Edwards © 2006 John Wiley & Sons, Ltd 82 Chapter 5: Grid Network Services and Implications for Network Service Design intermediate software that can provide for significantly more capability, flexibility, and adjustability than is possible on today’s networks. This chapter presents additional topics related to the basic requirements and architectural design of Grid network services. The design of Grid network services architecture currently is still at its initial stages. The development of this architecture is being influenced by multiple considerations, including those related to technology innovation, operational requirements, resource utilization efficiencies, and the need to create fundamentally new capabilities. This chapter discusses some of the consid- erations related to that emerging design, including functional requirements, network process components, and network services integration. 5.2 TRADITIONAL COMMUNICATIONS SERVICES ARCHITECTURE Traditional architectures for communication services, network infrastructure, and exchange facilities have been based on designs that were created to optimize network resources for the delivery of analog-based services, based on a foundation of core transport services. Such network infrastructure supported only a limited range of precisely defined services with a small, fixed set of attributes. The services have been fairly static because they have been based on an inflexible infrastructure, which usually required changes through physical provisioning. Such networks have also been managed through centralized, layered systems. Traditional communications models assume that services will be deployed on a fixed hierarchical stack of layered resources, within an opaque carrier cloud, through which “managed services” are provided. Providing new services, enhancing or expanding existing services, and customizing services is difficult, costly, and restrictive. Dedicated channel services, such as VPNs, are generally allowed only within single domains. Private, autonomous interconnections across domains are not possible. The quality of the services on these channels and their general attributes are not flexible and cannot be addressed or adjusted by external signaling. Today’s Internet is deployed primarily as an overlay network on this legacy infras- tructure. The Internet has made possible a level of abstraction that has led to a significantly more versatile communications services environment, and Grid network services are being designed to enhance the flexibility of that environment. 5.3 GRID ARCHITECTURE AS A SERVICE PLATFORM In contrast to traditional telecommunication services, Grid environments can be designed to provide an almost unlimited number of services. A Grid is a flexible infras- tructure that can be used to provide a single defined service, a set of defined services, or a range of capabilities or functions, from which it is possible for external entities to create their own services. In addition, within those environments, processes can request that basic infrastructure and topologies be changed dynamically – even as a continuous process. 5.3 Grid Architecture as a Service Platform 83 Just as the term “Grid” is analogous to the electric power system, a Grid service has been described as being somewhat analogous to the services provided by electrical utilities. Multiple devices can attach to the end of an electrical power grid, and they can use the basic services provided by that common infrastructure for different functions. However, the electrical power grid, like almost all previous infrastructure, has been designed, developed, and implemented specifically to provide a single defined service or a limited set of services. Previous chapters note that the majority of Grid services development initiatives have been oriented to applications, system processes, and computer and storage infrastructure – not to network services. Although network services have always been an essential part of Grid environments, they have been implemented as static, undif- ferentiated, and nondeterministic packet-routed services – rarely as reconfigurable, controllable, definable, deterministic services. Recently, research and development projects have been adapting Grid concepts to network resources, especially to techniques for services abstraction and virtualiza- tion. These methods are allowing network resources to be “full participants” within Grid environments – accessible, reconfigurable resources that can be fully integrated with other Grid resources. For example, with the advent of Grid Web Services described in Chapter 3, the constituent components of a network from the physical to the application layer can be represented as an abstraction layer that can fully interact with other Grid services on a peer-to-peer basis, rather than traditional hierarchical linkages in a stack as is now common with typical telecommunication applications. This approach represents a major new direction in network services provisioning, a fundamentally new way to create and implement such services. It does not merely provide a path to additional access to network services and methods of manipulating lower level resources functionality, it also provides an extensive toolkit that can be used to create complete suites of new networks services. 5.3.1 GRID NETWORK SERVICES ARCHITECTURE The Grid standards development community has adopted a general framework for a SOA based on emerging industry standards, described in Chapter 4. This architec- tural framework enables the efficient design and creation of Grid-based services by providing mechanisms to create and implement Grid service processes, comprising multiple modular processes that can be gathered and implemented into a new func- tioning service whose sum is greater than the parts. Such standards-based techniques can be used to create multiple extensible integrated Grid-based services, which can be easily expanded and enhanced over the services’ lifetime. In addition, this archi- tecture enables these modular services to be directly integrated to create new types of services. This architecture provides for several key components, which are described in Chapter 7. The higher level of service, and the highest level of services abstrac- tion, consists of capabilities or functions that are made available through advertise- ments through a standard, open communication process. These high-level processes interact with intermediate software components between that top layer and core 84 Chapter 5: Grid Network Services and Implications for Network Service Design facilities and resources. The core facilities and resources can consist of almost any information technology object, including any one of a wide array of network services and other resources. This infrastructure is currently being developed, and as it is implemented it is becoming clear that this new services approach will manifest itself in many forms. In some cases, organizations will focus on providing only end-delivered services, and rely on using Grid services provided by other organizations. In other cases, orga- nizations will focus on providing mid-level services to those types of organizations, while perhaps relying on Grid infrastructure providers for core resources. Other organizations may provide only basic infrastructure resources. However, this new model enables any organization to access and provide capabilities at any level. As a type of Grid service, individual network resources can become modular objects that could be exposed to any legitimate Grid process as an available, usable service. In general, these resources will probably be advertised to mid-level services rather than to edge processes, although that capability also remains an option. As part of a Grid services process or service workflow procedure, network resources can be directly integrated with any other type of Grid service, including those that are not network related. Consequently, multiple network resource objects, advertised as services, can be gathered, integrated, and utilized in virtually almost unlimited numbers of ways. They can be combined with other types of Grid objects in ad hoc integrated collections in order to create specialized communication services on demand. All resource elements become equal peers that can be directly addressable by Grid processes. Grid services-oriented architecture provides capabilities for external processes, on a peer-to-peer basis, to provision, manage, and control customized network services directly – without any artificial restrictions imposed from centralized networking management authorities, from server-based centralized controls, or from hierarchical layering. This design allows multiple disparate distributed resources to be utilized as equal peers, which can be advertised as available services that can be directly discovered, addressed, and used. This architecture allows different types of services, including highly specialized services, to co-exist within the same core, or foundation, infrastructure, even end-to-end across multiple domains. This approach significantly increases capabilities for creating and deploying new and enhanced services, while also ensuring cost effectiveness through infrastructure sharing. This approach can incorporate tradi- tional distributed management and control planes, e.g., as exposed resources within a services-oriented architecture, or it can completely eliminate traditional control and management functions. 5.4 NETWORK SERVICES ARCHITECTURE: AN OVERVIEW 5.4.1 SERVICES ARCHITECTURE BENEFITS Grid network services architecture provides multiple benefits. It supports a wider range of communication services, and it allows those services to have more attributes than traditional communication services. This architecture can be implemented 5.4 Network Services Architecture: An Overview 85 to expand services offerings, because basic individual resource elements can be combined in almost limitless ways. With the implementation of stateful services, or using workflow languages that maintain state, multiple network resources can be treated as individual components that can be used in any form or combination as required by external services and applications, thereby precisely matching applica- tion needs to available resources. A major advantage of this architecture is that it is more flexible and adaptive than traditional networks. This flexibility can be used to make communication services and networks more “intelligent,” for example by enabling an applications web service to be bound to a network web service, thereby enabling the combined service to be more “context aware.” Using this model, applications can even be directly integrated into network services, such that there is no distinction between the application and the network service. This architecture also provides integrated capabilities at all tradi- tional network layers, not just individual layers, eliminating dependencies on hierar- chical protocol stacks. Also, it provides for enhanced network scalability, including across domains, and for expandability, for example by allowing new services and technologies to be easily integrated into the network infrastructure. Processes external to the network can use these core component as resources in multiple varieties of configurations. Applications, users, infrastructure processes, and integrated services, all can be integrated with network service and resources in highly novel configurations. These external processes can even directly address core network resources, such as lightpaths and optical elements, which to date have not been accessible through traditional services. This approach does not simply provide access to lower level functionalities, but it also enables full integration of higher level services with those functionalities, in part by removing traditional concepts of hierarchical layers. As Chapter 3 indicates, the concept of layers and planes has been a useful abstrac- tion to classify sets of common network functions. The OSI layer model [1] depicted in Figure 3.2 is an artifact, designed to address such tasks as the limitations of buffering and of managing different types of telecommunication transport services. However, the Grid network services approach departs from the traditional vertical model of services provided through separate OSI network layers. The concept of a “stack” of layers from the physical through to the application largely disappears in the world of Grid services. This concept is also gaining acceptance by communica- tions standards bodies, as noted in Chapter 14, including the ITU, which produced a future directions document indicating that standard model may not be carried forward into future designs [2]. Although this architectural direction may engender some complexity for provisioning, it will also result in multiple benefits. Similarly, traditional network services incorporate concepts of management planes, control planes, and data planes, which are architectures that define specific, stan- dardized sets of compartmentalized functionality. Because Grid network services architecture includes basic definitions of the set of Grid network services functions, it would be possible to extend this approach to also incorporate a concept of a “Grid network services plane.” However, while convenient, this notion of a “plane” would obscure a fundamental premise behind the Grid network services architec- ture, which is being designed such that it is not limited to a set of functions within 86 Chapter 5: Grid Network Services and Implications for Network Service Design a traditionally defined “plane”; instead it provides a superset of all of these function- alities, incorporating all traditional functions within a broad standard shared-use set of capabilities. Another advantage of implementing Grid network resources within a SOA is that it provides for a transition path from traditional communications infrastructure. The enhanced levels of abstraction and virtualization provided through Grid network services architecture can be used as a migration path from limited legacy infrastruc- ture toward one that can offer a much wider and more powerful set of capabilities, from centrally managed processes with hierarchical controls to highly distributed processes. 5.5 GRID NETWORK SERVICES IMPLICATIONS Within a Grid network services environment, it is possible to accept either a prede- fined default service or to highly customize individualized network services. Network services, core components, specialized resources such as layer 3 services with customized attributes, dedicated layer 2 channels, reconfigurable cross-connections, and even lightpaths and individual physical network elements, such as ports, can be identified and partitioned into novel integrated services. These services can be provided with secure access mechanisms that enable organizations, individuals, communities, or applications to discover, interlink, and utilize these resources. For example, using this architecture, end-users and applications can provision end-to-end services, temporarily or permanently, at any individual level or at multiple levels. Because of these attributes, this architecture allows Grid network services to extend from the level of the communications infrastructure directly into the internal processes of other resources, such as computers, storage devices, or instruments. Using techniques based on this architecture, the network can also be extended directly into individual applications, allowing those applications to be closely inte- grated with network resources. Such integration techniques can be used to create novel communications-based services. The approach described here provides multiple advantages for Grid environments. However, even when used separately from Grid environments, this approach can be used to provide significantly more functionality, flexibility, and cost efficiency for digital communications services, and it can provide those benefits with much less complexity. These advantages are key objectives in the design of next generation digital communication services, and new architecture that provides for service level abstracts are important methods for achieving those goals. 5.6 GRID NETWORK SERVICES AND NETWORK SERVICES Among the most important advantages of Grid network services architecture is the ability to match application requirements to communication services to a degree that has not been possible previously. This capability can be realized through network services-oriented APIs that incorporate signaling among Web Services. At a basic level, 5.6 Grid Network Services and Network Services 87 such a signal could request any number of standard services, either connectionless and connection oriented, e.g., TCP/IP communications, multicast, layer 2 paths, VPNs, or any other common service. Grid network Web Services can allow for specialized signaling that can be used for instantiating new service types, in accordance with the general approach of Grid architecture. For example, such signaling can enable requests for particular highly defined services through interactions between applications and interfaces to the required network resources. Instead of signaling for standard best effort services, this signal could be a request for a service with a precisely defined level of quality assurance. Through this type of signaling, it is possible to integrate Grid applications with deterministic networking services. 5.6.1 DETERMINISTIC NETWORKING AND DIFFERENTIATED SERVICES 5.6.1.1 Defining and customizing services Today, almost all Internet services are best effort and nondeterministic. Few capa- bilities exist for external adjustments for individual service attributes. Specialized, high-quality services have been expensive to implement, highly limited in scalability, and difficult to manage. Particularly problematic is specialized, inter-domain services provisioning. The Internet primarily consists of an overlay network supported by a core network consisting of static, undifferentiated electronic switched paths at the network edge and static optical channels within the network core. Because the current Internet is an overlay network, operating on top of a fixed hierarchical physical infras- tructure with minimal interaction between the layer that routes packets (layer 3) and other layers, basic topologies usually cannot be changed dynamically to enhance layer 3 performance, for example by using complementary services from other layers. Consequently, differentiated services have not been widely implemented. They have usually been implemented within LANs or within specialized enterprise networks. Grid network services architecture can be used to provide determinism in networks. High-level signaling, in conjunction with intermediate software compo- nents, can provide for optimized matches between multiple application require- ments, which can be expressed as specified deterministic data flows and available network resources. These processes can be based on specialized communications (either in-band or out-of-band) comprising requests for network services signaled into the network, information on the network resources and status signaled by network elements, various performance monitoring and analysis reports, and other data. This architecture allows both link state and stateless protocol implementation, and provides for information propagation channels among core network elements. 5.6.1.2 Quality of service and differentiated services The need for differentiated services has been recognized since the earliest days of data networks. There have been attempts to create differentiated services at each traditional service level. Many earlier projects focused on signaling for specific Quality of Service (QoS) levels. A number of these initiatives have been formalized through standards bodies, such as the IETF DiffServ efforts, described in Chapters 6 and 8. 88 Chapter 5: Grid Network Services and Implications for Network Service Design Other projects attempted at QoS provisioning at layers 2 and 1. However, because of management, provisioning logistics and cost considerations, these services have not been widely implemented. Currently, almost all Grid services are being supported by undifferentiated, nondeterministic, best effort IP services. 5.6.1.3 Grid network services Through standard Grid abstraction techniques, individual users or applications (either ad hoc or through scheduling) are able to directly discover, claim, and control network services, including basic network resources. Such services can be standard, such as IP or transport (TCP or User Datagram Protocol (UDP)) or specialized (Stream Control Transmission Protocol, SCTP) [3], or they can be based on layers below layer 3, such as layer 2 paths and light paths. These capabilities can be utilized across multiple domains locally, regionally, nationally, and internationally. Furthermore, they can dynamically change the attributes and configurations of those resources, even at the application level. Grid applications have been demonstrated that can discover and signal for specific types of network services, including by dynamically configuring and reconfiguring lightpaths locally, within metropolitan areas, nation- ally, and internationally. Another powerfulcapabilityofthisnetworkservicesarchitecture isthatitcan provide for a unique capability that allows for a scalable, reliable, comprehensive integration of data flows, with various service parameters at all traditional service layers, i.e., layers 1, 2, 3, and 4 and above. Different types of services required by applications with various specified parameters (e.g., stringent security, low latency, minimal jitter, extra redun- dancy, minimal latency) can be blended dynamically as needed. This architecture can provide for the incorporation of integrated services at all levels, each with options for various service parameters, layer 3 services (e.g., high- performance IPv4, IPv6, unicast, and multicast), layer 2 services, including large-scale point-to-point layer 2 services, and layer 1 wavelength-based transport, including end-to-end lightpaths, with options for single dedicated wavelengths, multiple wave- lengths, and subwavelengths. Dynamically provisioned lightpaths have been demon- strated as a powerful capability whether integrated with layer 3 and layer 2 services or as direct layer 1-based dedicated channels. 5.7 GRID NETWORK SERVICE COMPONENTS A Grid network service architecture includes various processes that are common to other Grid services, including functions for resource discovery, scheduling, policy- based access control, services management, and performance monitoring. In addi- tion, the architecture includes components that are related specifically to network communication services. 5.7.1 NETWORK SERVICE ADVERTISEMENTS AND OGSA A key theme for Grid environments is an ability to orchestrate diverse distributed resources. Several standards bodies are designing architecture that can be used 5.7 Grid Network Service Components 89 for Grid resource orchestration. Many of these emerging standards are described in Chapter 4. Grid network services are being developed within the same frame- work as other Grid services, e.g., the Open Grid Services Architecture (OGSA), which is being created by the Global Grid Forum (GGF) [4]. The work of the GGF complements that of the OASIS standards group (Organization for the Advance- ment of Structured Information Standards) [5]. Also, W3C is developing the Web Services Definition Language (WSDL) and the Web Services Resource Framework (WSRF) [6]. These standardized software tools provide a means by which various Grid processes can be abstracted such that they can be integrated with other processes. The GGF has endorsed this architecture as a means of framing Grid service offerings. 5.7.2 WEB SERVICES In a related effort, OASIS is developing the Web Services Business Process Execution Language (WSBPEL or BPEL4WS). The WSBPEL initiative is designing a standard business process execution language that can be used as a technical foundation for innumerable commercial activities. At this time, there is a debate in the Web Services OGSA community about the best way to support state. The current OGSA approach is to create stateful Web Services. An alternative approach is to keep all Web Services stateless and maintain state within the BPEL. The latter approach is more consistent with recursive object-oriented design. Although oriented toward business transaction processing and common informa- tion exchange, this standard is being developed so that it can be used for virtually any process. The architecture is sufficiently generalized that it can be used for an almost unlimited number of common system processes and protocols, including those related to resource discovery and use, access, interface control, and initiating executable processes. This model assumes that through a SOA based on WSRF, multiple, highly distributed network resources will be visible through service advertisements. Over time, increasing numbers of these network services and related resources will be exposed as Web Services, e.g., using web tags to describe those services. Using these tools, a Web Services “wrapper” can be placed around a resource, which can then be advertised as a component for potential use by other services within Grid environments. Eventually, some of these resources may contain such Web Services components as an integral part of their basic structure. 5.7.3 WEB SERVICES DEFINITION LANGUAGE (WSDL) However, if they are to be widely advertised and discovered, a standard mechanism is required, such as a standards-based registry service devoted to supporting Web Services as defined by the W3C standards. The international advanced networking community has established a process, in part through an international organizational partnership, to create WSDL schema that will design supersets of User-to-Network Interface (UNI) functionality, including multiple WSRF stateful elements. The initial instantiations of this model have been designed and implemented, and are being used [...]... each of these layers Grid Networks: Enabling Grids with Advanced Communication Technology Gigi Karmous-Edwards © 2006 John Wiley & Sons, Ltd Franco Travostino, Joe Mambretti, 100 Chapter 6: Grid Network Services: Building on Multiservice Networks 6.2 GRID NETWORK SERVICES AND TRADITIONAL NETWORK SERVICES Grid network services are those that manage, control, or integrate some aspect of communication service... provide references to literature Grid Networks: Enabling Grids with Advanced Communication Technology Gigi Karmous-Edwards © 2006 John Wiley & Sons, Ltd Franco Travostino, Joe Mambretti, Chapter 7: Grid Network Middleware 1 14 7.2 DEFINITIONS 7.2.1 NETWORK SERVICES AND GRID NETWORK SERVICES Chapter 6 has reviewed the plurality of services that a network implements at layers 1 4 of the OSI stack Such network... ability to inform networks of their needs 7.3 Grid Infrastructure Software Grid network infrastructure is the ensemble of services that elevate the network to Grid- managed resource This software typically culminates with advertised Grid network services or Grid network services management tools To the point, Grid network infrastructure implements: • communication infrastructure for Grids and similar... Transactions on Communications E86-B, 8, 2263–2272 [16] T DeFanti, C De Laat, J Mambretti, and B St Arnaud (2003) “TransLight: A Global Scale Lambda grid for E-Science,” special issue on “Blueprint for the Future of High Performance Networking.” Communications of the ACM, 46 (11), 34 41 [17] www.glif.is Chapter 6 Grid Network Services: Building on Multiservice Networks Joe Mambretti 6.1 INTRODUCTION The Grid. .. Distributed Environments Based on Dynamic Lightpath Provisioning,” special issue with feature topic on Optical Control Planes for Grid Networks: Opportunities, Challenges and the Vision IEEE Communications Magazine, 44 (3), pp 92–99 [13] User Controlled Lightpaths, http://www.canarie.ca/canet4/uclp/ [ 14] http://obgp.canet4.net/ [15] T DeFanti, M Brown, J Leigh, O Yu, E He, J Mambretti, D Lillethun, and... These services are implemented as software agents that interface with the network A Grid network service is one with a function in alignment with the requirements described in Chapter 3 and with an interface that is compatible with Grid infrastructure software structured in the context of Service-Oriented Architecture (SOA) Examples of Grid network services include a broker for a minimal latency end-toend... large-scale, resource-intensive dynamic processes within highly distributed environments, such as Grids, to manage core resources within networks, primarily lightpaths [12] It has generally been implemented within an OGSA context, using standard software components from that model The initial implementations were based on OGSI It has also been integrated with other network-related components such as an... to layer 2 and layer 1 services within a Grid environment This section will note that the goal of providing infrastructure to transmit datagrams optimally is not mutually exclusive with incorporating circuit-oriented technologies within Grid environments The abstraction layers enabled by basic Grid architecture support the optimization of any resource Also, as RFC 343 9 emphasizes, this approach does... service (see Section 3.3 .4) The Global Grid Forum’s High Performance Networking Research Group (GHPN-RG) has introduced the concept of a Grid network service in a formal document [3] While network services and their derivatives are not directly exposed to elements of the Grid infrastructure, Grid network services are directly exposed to elements of the Grid infrastructure (such as a Grid meta-scheduler... appropriate allocation of bandwidth, and ensure that on-going QoS performance was met through process monitoring 6.8 GRIDS AND NONROUTED NETWORKS The advanced networking research community has been investigating the potential for complementing the general layer 4/ layer 3 services in Grid environments with methods based on complementary nonrouted services Such services range from flow switching to IP-based virtual . general Grid resources are also being used to abstract network capabilities from underlying infrastructure. This architecture can be expressed in Grid Networks: Enabling Grids with Advanced Communication. architectural and technical details of the services at each of these layers. Grid Networks: Enabling Grids with Advanced Communication Technology Franco Travostino, Joe Mambretti, Gigi Karmous-Edwards ©. High Performance Networking.” Communications of the ACM, 46 (11), 34 41 . [17] www.glif.is. Chapter 6 Grid Network Services: Building on Multiservice Networks Joe Mambr etti 6.1 INTRODUCTION The Grid network community