/ EUROPEAN COMMISSION
DIRECTORATE-GENERAL FOR AGRICULTURE AND RURAL DEVELOPMENT
DirectorateI. Management of resources
I.3. Information technology

AGRIDES web-services

Information for Member States

Introduction

1.1. Purpose

1.2. Scope

1.3. Definitions, Acronyms, and Abbreviations

2. AGRIDES Communication Flow using Web Services

2.1. Overview

2.2. Web services

3. Web service use cases

3.1. Description & scope

3.2. Actors & goals

3.3. UML diagram

3.3.1. Sequence diagram

3.4. Description of the services......

4. Technical Implementation

4.1. Technologies/Requirements

4.2. Operations overview (to be merged with the service description paragraph)

4.2.1. uploadFile

4.2.2. getFileStatus

4.3. Versioning & backward compatibility (useful for MS ?)

4.4. Request & response samples

4.5. Securing the web service

5. Usage guidelines

5.1. Web Service client guidelines

5.1.1. Example:

5.2. Eclipse WS client tutorial

6. ANNEX : WSDL

Introduction

1.1.Purpose

This document describes the web services provided by AGRIDES system for the use of other information systems.

1.2.Scope

The description covers the technical solution used to implement these web services.

1.3.Definitions, Acronyms, and Abbreviations

Abbreviations and Acronyms
Abbreviation / Meaning
AGRIDES / Agriculture Data Exchange System
DG AGRI / Directorate General for Agriculture
EC / European Commission
ISAMM / Information System for Agricultural Management and Monitoring
JDS / jCore Document Storage system, which provides DG AGRI applications with a single, shared document storage system.
JMS / Java Message Service
LogViewer / AGRIDES LogViewer, a separate application available to MSA, allowing them to consult the status of previously submitted consultations
MS / MemberState
MSA / MemberState Administration
UMAN / jCore User Management module. Allows applications to retrieve user-related data from a central database.
WUSI / AGRIDES Web Upload Secure Interface, a separate application available to MSA, allowing them to submit communications via file upload
Template / A schema defining the syntax and validation rules for a given type of communication
Communication / Set of informations, submitted to DG AGRI by an MSA, independent of how it is submitted
Target Application / An information system which processes communications which arrive in DG AGRI through the AGRIDES system, for example ISAMM
SOAP / formerly Simple Object Access Protocol
WSDL / Web Service Definition Language – an XML based language that provides a model for describing the Web services

Table 1: Abbreviations and Acronyms

2.AGRIDES Communication Flow using Web Services

2.1.Overview

This flow is initiated by the MSA, who use AGRIDES for submitting their communication to DG AGRI.

Fig. 1 - Flow of a communication submitted through AGRIDES

The MSA submit a communication (1). This could be via a Web Service, Web Input Form or Web Upload Form provided by AGRIDES[i]. This communication will then be stored in the central document store (JDS, 2 & 3) by AGRIDES and validated. If the validation is successful, the ID of this communication will be passed on to the target application (4), which in turn will use this ID to retrieve the communication itself (5 & 6), validate it and send back the resulting validation report to AGRIDES (7). AGRIDES will notify the MSA based on this validation report (8), and the flow ends there.

If the AGRIDES validation results in a refusal of the communication, steps 4 to 7 are not performed, i.e. the target application will not be informed about this communication.

Starting with Phase II, the communication of the MSA files/messages will be also supported via web services. It differentiates itself from the other transfer methods by the fact that it allows automated transfers between IT systems (no human intervention is required).

2.2.Web services

Web services provide web-based APIs to support machine-to-machine communication over networks. Because these APIs are web-based, they inherently support interaction between devices running on different architectures and speaking different native languages.

Actually a web service is a self-contained software component with a well-defined interface that implements a specific business task. In the AGRIDES context, the offered service is a file (data) transfer service.As its name suggests, a web service is built on top of web technologies and open-standards making it particularly suitable for B2B application integration. All web service schemes utilize a web-based transport mode, such as HTTP, HTTPS, or SMTP, and a method for packaging the queries and responses, typically some sort of XML schema.

The base protocols that formed the initial specification of web services are SOAP, WSDL and UDDI. These have effectively become de facto standards, with universal acceptance and widespread implementation by vendors. A wide range of additional protocols have and are still being proposed to address important aspects such as security and reliability as well as more complex business scenarios.

Some terminology is unique to web services. Creating an XML file containing outbound messages is sometimes called serialization while extracting information from an incoming XML file is sometimes called deserialization. Calling a function or method on a remote server is often called an invocation. A web service may expose procedural functions or object-oriented methods. The general term operation is used to describe either a function or a method.

Figure 2 – Web Services AGRIDES Architecture

As depicted in the Figure 2, we propose to create one webservice with two main operations: one for handling the submissions of new files, one for interrogating the status of the submission.

Figure 3– AGRIDES Component Overview

The web service will be implemented using Apache CXF (an open source services framework), Spring framework and will be deployed on a J2EE application server (Weblogic).

The use of the HTTP(S) protocol for the transport of XML requests is a major factor of Web services success: it enables the messages to be transported across firewalls and other devices such as proxies or reverse proxies as for usual web traffic.

Several aspects have been taken into consideration for this new transfer method, and they will be explained in details over the next sections of this document:

  • security
  • data exchange protocol : service description and data transfer
  • notifications

3.Web service use cases

3.1.Description & scope

The current implementation of the AGRIDES web services covers only :

  • the files upload and
  • the retrieval of the file submission status operations.

Also, we'll keep the existing limitations in the WUSI module:

–the maximum file size set to: 20Mb

–the web-service for retrieving the status of a submission will use the same security validation rules as in AGRIDES WUSI's log viewer. If the security constraints are not respected, the service will return an error message.

–the supported file types are plain text and xml files only.

3.2.Actors & goals

The use cases below have only non-human actors

  • MS client application (non-human actor): submission of new files and retrieval of the submission status. It is practically behaving like the Correspondent actor in the AGRIDES WUSI use case.
  • AGRIDES DCS: data collection service used to validate and process the file
  • AGRIDES database: used in saving the file metadata and querying the submission status

Remarks:

  • In the description of the use cases, there will be also made remarks when security features are also playing a role, with a specific paragraph format (italic, and in red color: e.g. security feature)

3.3.UML diagram

3.3.1.Sequence diagram

Figure 4 - Sequence diagram

As depicted in the sequence diagram, there are a few activities to be executed on the Member States side:

  • Generate/prepare the files for upload
  • Execute calls to the published AGRIDES web services (uploadFile, getStatus)

4.Technical Implementation

4.1.Technologies

SOAP

SOAP is a protocol for exchanging information between applications, through XML messages. It defines how the messages can be structured in an independent way of any programming language or platform (this is being called interoperability). In essence, SOAP is only a specification, a set of rules about how information can be exchanged between applications, and running web services using SOAP merely stands for creating web services applications that follows SOAP specifications (called SOAP implementation).

The following technologies can be used for the implementation of the clients on the MS side.

JAX-WS

It is a Java programming language API for creating/consuming web services. It uses annotations to simplify the development and deployment of web service clients and endpoints. It replaced the JAX-RPC API in JAVA platform EE 5.

Apache CXF Web serviceframework

Is an open-source Web Services framework which helps building/developing services and clients using frontend programming APIs, like JAX-WS. It supports a variety of protocols like SOAP, XML/HTTP, RESTful HTTP, and work over a large suite of transport protocols, like HTTP, JMS or JBI.

It's intuitive and easy to use, providing a flexible deployment and a strong integration with Spring framework.

4.2.Operations overview

This chapter contains the descriptions of the web services methods and their parameters, exceptions and error handling.

4.2.1.uploadFile

  • In Agrides Phase II, the upload file service is provided with several flavors:

–Upload file as strings

–Upload file as a bytes array

–Upload of binary files (MTOM support with base64 encoding)

–Upload of a compressed file (the files should be compressed using ZLIB format – more about this open source tool can be found on its site – JAVA programming language is providing out of the box a package to handle compressing/decompressing operations for the standard ZIP and GZIP formats: java.util.zip

  • There are a few mandatory parameters , common to all operations in a direct or indirect way (part of a structure):

–username (author of the file) – should be an authorized and active AGRIDES/ISAMM user

–Member state – a valid ISO 3166-1Country Char 2 code

–File name

–The file content (should not be empty or exceed the maximum upload file size)

MTOM upload

Request Parameters

Parameter name / Type / Description
fileMetadata / FileMetadata / FileMetadata structure contains:
- username - should be a valid AWAI username
- memberstate- (ISO 3166-1Country Char 2)
- filename – the name of the file (maximum length is 100)
- isForValidationOnly – Flag to control the type of message – when set to true, the file only passes through AGRIDES validation without creating reports in ISAMM
- isSigned - is not yet operational (intented to be used in the secured implementation of the web service, in case of using embedded digital signatures)
- isCompressed– is not yet operational for this method.
fileBinary / DataHandler / content of the file

String upload

Request Parameters

Parameter name / Type / Description
username / String / Author of the file – should be a valid AWAI username
memberState / String / Member state code according to ISO 3166-1Country Char 2 standard
fileName / String / Name of the file (maximum length is 100)
isForValidationOnly / boolean / Flag to control the type of message – when set to true, the file only passes through AGRIDES validation without creating reports in ISAMM
isSigned / boolean / Is not yet operational; this flag is intended to be used in the secured implementation of the web service, in case of using digital signatures
fileContent / String / content of the file

Byte[] upload

Request Parameters

Parameter name / Type / Description
username / String / Author of the file – should be a valid AWAI username
memberState / String / Member state code according to ISO 3166-1Country Char 2 standard
fileName / String / Name of the file (maximum length is 100)
isForValidationOnly / boolean / Flag to control the type of message – when set to true, the file only passes through AGRIDES validation without creating reports in ISAMM
isSigned / boolean / It is not yet operational (intented to be used in the secured implementation of the web service, in case of using embedded digital signatures)
isCompressed / boolean / Flag which specifies if the file is compressed or not. It supports files compressed using standard ZIP formats.
data / byte[] / File content sent as an array of bytes

Response

This is common for all the operations:

Type / Description
AgridesUploadResponse / Contains 2 fields:
fileId – as registered in the Agrides DB
message – a description of the operation execution status (success or error message).

Remark: in case of any error preventing registering the file in AGRIDES system, the file ID in service response will be set with value -1.

4.2.2.getFileStatus

The service takes as input parameters the fileId and the username and returns the current status of the file in Agrides

Currently, since the security model to be used is still under review, the service makes use of the username parameter value to perform a basic authorization check if the user is entitled to read the status of this specific file

Request Parameters

Parameter name / Type / Description
fileId / long / This parameter contains the fileId of a file being already registered in the AGRIDES system
author / String / Identifies the author of the file.

The service returns a string with the file status (as recorded in the MSA_FILE table).

4.3.AGRIDES file status – state diagram

Figure 5 - AGRIDES file status - state diagram

4.4.Versioning & backward compatibility

Considering the constant need of software for functionality adjustments as well as increase in accuracy, speed or stability of it's services, a versioning mechanism for the web services APIs must be taken into consideration from the initial stages of the design and implementation.

Regarding the backward compatibility of the web service clients, there are 2 types of changes in the WSDL document that will not break an existing client implementation (addition of new operations to an existing WSDL document and the addition of new XML schema types within the document) and a few others which can break this rule (removing/renaming an operation, changing the parameters – type/order, changing the structure of a complex data type).

Therefore, a combination of the next strategies should be used for the management of the webservices changes:

  • In case of changes not-impacting the backward compatibility, it's enough to add the WSDL document into a revision control system (on the service provider side)
  • For backward compatibility breaking changes, the solution is to use XML namespaces to clearly define the versions of a document that are compatible with the web client applications. This depends on the SOAP messaging model and DIGIT recommendation is to use a document message with literal encoding, meaning that the namespace will be specified in the definition of the messages as part of the XML schema namespace definitions. Once a decision is made for a new version to be created, this translates into changing the namespace. The problem is to determine what to do with the old requestors. There are a few option to handle them:
  • The response message from the server contains a message for the clients about the deprecation of the service, while routing the handling of the message to the old logic (the idea is to allow a transition period for the clients to update)
  • Generate a failure on the server (no transitional period)

4.5.Request & response samples

Upload file service request and response sample

4.6.Securing the web service

Describes the security features: protocols, authentication, authorisation and encryption, audit and non-repudiation.

Remark: It is pending for a decision on the security model to be adopted!

5.Usage guidelines

Member States are free to use the technology they want to implement the calls to the web services. Any reference to technology used at DG AGRI side is given for information only.

5.1.Web Service client guidelines

For the web service client application, there are 2 types of client APIs for invoking web services:

  • Dispatch client: use this client when it's needed to perform work at the XML message level or when it's being required to work without any generated artifacts.
  • Dynamic proxy client: use this client when invoking a web service based on its endpoint interface (a proxy is a local object with the same interface as the actual web service and allows for calling web service as if it were a local object).

Before developping web services clients based on JAX-WS programming API, it is recommended first to determine the client model that best suits its needs.

The remainder of this section will describe how to develop a static web service client starting with a WSDL file.

For a Java client application it is necessary to perform a set of actions for making use of the services provided:

  • Use the web service's URL to retrieve the WSDL file
  • Use the WSDL file to create web service client artifacts that can access and consume a web service (there are tools specific to frameworks/languages, likewsimportutility).
  • Write the client application specific code and web-service invocation code using the artifacts
  • Assembly a web-service client jar/war/ear and test it

5.1.1.Example:

Web service side

This is the code of the business interface:

@WebService(name="UploadFileService")

publicinterfaceUploadFileService {

@WebMethod(operationName="uploadFile")

publicAgridesUploadResponseuploadFile(…);

}

This is the code of the service implementation:

@WebService(endpointInterface = "agrides_ws.UploadFileService", serviceName ="UploadFileServiceImpl")

publicclassUploadFileServiceImplimplementsUploadFileService {

final static int FAILURE = -1;

public AgridesUploadResponseuploadFile() {

// …………………………

returnfile ID together with a descriptive message;

}

}

Sample client

publicclass Client {

/**mainmethod. */

publicstaticvoid main(String[] args) {

UploadFileService clientUpload =new UploadFileService();

AgridesUploadResponse response=clientUpload.uploadFile(…);

int fileId=response.getFileId();

if (fileId!= -1) {

//store fileId for further references

} else {

Logger.error(response.getMessage());

}