DataMover

DataMover is a file transfer system that utilizes the capabilities of WebSphere MessageQ (WMQ) to quickly and reliably move data from one location to another. DataMover is used by many companies that already have WMQ installed and can take advantage of this transport to greatly improve the efficiency and reliability of the data transfer needs.

Using DataMover, you simply copy files into a directory on one system and they are moved to a directory on a destination system. There is no need to schedule a file transfer program (such as ftp) to run periodically since the DataMover is always running and monitoring the arrival of files in a specified directory .

A DataMover instance consists of a sender and receiver, however, Senders on different systems can use a single receiver in a hub-and-spoke configuration where files are collected on a central system. An outstanding feature of DataMover is the capability to select the target directory for files based on the source directory, filename and extension.

Each side of the DataMover is implemented with a WMQ Server application that is configured as a sender or receiver at startup. Senders and receivers employ a messaging protocol that insures complete transmission of the messages transporting the file without using WMQ persistent messaging.

This implementation significantly decreases the WMQ administration since there is no need for log file administration. Additionally, the entire file is pre-allocated on the receiver system to insure that sufficient file system space is available.

The following is the sequence of events that the DataMover uses to transfer files from one system location to another:

  1. The data file to be moved is copied to a single source directory on the source system.
  2. The DataMover Sender periodically examines the source directory for files to be transferred. The DataMover Sender can be configured to move all files present in the directory or to move only those files that have a trigger file. A trigger file:
    1. Can be used to control the transfer of files, when it is important to send files in a particular order or at a specific time and to control the removal of the file at the receiver side. Use of the trigger file is optional but is either enabled or disabled at start-up.
    2. Has the same filename as the data file but always has a single unique extension (typically “.TRG”)
    3. Is zero-length
  3. A selected file is transferred to a single target directory on the target system.
  4. Upon successful completion of the transfer:
    1. A user-specified completion script is executed on the sender system. This facility allows you to dispose of the transferred file on the sender system.
    2. The trigger file (if present) is deleted from the source directory
    3. A trigger file is created in the target directory (this facility allows applications using the transferred file to determine when transfer is complete).
  5. If a transfer is unsuccessful
    1. A user-specified completion script is executed on the sender system.
    2. The source file remains in the source directory.
Feature
Benefit
Support for Multiple Platforms The DataMover was written using coding methods that make it very easy to port to other platforms while it maintains a high level of reliability and performance. It is currently supported on AIX, Linux, HP_UX, HP_UX11i, Windows and OpenVMS.
Easy to configure. All configuration information is in the XML style text file. This makes it easy to define the parameters needed for your particular topography and it is portable to most platforms.
Can generate notifications. A number of user definable exits have been provided to enable users to expand the capability of DataMover to perform specific tasks on successful or unsuccessfully completion of a file transfer. These are particularly useful for archiving files that have been moved, etc..
Message notification. A configuration option is provided to enable DataMover to send notification messages to a specified WMQ queue on successful or unsuccessful completion of a file transfer or any event such as an error or loss of message transport, etc.
Detailed logging of all events. A log file will be generated and updated during the file transfer process. All events will be recorded in this file. There is an option to provide multiple levels of logging and tracing.
File size limitations. DataMover does not limit the size of files that can be transferred nor does it limit the size of the transfer messages. In fact it enables the user to select ranges of message sizes relative to the files size. This enables the user to tune the system for the best performance and transfer efficiency.
 

Certified by IBM for Linux on Intel & Linux on Power:

Certified for HP_UX11i on Itanium:

.For a complete DataMover User Guide, in PDF form, please click here.