Watching a Directory and Sending Files with Universal File Mover (How To #11)

In this blog posting, I will show you how to configure Universal File Mover to watch a directory for new files and send the new files immediately. In this example, Universal File Mover will connect to the queue manager in “client mode”.

In the Universal File Mover’s Watch Action, the user can specify all files (‘*’) or select particular files by their file extension (i.e. txt) to have UFM actions done against the file (i.e. MQSend).


In this example, the following servers are used but they only have WebSphere MQ (WMQ) Client installed (no queue managers):
- aix002 is an AIX server with WMQ Client and UFM software installed
- linux002 is a Linux server with WMQ Client and UFM software installed

In this example, the following queue managers are used:
- MQA1 is a queue manager residing on a AIX (aix001) server (sender)
- MQL1 is a queue manager residing on a Linux (linux001) server (receiver)

- TEST.LINUX.QL and TEST.LINUX.QL.BK are local queues defined in queue manager MQL1 (receiver)
- TEST.LINUX.QR is a remote queue defined in queue manager MQA1 (sender)

If you do not know how to define/setup communication between 2 queue managers then follow the instructions in this blog posting:
http://www.capitalware.com/rl_blog/?p=509

Step #1: On the linux002 server, create a file in the mq directory called mql1.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_MQ SYSTEM "UFM_MQ.dtd">
<UFM_MQ>
    <QMgrName>MQL1</QMgrName>
    <QueueName>TEST.LINUX.QL</QueueName>
    <Hostname>linux001</Hostname>
    <ChannelName>SYSTEM.DEF.SVRCONN</ChannelName>
    <Port>1414</Port>
    <UserID>tester</UserID>
</UFM_MQ>

The mql1.xml (UFM_MQ XML) file describes how to connect to the remote MQL1 queue manager on server linux001. The connection will use UserID of tester. If you do not have that UserID defined on the linux001 server then use a UserID that is defined.

Step #2: On the linux002 server, in the UFM install directory, create a file called ufm_receive_test_11.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>

  <Actions>
    <MQReceive run="D" >
      <MQ>
        <MQFile>mql1.xml</MQFile>
        <BackOutQName>TEST.LINUX.QL.BK</BackOutQName>
      </MQ>
      <Default>
         <Directory override="Y">/home/roger/UFM/</Directory>
      </Default>
    </MQReceive>
  </Actions>

</UFM_Workflow>

When UFM is started, it will run as a daemon (run=”D”) and use a backout queue called TEST.LINUX.QL.BK just in case there is an issue with a message. UFM will override the message’s specified directory and use the one provided. Either create /home/roger/UFM/ directory on your Linux server or use a directory that already exist on your Linux server.

Step #3: On the linux002 server, start UFM to receive the file transfers:

./ufm.sh ufm_receive_test_11.xml &

Step #4: On the aix002 server, create a file in the mq directory called mqa1.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_MQ SYSTEM "UFM_MQ.dtd">
<UFM_MQ>
    <QMgrName>MQA1</QMgrName>
    <QueueName>TEST.LINUX.QR</QueueName>
    <Hostname>aix001</Hostname>
    <ChannelName>SYSTEM.DEF.SVRCONN</ChannelName>
    <Port>1414</Port>
    <UserID>tester</UserID>
</UFM_MQ>

The mqa1.xml (UFM_MQ XML) file describes how to connect to the remote MQA1 queue manager on server aix001. The connection will use UserID of tester. If you do not have that UserID defined on the aix001 server then use a UserID that is defined.

Step #5: On the AIX server, in the UFM install directory, create a file called ufm_watch_test_1.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>

  <Actions>
    <Watch pollinterval="5">

      <WatchItem type="F">
        <File>/apps/test/amce.txt</File>
        <Actions>
          <MQSend delete="Y" format="S">
            <File>${WATCH_FOUND_FILE}</File>
            <MQ>
              <MQFile>mqa1.xml</MQFile>
            </MQ>
            <Remote>
              <Directory>/apps/other/</Directory>
            </Remote>
          </MQSend>
        </Actions>
      </WatchItem>

      <WatchItem type="D">
        <Directory>/apps/test2/</Directory>
        <Actions>
          <MQSend delete="Y" format="S">
            <File>${WATCH_FOUND_FILE}</File>
            <MQ>
              <MQFile>mqa1.xml</MQFile>
            </MQ>
            <Remote>
              <Directory>/apps/other2/</Directory>
            </Remote>
          </MQSend>
        </Actions>
      </WatchItem>

      <WatchItem type="D">
        <Directory>/apps/test3/</Directory>
        <Extension>txt</Extension>
        <Extension>csv</Extension>
        <Actions>
          <MQSend delete="Y" format="S">
            <File>${WATCH_FOUND_FILE}</File>
            <MQ>
              <MQFile>mqa1.xml</MQFile>
            </MQ>
            <Remote>
              <Directory>/apps/other3/</Directory>
            </Remote>
          </MQSend>
        </Actions>
      </WatchItem>

    </Watch>
  </Actions>

</UFM_Workflow>

When UFM is started, the Watch Action will run forever. The poll interval for the Watch Action tells UFM to continuously scan (every 5 seconds) the files/directories given in the WatchItem elements of the Watch element.

(1) The first WatchItem element has UFM waiting for the file “/apps/test/amce.txt” to appear and when it does, UFM performs the Actions associated (i.e. MQSend) within the WatchItem element. After the MQSend action sends the file, the file is delete.

(2) The second WatchItem element has UFM watching a directory (/apps/test2/) for any file to appear in that directory. When a file appears, UFM performs the Actions associated (i.e. MQSend) within the WatchItem element.

(3) The third WatchItem element has UFM watching a directory (/apps/test3/) for any file with the file extension of “txt” or “csv” to appear in that directory. When a matching file appears, UFM performs the Actions associated (i.e. MQSend) within the WatchItem element.

Note: The ${WATCH_FOUND_FILE} variable is by the user as a place holder to the “found file” from the WatchItem. UFM will transform the variable to the actual file name when the Action is executed.

Step #6: On the aix002 server, start UFM to begin monitoring for files:

./ufm.sh ufm_watch_test_1.xml &

UFM will start, watch for files to arrive in the specified directory. Note: To stop the Watch Action, the user will need to use the Ctl-C or kill command.

Step #7: On the linux002 server, verify that the test file was put into the /home/roger/UFM/ directory or whatever directory you specified in the ufm_receive_test_11.xml file.

Step #8: Finally, we need to stop UFM daemon that is running on the linux002 server. In the UFM install directory, create a file called ufm_putquit_test_11.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>
  <Actions>
    <MQPutQuit>
      <MQ>
        <MQFile>mql1.xml</MQFile>
      </MQ>
    </MQPutQuit>
  </Actions>
</UFM_Workflow>

Step #9: On the linux002 server, run UFM with the MQPutQuit action:

./ufm.sh ufm_putquit_test_11.xml

This blog demonstrates how easy it is to watch a directory for files or for a single file to arrive then immediately send the files using UFM.

Remember, when possible, use the compression option for any MQSend Action used within an WatchItem element to speed up the transfer time and reduce the load placed on MQ. You can combine both encryption and compression when sending files (any size) using UFM.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), Java, Linux, Mac OS X, MQ, Open Source, Universal File Mover, Unix, Windows Leave a comment

Capitalware’s Products and the Heartbleed Issue

None of Capitalware’s Products directly use OpenSSL, so the Heartbleed issue does not directly affect our products.

If you use MQAUSX on Linux and have configured MQAUSX to target an LDAP server using “LDAP over SSL”, then you will want to check what OpenSSL package is installed on your Linux server.
i.e.

rpm -qa | grep openssl

The affected versions of OpenSSL are OpenSSL 1.0.1 through 1.0.1f (inclusive). If you have an affected version then have your Linux SysAdmin upgrade your Linux server ASAP.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Open Source, Operating Systems, Security Leave a comment

WebSphere MQ FixPack 7.0.1.12 Released

IBM has just released FixPack 7.0.1.12 for WebSphere MQ.
http://www.ibm.com/support/docview.wss?uid=swg21666487

Regards,
Roger Lacroix
Capitalware Inc.

IBM i (OS/400), Linux, MQ, Unix, Windows Leave a comment

UFM and MQJE001: Completion Code ’2′, Reason ’2495′

Universal File Mover (UFM) can work with both 32-bit and 64-bit JVMs. If you are using client mode for connecting to your queue manager then life will be fine with either of the JVMs.

If you are on Windows, attempting to connect to the queue manager in bindings mode with a 64-bit JVM, you may get the following error:

MQException: MQJE001: Completion Code '2', Reason '2495'.
MQException: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2495;
AMQ8568: The native JNI library 'mqjbnd' was not found. [3=mqjb MQCC=2 : MQRC=2495

The reason for this error is that when WMQ Server was installed, the installer updated the PATH environment variable with both the 32-bit and 64-bit paths to the mqjbnd.dll. The problem is that the 32-bit path is first, hence, the failure as the user is using the 64-bit JVM.

You have 2 ways to solve this issue:

(1) Change your PATH so that the {WMQ_Install_dir}\java\lib64 is before {WMQ_Install_dir}\java\lib directory but this may affect other running MQ application that may have their own JVM that is 32-bit.

(2) Add the following line to the ufm.bat file:

set PATH=%MQ_FILE_PATH%\java\lib64;%PATH%

Personally, I think option # 2 is the simplest and easiest option to implement to fix the above error.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), Java, Linux, Mac OS X, MQ, Open Source, Universal File Mover, Unix, Windows Leave a comment

Java 8 Released

Oracle has just released Java 8.
http://www.oracle.com/technetwork/java/javase/8train-relnotes-latest-2153846.html

Java Platform, Standard Edition (Java SE) lets you develop and deploy Java applications on desktops and servers, as well as in today’s demanding embedded environments. Java offers the rich user interface, performance, versatility, portability, and security that today’s applicationsrequire.

Regards,
Roger Lacroix
Capitalware Inc.

Java Leave a comment

Mozilla Firefox 28.0 Released

Mozilla Firefox has just released Mozilla Firefox v28.0.
http://www.mozilla.com/firefox/

Mozilla Firefox is a free and open source web browser descended from the Mozilla Application Suite and managed by Mozilla Corporation. To display web pages, Firefox uses the Gecko layout engine, which implements most current web standards in addition to several features that are intended to anticipate likely additions to the standards

Regards,
Roger Lacroix
Capitalware Inc.

Linux, Mac OS X, Open Source, Windows Leave a comment

Power Development Cloud Horror Story

Does anyone know who is responsible (director or VP) for IBM’s Power Development Cloud (aka Virtual Loaner Program)? I have not been able to load an ‘IBM i (OS/400) saved image’ on IBM’s Power Development Cloud since February 19th, 2014. I have opened ticket after ticket after ticket after ticket after ticket after ticket with IBM support but all I ever get is ‘we are working on it’, ‘it will be fixed in a few days’, etc. and I still cannot load my saved IBM i image.

I have a customer who reported an issue with a product running on IBM i (OS/400) on February 18th, 2014. After I reviewed the logfiles and the code, I made a change to the code and want to test it on IBM’s Power Development Cloud. Since February 19th, I have been waiting for IBM to fix the Power Development Cloud issue and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting and waiting

I have tried all of my contacts at IBM but have not reached the right person. I need to end this horror story. If anyone knows the name of the director or VP for IBM’s Power Development Cloud, please email it to me at roger.lacroix@capitalware.com or call me at +1-226-980-7307.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400) 1 Comment

MQTC v2.0.1.4 Gold Sponsor: Nastel Technologies

Capitalware would like to announce that Nastel Technologies is a Gold Sponsor of MQ Technical Conference v2.0.1.4 (MQTC).

Nastel Technologies will also be giving a technical session on AutoPilot® for WebSphere MQ.

For more information, please go to: http://www.mqtechconference.com

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Education, MQ, MQ Technical Conference Leave a comment

Cindy Gregoire, Doyle Reese and Allan Bartleywood will be Speaking at MQTC

Cindy Gregoire, Doyle Reese and Allan Bartleywood of TxMQ will be speaking at Capitalware’s MQ Technical Conference v2.0.1.4 (MQTC).

Here are the sessions they will be presenting at MQTC:

  • IIB Scalability & Performance Management by Cindy Gregoire of TxMQ
  • Leveraging DataPower to Secure & Deliver Mobility Use Cases by Doyle Reese of TxMQ
  • WebSphere MQ Just Works by Allan Bartleywood of TxMQ

Note: We will do our best to make sure that each session is presented twice.

For more information about MQTC, please go to:
http://www.mqtechconference.com

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Education, MQ, MQ Technical Conference Comments Off

It’s Alive!!

We have switch our primary domain name from capitalware.biz to capitalware.com and it’s alive. Yeah!! :)

Please update your bookmarks and address books.

1. For sales inquiries or to send corporate POs (purchase orders) to Capitalware Inc., please use the new email address of sales@capitalware.com

2. For support questions or product issues, please use the new email address of support@capitalware.com

Please let me know if you have any issues reaching our new web site or sending emails to us.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware Comments Off