Using Universal File Mover on Mac OS X (How To #6)

January 25, 2013 Roger Lacroix

In this blog posting, I will discuss the prerequisites and issue related to using UFM on Mac OS X. First, as with any other platform that you run UFM on, Mac OS X must have Java v1.5.0 or higher installed.

Second, follow the “Unix and Linux” instructions from the UFM Installation and Operation manual for installing UFM on Mac OS X.

Third, you will need WebSphere MQ (WMQ) to be installed on Mac OS X. This is where everyone gets stumped. IBM does not release either WMQ Server or WMQ Client for Mac OS X, although Willow Technology does have a port of WMQ for Mac OS X but it is extremely old (v5.3) and it is NOT free. I do not understand why IBM does not release a current version of WMQ for Mac OS X, as Mac OS X is a popular platform.

So, how do you get around this issue? You simply copy the MQ Java JAR files from an existing installation of WMQ to the “libs” directory of UFM on Mac OS X.

1) If you only have WMQ v6 then copy the following 2 MQ Java JAR files to Mac OS X:

com.ibm.mq.jar
connector.jar

2) If you have WMQ v7 then copy the following 5 MQ Java JAR files to Mac OS X:

com.ibm.mq.jar
com.ibm.mq.commonservices.jar
com.ibm.mq.headers.jar
com.ibm.mq.jmqi.jar
connector.jar

That’s it. You are now ready to use UFM on Mac OS X.

Note: If you want to use UFM on BSD or any other OS variant then simply follow the above steps and you will be good to go.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, Java, macOS (Mac OS X), Open Source, Universal File Mover Comments Off on Using Universal File Mover on Mac OS X (How To #6)

Scheduling a Simple File Transfer using Universal File Mover (How To #5)

January 21, 2013 Roger Lacroix

At mqseries.net, a user asked the following question:

I’m using MQ FTE version 7.0.4.1 , and I’d like to make a File Transfer to occur from Tuesday to Saturday , is it possible?

Since this is very easy to do in UFM, I thought I would make a ‘UFM How To’ from the question. I will show how to schedule a simple file transfer in “client mode” when UFM connects to the queue managers. Client mode connectivity means that the queue manager resides on a different server than the client application (i.e. UFM).


In this example, the following servers are used but they only have WebSphere MQ (WMQ) Client installed (no queue managers):
– aix002 is an AIX server with WMQ Client and UFM software installed
– linux002 is a Linux server with WMQ Client and UFM software installed

In this example, the following queue managers are used:
– MQA1 is a queue manager residing on a AIX (aix001) server (sender)
– MQL1 is a queue manager residing on a Linux (linux001) server (receiver)

– TEST.LINUX.QL and TEST.LINUX.QL.BK are local queues defined in queue manager MQL1 (receiver)
– TEST.LINUX.QR is a remote queue defined in queue manager MQA1 (sender)

If you do not know how to define/setup communication between 2 queue managers then follow the instructions in this blog posting:
https://www.capitalware.com/rl_blog/?p=509

Step #1: On the linux002 server, create a file in the mq directory called mql1.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_MQ SYSTEM "UFM_MQ.dtd">
<UFM_MQ>
    <QMgrName>MQL1</QMgrName>
    <QueueName>TEST.LINUX.QL</QueueName>
    <Hostname>linux001</Hostname>
    <ChannelName>SYSTEM.DEF.SVRCONN</ChannelName>
    <Port>1414</Port>
    <UserID>tester</UserID>
</UFM_MQ>

The mql1.xml (UFM_MQ XML) file describes how to connect to the remote MQL1 queue manager on server linux001. The connection will use UserID of tester. If you do not have that UserID defined on the linux001 server then use a UserID that is defined.

Step #2: On the linux002 server, in the UFM install directory, create a file called ufm_receive_test_5.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>

  <Actions>
    <MQReceive getwithconvert="Y" run="D">
      <MQ>
        <MQFile>mql1.xml</MQFile>
        <BackOutQName>TEST.LINUX.QL.BK</BackOutQName>
      </MQ>
      <Default>
         <Directory override="Y">/home/roger/UFM/</Directory>
      </Default>
    </MQReceive>
  </Actions>

</UFM_Workflow>

When UFM is started, it will run as a daemon (run=”D”) and use a backout queue called TEST.LINUX.QL.BK just in case there is an issue with a message. UFM will use ‘get with convert’ option when retrieving the messages. UFM will override the message’s specified directory and use the one provided. Either create /home/roger/UFM/ directory on your Linux server or use a directory that already exist on your Linux server.

Step #3: On the linux002 server, start UFM to receive the file transfers:

./ufm.sh ufm_receive_test_5.xml &

Step #4: On the aix002 server, create a file in the mq directory called mqa1.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_MQ SYSTEM "UFM_MQ.dtd">
<UFM_MQ>
    <QMgrName>MQA1</QMgrName>
    <QueueName>TEST.LINUX.QR</QueueName>
    <Hostname>aix001</Hostname>
    <ChannelName>SYSTEM.DEF.SVRCONN</ChannelName>
    <Port>1414</Port>
    <UserID>tester</UserID>
</UFM_MQ>

The mqa1.xml (UFM_MQ XML) file describes how to connect to the remote MQA1 queue manager on server aix001. The connection will use UserID of tester. If you do not have that UserID defined on the aix001 server then use a UserID that is defined.

Step #5: On the aix002 server, create a file in the data directory called test.txt and put a simple text message in the file (i.e. This is a test message.)

Step #6: On the AIX server, in the UFM install directory, create a file called ufm_send_test_5.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>

  <Actions>
    <MQSend delete="N" format="S">
      <File>data/test.txt</File>
      <MQ>
        <MQFile>mqa1.xml</MQFile>
      </MQ>
      <Remote>
        <Directory>/var/mqm/</Directory>
      </Remote>
    </MQSend>
  </Actions>

</UFM_Workflow>

When UFM is started, it will send the specified file and mark the message’s MQMD format as ‘string’.

Step #7: On the aix002 server, create a file in the schedule directory called schedule_5.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Schedule SYSTEM "UFM_Schedule.dtd">
<UFM_Schedule>
  <Schedule xmlfile="ufm_send_test_5.xml">
    <Minute>0</Minute>
    <Hour>5</Hour>
    <DayOfWeek>2,3,4,5,6</DayOfWeek>
  </Schedule>
</UFM_Schedule>

The schedule_5.xml (UFM_Schedule XML) file describes a scheduling event that will happen on Tuesday, Wednesday, Thursday, Friday and Saturday at 5:00AM each day. When the scheduling event fires, UFM will launch the specified UFM Workflow XML (i.e. ufm_send_test_5.xml). For the DayOfWeek XML element, the user can use numbers (i.e. 2, 3, etc.) or use the day of the week names (i.e. TUESDAY, WEDNESDAY, etc.).

Note: UFM_Schedule XML also supports a HolidayFile element that contains holidays when the scheduling event should NOT be handled.

Step #8: On the AIX server, in the UFM install directory, create a file called ufm_schedule_test_5.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>

  <Actions>
    <Schedule xmlfile="schedule_5.xml">
  </Actions>

</UFM_Workflow>

The UFM Workflow XML (i.e. ufm_schedule_test_5.xml) contains the Schedule action that the user wants UFM to process and the xmlfile attribute contains the UFM_Schedule XML file that holds all of the scheduling events.

Step #9: On the aix002 server, start UFM to handle the scheduling of events:

./ufm.sh ufm_schedule_test_5.xml

UFM will start and run forever processing scheduling events.

Step #10: On the linux002 server, verify that the test file (i.e. test.txt) was put into the /home/roger/UFM/ directory or whatever directory you specified in the ufm_receive_test_5.xml file.

Step #11: Finally, we need to stop UFM daemon that is running on the linux002 server. In the UFM install directory, create a file called ufm_putquit_test_5.xml and copy the following into the file:

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE UFM_Workflow SYSTEM "UFM_Workflow.dtd">
<UFM_Workflow>
  <Actions>
    <MQPutQuit>
      <MQ>
        <MQFile>mql1.xml</MQFile>
      </MQ>
    </MQPutQuit>
  </Actions>
</UFM_Workflow>

Step #12: On the linux002 server, run UFM with the MQPutQuit action:

./ufm.sh ufm_putquit_test_5.xml

This blog demonstrates how to use UFM at both the sender and receiver ends as MQ clients that connect to remote queue managers. The client-side uses UFM scheduling action to manage the sending of files via MQ.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Java, Linux, macOS (Mac OS X), Open Source, Universal File Mover, Unix, Windows Comments Off on Scheduling a Simple File Transfer using Universal File Mover (How To #5)

O’Reilly Giving Away Open Government Book

January 21, 2013 Roger Lacroix

O’Reilly is giving away electronic copies of the ‘Open Government’ book as a tribute to Aaron Swartz:
http://radar.oreilly.com/2013/01/open-government-files-released.html

In a world where web services can make real-time data accessible to anyone, how can the government leverage this openness to improve its operations and increase citizen participation and awareness? Through a collection of essays and case studies, leading visionaries and practitioners both inside and outside of government share their ideas on how to achieve and direct this emerging world of online collaboration, transparency, and participation.

Regards,
Roger Lacroix
Capitalware Inc.

E-Book, Education, Open Source Comments Off on O’Reilly Giving Away Open Government Book

3 Hours of Wasted Time That I’ll Never Get Back

January 17, 2013 Roger Lacroix

I have some code for outputting a buffer as a HEX dump like this:

000000: 00000000 00000000 00000000 00000000 ................
000010: 00000000 00000000 00000000 00000000 ................
000020: 00000000 00000000 00000000 00000000 ................

I have used this code for years on AIX, HP-UX, IBM i (OS/400), Linux, Solaris, Windows and z/OS without issue. I have compiled it for both 32-bit and 64-bit on various platforms without issue.

I recently purchased a new PC and last week I install Microsoft’s Visual C# and Visual C++ 2010 Express followed by Microsoft’s Windows SDK for Windows 7 and .NET Framework 4.

Today, I wanted to do some Windows 64-bit testing, so I rebuild the application using VS C++ 2010 and started testing. Everything was running along fine then I decided to turn on logging in a MQ exit and bam, I get the following error:

I scratched my head and thought, I do not have any source code called ‘isctype.c’ and the code is not using a C function called ‘ isctype’. So, I did the same test with the 32-bit version of the program it was fine. After swearing for a while, I started adding debug statements through the code and finally narrow it down to the following:

for (i=0; i < lenSrc; i++) pTgt[j+i] = isprint(pSrc[i]) ? pSrc[i] : '.';[/sourcecode] After some more debugging, I noticed that it crashed when the high-order bit was set. i.e. pSrc[i] is 'AA'. The VS C++ runtime was treating pSrc[i] as a signed integer. pSrc is typed as 'char *', so technically is it a signed char but who in the world deals with signed chararcters. So, after some searching on the internet, I found many other people with similar issue. The simplest solution is to cast pSrc[i] to unsigned char. [sourcecode language="c"]for (i=0; i < lenSrc; i++) pTgt[j+i] = isprint((unsigned char)pSrc[i]) ? pSrc[i] : '.';[/sourcecode] I don't know what's up with Microsoft's runtime for VS C++ 2010 but I wasted 3 hours of my life that I will never get back. Regards, Roger Lacroix Capitalware Inc.

C, Programming, Windows Comments Off on 3 Hours of Wasted Time That I’ll Never Get Back

Football Conference Championship Games

January 17, 2013 Roger Lacroix

Small Sliding scale picks for Conference Championship games (least probable to most probable):

F1   San Francisco 49ers  F3.5       at      Atlanta Falcons
F2   Baltimore Ravens                at      New England Patriots   F9.0

Fibonacci Sliding scale picks for Conference Championship games (least probable to most probable):

F1   San Francisco 49ers  F3.5       at      Atlanta Falcons
F2   Baltimore Ravens                at      New England Patriots   F9.0

So what does it mean?

F2   Baltimore Ravens                at      New England Patriots   F9.0

F2 means the favorite team will cover the spread with the highest probability of success.

Enjoy. 🙂 As they say in the stock market, past perform is no indication of future performance.

Regards,
Roger Lacroix

Football 1 Comment

CentOS 5.9 Released

January 17, 2013 Roger Lacroix

The CentOS Project has just released CentOS v5.9.
http://wiki.centos.org/Manuals/ReleaseNotes/CentOS5.9

CentOS is an Enterprise Linux distribution based on a rebuild of the freely available sources from Red Hat Enterprise Linux.

Regards,
Roger Lacroix
Capitalware Inc.

Linux, Open Source, Operating Systems Comments Off on CentOS 5.9 Released

Fedora 18 Released

January 15, 2013 Roger Lacroix

Fedora Project Contributors has just released Fedora v18.
http://docs.fedoraproject.org/en-US/Fedora/18/html/Release_Notes/index.html

Fedora is a Linux-based operating system, a suite of software that makes your computer run. You can use the Fedora operating system to replace or to run alongside of other operating systems such as Microsoft Windows or Mac OS X. The Fedora operating system is 100% free of cost for you to enjoy and share.

Regards,
Roger Lacroix
Capitalware Inc.

Linux, Open Source, Operating Systems Comments Off on Fedora 18 Released

Middleware Messaging FUD

January 14, 2013 Roger Lacroix

I was reading various articles today and I stumbled upon this article: AMQP: The new king of middleware messaging at SD Times. When I first read it, I started laughing so hard because I thought it was a joke. Then I realized it was not some sort of time-warped joke but rather just straight FUD (Fear, Uncertainty and Doubt).

I have been doing middleware messaging for businesses since 1989, yup, 24 years. This article is AT LEAST 2 decades too late to be relevant. The author, Angus Telfer, clearly has limited or no knowledge of the middleware messaging market for businesses and I do not know why he would write the article except for FUD or advertising.

Now for the full disclosure … I am a co-chair of the OASIS AMQP 1.0 Technical Committee, and the CTO of a company that uses AMQP in its main product

So, Angus Telfer is doing some self promotion. Here are some other gems:

The standardization of AMQP marks a turning point in the industry, a turning point from the past, consisting of a small number of enterprise-capable proprietary systems plus numerous, specific-purpose “roll your own” protocols, to the future, consisting of a standard messaging protocol used to interconnect products and services from a large number of vendors.

This is the sentence that made me laugh. This would be like me saying that I have launched a new social networking web site and claim that the small players have nothing on me but totally ignoring the 800 pound gorilla (Facebook). Yes, I am calling WebSphere MQ the 800 pound gorilla in the business middleware messaging market place. So, let’s talk about the 800 pound gorilla:

  • Gartner says that WebSphere MQ has roughly 72% of the middleware messaging software market
  • IBM has stated that over 15,000 customers worldwide are using WebSphere MQ
  • A couple of years ago, a senior IBM MQ person told me that 95% of the fortune 500 companies use WebSphere MQ.
  • IBM claims that WebSphere MQ now supports 80 different environments
  • What are the other 2 big proprietary middleware messaging products? Oracle AQ and MSMQ

Active support by JPMorgan, Bank of America, Deutsche Börse, Goldman Sachs, Credit Suisse, the US Department of Homeland Security, and other large end users guarantees it’s a standard that will be seen in the enterprise at the very least.

The standardization of AMQP signals the death knell for proprietary messaging systems. Large end users have signaled by their participation in the AMQP standards group-and their use of products developed with the protocol-that they are tired of being held hostage to proprietary messaging systems.

Yes, I have heard rumors that several of those companies are tired of paying fees to IBM for WebSphere MQ, so this is their revolution.

There are currently no competing standards that offer similar capabilities, enjoy similar support, and are application/broker independent.

Really!! Now that’s seriously funny. Angus Telfer doesn’t think that the 800 pound gorilla (WebSphere MQ) has more features, greater 3rd party vendor support and larger customer base? I have been helping Financial Services companies interconnect business applications (internally and externally) using WebSphere MQ since the 90’s and I have no clue what planet Angus Telfer is on.

So, where does that leave us? If you need a good laugh then have a read of this article. It is FUD just don’t get too bothered by it.

Am I a WebSphere MQ only guy – no. I follow both ActiveMQ and RabbitMQ closely. Both ActiveMQ and RabbitMQ appear to be the market leaders in the open source middleware messaging market and RabbitMQ seems to be gaining momentum.

I’m surprised that a technology magazine like SD Times would publish such FUD. As always with the internet, take what you read on the internet with a grain of salt.

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, Open Source Comments Off on Middleware Messaging FUD

New: MQ Message Encryption v2.0.0

January 10, 2013 Roger Lacroix

Capitalware Inc. would like to announce the official release of MQ Message Encryption v2.0.0.

MQ Message Encryption (MQME) provides encryption for MQ message data while it resides in a queue and in the MQ logs (i.e. all data at rest). In cryptography, encryption is the process of transforming information into an unreadable form (encrypted data). Decryption is the reverse process. It makes the encrypted information readable again. Only those with the key (PassPhrase) can successfully decrypt the encrypted data. MQME uses Advanced Encryption Standard (AES) to encrypt the data. AES is a data encryption scheme, adopted by the US government, that uses three different key sizes (128-bit, 192-bit, and 256-bit).

One of the features that MQME offers is the ability to control who accesses protected queues. Normally, the ‘mqm’, ‘QMQM’ or ‘MUSR_MQADMIN’ MQ UserIDs or any UserID in the ‘mqm’ group get full access to all messages in all queues. For queues protected by MQME, those privileged UserIds do NOT get access to the messages in the protected queues unless they are explicitly added to the authorized list of users or groups.

Another feature of MQME is its ability to generate and validate the message via a digital signature. MQME uses the SHA-2 to create a cryptographic hash function (digital signature) for the message data. The digital signature provides verification that the message data has not been altered.

MQME is an MQ API Exit that operates with WebSphere MQ v6.0, v7.0, v7.1 or v7.5 in Windows, Unix, IBM i (OS/400) and Linux platforms.

For more information about MQME, please go to:
https://www.capitalware.com/mqme_overview.html

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Encryption, Security, Unix, Windows Comments Off on New: MQ Message Encryption v2.0.0

Football Divisional Playoff Games

January 10, 2013 Roger Lacroix

Small Sliding scale picks for divisional playoff games (least probable to most probable):

U1   Green Bay Packers    F3.5       at      San Francisco 49ers
U2   Houston Texans       F9.5       at      New England Patriots
F3   Seattle Seahawks     F2.5       at      Atlanta Falcons
U4   Baltimore Ravens     F9.5       at      Denver Broncos

Fibonacci Sliding scale picks for divisional playoff games (least probable to most probable):

U1   Baltimore Ravens     F9.5       at      Denver Broncos
U2   Green Bay Packers    F3.5       at      San Francisco 49ers
U3   Houston Texans       F9.5       at      New England Patriots
F4   Seattle Seahawks     F2.5       at      Atlanta Falcons

So what does it mean?

U1   Green Bay Packers    F3.5       at      San Francisco 49ers

U1 means that underdog team will beat the spread with the lowest probability of success (don’t bet on it).

F4   Seattle Seahawks     F2.5       at      Atlanta Falcons

F4 means the favorite team will cover the spread with the highest probability of success.

Enjoy. 🙂 As they say in the stock market, past perform is no indication of future performance.

Regards,
Roger Lacroix

Football 1 Comment