Windows Print Screen

Ok. Here’s something not related to MQ or programming. 🙂

Everyday, someone sends Capitalware a support request with an image or two. Its not a problem. I review what they asking and send back a reply.

Most of the images I receive are some sort of cropped image of the whole desktop. Sometimes the image is tightly cropped to whatever they are sending and sometimes I’m getting too much information about their desktop.

A far better way of sending Capitalware a screen shot of information is to use one of the following 2 options:

(1) Bring the window to the foreground (so that it is active) then press ALT-PrtScn. i.e. Hold the ALT key then press the PrtScn (Print Screen) key. Windows will capture an image of ONLY the active window and place it in the clipboard and then the user can paste it into an email.

(2) Use the Snipping Tool that has been available since Windows Vista. Start it, select the Mode (i.e. Rectangular Snip), click New button and select whatever you want then paste it into the email.

Note: None of the images are shared with anyone or other companies but it is far, far better for people to only send Capitalware information that I need to see.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Windows Comments Off on Windows Print Screen

MQ Internals Explained (a little bit)

IBM likes to market IBM MQ (aka WebSphere MQ, MQSeries) as a simple and complete solution for business’s messaging needs. Going with the KISS approach is wise (very wise in some cases).

IBM talks about the 2 different messaging models that IBM MQ supports: Point-to-Point (P2P) and Publish/Subscribe (Pub/Sub).

    IBM will:

  • show you how to put messages to local/alias/cluster/remote queues (P2P) or get messages from local/alias/cluster queues (P2P).
  • show you how to publish messages to a topic string or object (Pub/Sub) or subscribe to a subscription (or topic) to retrieve messages from a topic string or object (Pub/Sub).
  • show you how to create an alias queue that can point to (1) a local queue (P2P) or (2) a topic (Pub/Sub) and if you use a topic, how to cross messages from the P2P model to the Pub/Sub model.
  • show you how to create a subscription that can point to (1) a local queue (P2P) or (2) a topic (Pub/Sub) and if you use a local or alias queue, how to cross messages from the Pub/Sub model to the P2P model.

IBM likes to show the world that IBM MQ use of Point-to-Point and Publish/Subscribe is a single messaging model. While IBM markets IBM MQ as a single messaging model is fine (aka the KISS approach), it sometimes makes my life a little difficult when explaining either MQ Message Encryption or MQ Auditor to customers (although, less so with MQ Auditor, because by default, it collects information from both messaging models).

Here’s an example: A customer recently sent me a picture of a setup they were trying to implement with MQ Message Encryption (MQME) and they were confused why MQME did not do its job of encrypting messages when the messages were put/published at different points in the setup.

Here’s the picture of their setup:

For those who prefer MQSC definitions, here they are:

DEFINE QALIAS('TEST_ALIAS') TARGET('TEST_TOPIC') TARGTYPE(TOPIC)
DEFINE QREMOTE('TEST_QUEUE_REM') RQMNAME('QMGR2') RNAME('TEST_QUEUE') XMITQ('QMGR2')
DEFINE QLOCAL('TEST_QUEUE_LOC')
DEFINE TOPIC('TEST_TOPIC') TOPICSTR('MQME/TEST')
DEFINE SUB('TEST_SUB_1') TOPICSTR('') TOPICOBJ('TEST_TOPIC') DEST('TEST_QUEUE_REM')
DEFINE SUB('TEST_SUB_2') TOPICSTR('') TOPICOBJ('TEST_TOPIC') DEST('TEST_QUEUE_LOC')

When they put a message to the alias queue, everything was fine but when they published a message to the topic ‘MQME/TEST’ then the message was not encrypted.

In the user’s MQME IniFile, they only had set the protected queue(s) of ‘TEST*’. They did NOT have any protected topic definitions.

There are many things I need to explain that goes on under the covers with MQ. I will do my best to explain it in a logical order but I know that I’m about to confuse many people.

Things to remember:

(1) When messages flow through P2P model/environment, the message can be a plain message, JMS message (aka MQRFH2), CICS message, IMS message, SAP message, etc.

Layout for 5 example messages for P2P:

{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}
{CICS header}{message payload}
{IMS header}{message payload}
{SAP header}{message payload}

(2) When messages flow through Pub/Sub model/environment, the message MUST be a JMS message (aka MQRFH2) formatted message. Now a JMS message can contain a CICS message, IMS message, SAP message, etc.

Layout for 4 example messages for Pub/Sub:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{CICS header}{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{IMS header}{message payload}
{RFH2 header}{mcd folder}{jms folder}{usr folder}{SAP header}{message payload}

Note: The user can change the queue’s property attribute (PROPCTL) or subscription’s property attribute (PSPROP) to ‘NONE’, so that MQ removes the message properties (aka named properties) and folders when the message is retrieved by the application.

Item #1:
When an application opens an alias queue, internally that object type is called MQOT_Q. Hence, MQ is operating in the P2P model. Since, it is the P2P model, MQME will check its protect queue definitions for the a match. If MQME finds a match then all messages put to or retrieved from that object handle (aka opened queue) will be encrypted or decrypted (assuming permissions are correct).

When the user ran an application to publish a message to topic ‘MQME/TEST’, the topic was opened and the object type was MQOT_TOPIC. Since, it is the Pub/Sub model, MQME will check its protect topic definitions for the a match. In this particular case, MQME did not find a match, so it will not encrypt the messages or decrypt the messages for subscribers.

Now if the user adds a protect topic definition to the MQME IniFile with something like ‘MQME/TEST’ or ‘MQME/*’ or ‘MQME/TEST*’ then MQME would find a match and encrypt or decrypt messages of that particular topic.

Here’s the analogy that I like to use with customers, think of IBM MQ as a brain (human brain if you will). Each person has a single brain. IBM MQ provides a single messaging infrastructure. But if you look closely at the brain, you will see that it is made up of 2 hemispheres and if you look closely, you will see that IBM MQ is made up of 2 different messaging models: Point-to-Point and Publish/Subscribe.

So, how an application opens a queue or topic will dictate which side of the brain (aka messaging system) MQ will use to handle the getting or putting of messages. And if companies have a configuration where messages flow back and forth between the 2 messaging systems then the users of MQME need to beware of the point of contact with the MQ (aka the brain).

Item #2:
Did you know that when a message hops between P2P and Pub/Sub that the message actually changes! Also, for those of you that are using MQ’s Pub/Sub for message replication, did you know that not all MQMD fields of the original message are replicated to the copied messages!

Example:

Using the above customer setup, if you start MQ Explorer and put a test message to the alias queue ‘TEST_ALIAS’ then the following is what happens to the message:

– Putting a plain message to an alias queue

{message payload}

– When MQ transfers the message to a topic object or subscription then the message becomes

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

– When the message is put to the XMITQ queue it looks like

{XMIT header}{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

– When it is put to the local queue of the local queue manager, it is still in the format:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

– When it is put to the local queue of the remote queue manager, it is still in the format:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

Note: For those newbies out there, MQRFH2 is the internal name for a JMS message. Also, yes, I know on the subscription, the user can change the Property attribute (PSPROP) to ‘NONE’, so that MQ removes the message properties (aka named properties) which in theory turns the message back to a regular message.

Users of MQ should really be aware of what goes on under the covers with MQ, so that they don’t blame MQ when they don’t get the results they want.

Also, there are far simpler solutions for handling message replication. i.e. MQ Message Replication

If you want, you can sub-title this blog posting as ‘your brain on MQ!’ 🙂

Regards,
Roger Lacroix
Capitalware Inc.

.NET, Assembler, C, C#, C++, HPE NonStop, IBM i (OS/400), IBM MQ, Java, JMS, Linux, macOS (Mac OS X), MQ Auditor, MQ Message Encryption, Programming, Unix, Windows, z/OS 2 Comments

MQME Issue with ExcludeQueue keyword on an MQPUT1 API Call

Late last week, a customer discovered an obscure issue in MQ Message Encryption (MQME) with the ExcludeQueue keyword on an MQPUT1 API call.

I have since fixed it but here is what I mean by obscure:

– If the user has wild-carded their protected queue setup i.e. ‘TEST.*’
– And then set the ExcludeQueue keyword value to ‘TEST.FRED’ because they want all ‘TEST.*’ queues to have their messages encrypted EXCEPT for queue ‘TEST.FRED’
– There was a bug in MQME that it did not successfully handle the values for ExcludeQueue keyword on an MQPUT1 API call.
– Hence, the messages for queue ‘TEST.FRED’ would be encrypted.
– And this only happened for MQPUT1 API calls – MQPUT API calls were correctly handled.

This is an obscure but supported setup. Most customers either explicitly set the queue name for a protect queue (i.e. ‘TEST.BARNEY’) or they wildcard it (i.e. TEST.*) but they generally don’t wildcard the protected queues then exclude a single queue. Note: The user could actually exclude queues with a wildcard.

This too is a supported setup:
– Have protect queues of ‘TEST.*’
– And exclude queues of ‘TEST.FRED.*’
– Or the user could exclude queues for ‘TEST.FRED.*’ and ‘TEST.WILMA.*’

Hence, any queue listed in the ExcludeQueue would NOT have their messages encrypted.

If anyone would like to test out the latest release then send the email to support@capitalware.com

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Encryption, Security, Unix, Windows Comments Off on MQME Issue with ExcludeQueue keyword on an MQPUT1 API Call

wc3270 and IBM RDP Issue

I use IBM’s Remote Development Program (RDP) to host my z/OS and Linux for System z LPARs. I installed the latest release of wc3270 of the x3270 project on my new Windows 10 PC and copied my settings from my old Windows 7 PC. Hence, I should be good to go but no.

I have been getting the following error and at first I thought maybe IBM was re-configuring their Dallas data center (which they do from time to time):

SSL: InitializeSecurityContext: error 0x80090322 (The target principal name is incorrect.)

I checked the online documentation for IBM’s RDP and the IP address (requires TLS/SSL tunnel) for the master VM is still the same. I checked my wc320 session file for corruption and it was fine. I compared the old release of wc3270 wizard with the current release and I noticed a new option called “10. Verify host certificates”. It is set to “Yes” by default. I decided set it to “No” and try the connection again. Surprise, I got a successful connection.

I don’t know when the option “Verify host certificates” was added to wc3270 but if you upgrade wc3270 and start getting the above error messages then set “Verify host certificates” to “No”.

Or you can edit your wc3270 session file(s) and add the following line:

wc3270.verifyHostCert: false

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Open Source, Security, Windows, z/OS Comments Off on wc3270 and IBM RDP Issue

IBM MQ Fix Pack 9.1.0.4 Released

IBM has just released Fix Pack 9.1.0.4 for IBM MQ V9.1 LTS:
https://www.ibm.com/support/pages/node/1119051

Regards,
Roger Lacroix
Capitalware Inc.

Fix Packs for MQ, IBM MQ, Linux, Unix, Windows Comments Off on IBM MQ Fix Pack 9.1.0.4 Released

IBM MQ V9.1.4 Announced

IBM has announced IBM MQ V9.1.4 for Multiplatforms:
https://www.ibm.com/common/ssi/ShowDoc.wss?docURL=/common/ssi/rep_ca/7/897/ENUS219-487/index.html
Planned availability for IBM MQ V9.1.4 is December 5, 2019 for Electronic software delivery.

IBM MQ (aka WebSphere MQ) homepage
https://www.ibm.com/products/mq

Regards,
Roger Lacroix
Capitalware Inc.

Fix Packs for MQ, IBM MQ, IBM MQ Appliance, Linux, Unix, Windows, z/OS Comments Off on IBM MQ V9.1.4 Announced

MQ Batch Toolkit v3.2.0 Released

Capitalware Inc. would like to announce the official release of MQ Batch Toolkit v3.2.0. This is a FREE upgrade for ALL licensed users of MQ Batch Toolkit. MQ Batch Toolkit allows users to manipulate, monitor and manage messages in a queue of an IBM MQ (formally WebSphere MQ & MQSeries) queue manager from a command-line or shell scripting environment.

For more information about MQ Batch Toolkit go to:
https://www.capitalware.com/mqbt_overview.html

    Changes for MQ Batch Toolkit v3.2.0:

  • Updated the QList and TopicList function to have ‘-S’ parameter that will allow SYSTEM queues or topics to be included in the output list
  • Fixed an issue with parsing event (PCF) messages and displaying them.
  • Updated SSL/TLS support
  • Enhanced the error message regarding an expired license key
  • Added warning message that the trial-only release cannot be registered.
  • Updated docs (English only)

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, Linux, macOS (Mac OS X), MQ Batch Toolkit, Windows Comments Off on MQ Batch Toolkit v3.2.0 Released

Java on Windows 10 Issue: “Could not open/create prefs root node”

I was getting the following error message when I would run a Java application on my new Windows 10 Pro PC:

Nov 21, 2019 3:57:37 PM java.util.prefs.WindowsPreferences <init>
WARNING: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKeyEx(...) returned error code 5.

I searched the internet regarding the error message, people say it is a known bug and you need to manually create the following Windows registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Prefs

When I opened the RegEdit, I found that the key already exists.

After a little more searching on the internet, I came across another comment that says you also need the following Windows registry key:

HKEY_LOCAL_MACHINE\Software\WOW6432Node\JavaSoft\Prefs

So, I opened the RegEdit and found the key did not exist. I created it and then the error message went away.

Hence, for future users who encounter this error message, create a text file called javasoft_prefs.reg and put the following in it:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\JavaSoft\Prefs]

[HKEY_LOCAL_MACHINE\Software\WOW6432Node\JavaSoft\Prefs]

Now start RegEdit and click File – > Import and select the javasoft_prefs.reg file and the error message will go away.

Or you can run RegEdit from the Command Line and pass the file name as a parameter:

regedit  javasoft_prefs.reg

Regards,
Roger Lacroix
Capitalware Inc.

Java, Programming, Windows Comments Off on Java on Windows 10 Issue: “Could not open/create prefs root node”

Customer Requested Update to MQ Message Encryption

MQ Message Encryption (MQME) is a solution that provides encryption for MQ message data while it resides in a queue or topic and in the MQ logs (i.e. data at rest).

By default, when an application puts a message to a protected queue, MQME will encrypted the entire message data. If the application is putting messages that contain an embedded message (i.e. MQRFH, MQRFH2, MQCIH, etc.) then MQME will encrypt the entire message data which includes the MQ embedded headers. After MQME encrypts the entire message, it changes the MD.Format field from the current value to blanks (the MD.Format value is saved).

When the receiving application issues a get on the protected check (assuming it passes authorization checks), MQME will decrypt the message and reset the MD.Format field to its original value.

For 99.9% of applications, the above scenario works without any issues.

A JMS message (aka MQRFH2) has the following layout:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

Message properties (aka named properties) are stored in the ‘usr folder’.

MQ has special features/functionality for JMS messages and there are 2 situations where the above encryption scenario will not work:

  • If the protected queue has the attribute PROPCTL set to NONE
  • If the receiving application uses message selectors

1. When the queue’s PROPCTL attribute is set to NONE then MQ will strip off the RFH2 header and folders of a JMS message (aka MQRFH2) when the receiving application issues an MQGET. A useful feature for applications that cannot handle JMS messages.

2. JMS application can request a particular message from a queue by using message selectors. The message selector match on a value of a message property in the usr folder.
i.e.

String selector = "category='scifi'";
consumer = session.createConsumer(destination, selector);

The MQ internal code for handling those 2 features is processed before MQME is invoked for the MQGET operation which means MQ’s internal code cannot do either of those features because the message is encrypted.

I have asked IBM about moving the point at which those 2 features are processed to be after the API Exit (MQME) is invoked for MQXF_DATA_CONV_ON_GET. IBM has said no.

The reason I asked IBM to move the point at which those 2 features are handled is because some client applications put sensitive data in the message properties (aka named properties) in the usr folder of an MQRFH2. Yes, yes, I tell them all the time that it is a bad idea but it is what it is.

I have added a new keyword to MQME called EncryptRFH2Header. Its default value is set to ‘Y’ (Yes) which means it will retain existing behavior. When the EncryptRFH2Header keyword is set to ‘N’ then MQME will only encrypt the message payload of the JMS message and not encrypt the RFH2 header or any folders.

So, here’s the catch 22: if the application is using either the queue’s PROPCTL set to NONE or message selectors then the application MUST NOT put sensitive data in the message properties (aka named properties) because the usr folder will not be encrypted.

If anyone would like to test out the latest release then send the email to support@capitalware.com

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Encryption, Security, Unix, Windows Comments Off on Customer Requested Update to MQ Message Encryption

AMQ7234 Question from the MQ ListServer

Doug posted a question on MQ List Server regarding the MQ error message AMQ7234. His question was:

My shop has always struggled with large number of messages just being left on the queue. Sometimes to be processed later, sometimes because the application does not know what to do with them, but they cannot be removed. The following error “AMQ7234” is generated frequently about message being loaded onto the queue.

What I have observed, is that when this “loading” occurs all processing of MQSeries halts until the entire queue is read and loaded into memory. Sometimes this takes more than a couple of seconds causing delays in time sensitive responses to other queues for other applications. This queue manager is used, by executive mandate, by many applications – it would appear to me that this delay is the cause of other applications not meeting their performance SLA.

My reply was:

Now that’s an interesting problem. I had to Google the error messages because I have not see it before.

Here’s a good explanation: https://www.ibm.com/support/pages/amq7234-issued-periodically-wmq

It gives 4 solutions:

  • Avoid deep queues (MQ was not designed to be a database to keep messages for long periods of time)
  • Ensure that the queue is referenced very often by putting or getting messages
  • Keep an open handle on the queue so that the queue will not be unloaded to disk.
  • One possible way to do this would be to write a simple program that opens the queue for MQOO_INQUIRE, and then sleep indefinitely. The queue will be unloaded from memory if the last application accessing it has closed the queue. Therefore, if at least one application has the queue open, then the queue will not be unloaded from memory.

Pretty standard stuff but I do like the last solution. It is weird just like me. 🙂 But the only problem about sleeping forever is that if you try to stop the queue manager it may wait on the application. Hence, I would change it to do a Get and match on a crazy CorrelId. i.e. CorrelId = “Doug is an awesome MQAdm” (max 24 characters). Issue an MQGET with wait-forever and the “Fail if Quiescing” option.

Therefore, the queue will never to be unloaded and the problem is solved. Bonus: Set it up in as a ‘Server’ service in the queue manager then you never have to worry about it again.

I took one of my C sample programs and created a simple program called ‘GetMatchNone.c’. It opens a queue, performs a non-destructive get (browse) for a crazy Correlation Id and waits forever. You can download the program from here.

Here’s a sample MQSC service definition for GetMatchNone:

DEFINE SERVICE ('GetMatchNone') +
       DESCR('Keep QMgr from unloading the messages of this queue.') +
       STARTCMD('C:\Capitalware\Utils\GetMatchNone.exe') +
       STARTARG('TEST.Q1 +QMNAME+') +
       STOPCMD(' ') +
       STOPARG(' ') +
       STDOUT('C:\Capitalware\Utils\stdout.log') +
       STDERR('C:\Capitalware\Utils\stderr.log') +
       CONTROL(STARTONLY) +
       SERVTYPE(SERVER) +
       REPLACE
    where:

  • TEST.Q1 is the queue name that we don’t want messages to be unloaded.
  • C:\Capitalware\Utils\ is the directory where the executable is located and where the output files will be written to.
  • +QMNAME+ is an MQ environment variable for the name of the queue manager
  • Control is set to ‘STARTONLY’ because when the queue manager ends, the GetMatchNone program will gracefuly end.

Regards,
Roger Lacroix
Capitalware Inc.

C, HPE NonStop, IBM i (OS/400), IBM MQ, Linux, Programming, Unix, z/OS Comments Off on AMQ7234 Question from the MQ ListServer