Customer Requested Update to MQ Message Encryption

MQ Message Encryption (MQME) is a solution that provides encryption for MQ message data while it resides in a queue or topic and in the MQ logs (i.e. data at rest).

By default, when an application puts a message to a protected queue, MQME will encrypted the entire message data. If the application is putting messages that contain an embedded message (i.e. MQRFH, MQRFH2, MQCIH, etc.) then MQME will encrypt the entire message data which includes the MQ embedded headers. After MQME encrypts the entire message, it changes the MD.Format field from the current value to blanks (the MD.Format value is saved).

When the receiving application issues a get on the protected check (assuming it passes authorization checks), MQME will decrypt the message and reset the MD.Format field to its original value.

For 99.9% of applications, the above scenario works without any issues.

A JMS message (aka MQRFH2) has the following layout:

{RFH2 header}{mcd folder}{jms folder}{usr folder}{message payload}

Message properties (aka named properties) are stored in the ‘usr folder’.

MQ has special features/functionality for JMS messages and there are 2 situations where the above encryption scenario will not work:

  • If the protected queue has the attribute PROPCTL set to NONE
  • If the receiving application uses message selectors

1. When the queue’s PROPCTL attribute is set to NONE then MQ will strip off the RFH2 header and folders of a JMS message (aka MQRFH2) when the receiving application issues an MQGET. A useful feature for applications that cannot handle JMS messages.

2. JMS application can request a particular message from a queue by using message selectors. The message selector match on a value of a message property in the usr folder.
i.e.

String selector = "category='scifi'";
consumer = session.createConsumer(destination, selector);

The MQ internal code for handling those 2 features is processed before MQME is invoked for the MQGET operation which means MQ’s internal code cannot do either of those features because the message is encrypted.

I have asked IBM about moving the point at which those 2 features are processed to be after the API Exit (MQME) is invoked for MQXF_DATA_CONV_ON_GET. IBM has said no.

The reason I asked IBM to move the point at which those 2 features are handled is because some client applications put sensitive data in the message properties (aka named properties) in the usr folder of an MQRFH2. Yes, yes, I tell them all the time that it is a bad idea but it is what it is.

I have added a new keyword to MQME called EncryptRFH2Header. Its default value is set to ‘Y’ (Yes) which means it will retain existing behavior. When the EncryptRFH2Header keyword is set to ‘N’ then MQME will only encrypt the message payload of the JMS message and not encrypt the RFH2 header or any folders.

So, here’s the catch 22: if the application is using either the queue’s PROPCTL set to NONE or message selectors then the application MUST NOT put sensitive data in the message properties (aka named properties) because the usr folder will not be encrypted.

If anyone would like to test out the latest release then send the email to support@capitalware.com

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Encryption, Security, Unix, Windows Comments Off on Customer Requested Update to MQ Message Encryption

AMQ7234 Question from the MQ ListServer

Doug posted a question on MQ List Server regarding the MQ error message AMQ7234. His question was:

My shop has always struggled with large number of messages just being left on the queue. Sometimes to be processed later, sometimes because the application does not know what to do with them, but they cannot be removed. The following error “AMQ7234” is generated frequently about message being loaded onto the queue.

What I have observed, is that when this “loading” occurs all processing of MQSeries halts until the entire queue is read and loaded into memory. Sometimes this takes more than a couple of seconds causing delays in time sensitive responses to other queues for other applications. This queue manager is used, by executive mandate, by many applications – it would appear to me that this delay is the cause of other applications not meeting their performance SLA.

My reply was:

Now that’s an interesting problem. I had to Google the error messages because I have not see it before.

Here’s a good explanation: https://www.ibm.com/support/pages/amq7234-issued-periodically-wmq

It gives 4 solutions:

  • Avoid deep queues (MQ was not designed to be a database to keep messages for long periods of time)
  • Ensure that the queue is referenced very often by putting or getting messages
  • Keep an open handle on the queue so that the queue will not be unloaded to disk.
  • One possible way to do this would be to write a simple program that opens the queue for MQOO_INQUIRE, and then sleep indefinitely. The queue will be unloaded from memory if the last application accessing it has closed the queue. Therefore, if at least one application has the queue open, then the queue will not be unloaded from memory.

Pretty standard stuff but I do like the last solution. It is weird just like me. 🙂 But the only problem about sleeping forever is that if you try to stop the queue manager it may wait on the application. Hence, I would change it to do a Get and match on a crazy CorrelId. i.e. CorrelId = “Doug is an awesome MQAdm” (max 24 characters). Issue an MQGET with wait-forever and the “Fail if Quiescing” option.

Therefore, the queue will never to be unloaded and the problem is solved. Bonus: Set it up in as a ‘Server’ service in the queue manager then you never have to worry about it again.

I took one of my C sample programs and created a simple program called ‘GetMatchNone.c’. It opens a queue, performs a non-destructive get (browse) for a crazy Correlation Id and waits forever. You can download the program from here.

Here’s a sample MQSC service definition for GetMatchNone:

DEFINE SERVICE ('GetMatchNone') +
       DESCR('Keep QMgr from unloading the messages of this queue.') +
       STARTCMD('C:\Capitalware\Utils\GetMatchNone.exe') +
       STARTARG('TEST.Q1 +QMNAME+') +
       STOPCMD(' ') +
       STOPARG(' ') +
       STDOUT('C:\Capitalware\Utils\stdout.log') +
       STDERR('C:\Capitalware\Utils\stderr.log') +
       CONTROL(STARTONLY) +
       SERVTYPE(SERVER) +
       REPLACE
    where:

  • TEST.Q1 is the queue name that we don’t want messages to be unloaded.
  • C:\Capitalware\Utils\ is the directory where the executable is located and where the output files will be written to.
  • +QMNAME+ is an MQ environment variable for the name of the queue manager
  • Control is set to ‘STARTONLY’ because when the queue manager ends, the GetMatchNone program will gracefuly end.

Regards,
Roger Lacroix
Capitalware Inc.

C, HPE NonStop, IBM i (OS/400), IBM MQ, Linux, Programming, Unix, z/OS Comments Off on AMQ7234 Question from the MQ ListServer

MQ Visual Edit on Windows, macOS & Linux

I have mentioned this before, Capitalware has a fair number of users who run MQ Visual Edit on macOS.

I purchased and use Excelsior Jet for Windows, macOS and Linux. Excelsior Jet compiles and links Java code into an optimized C++ native executable. This means that the end-user does not need to run MQ Visual Edit in a VM (Virtual Machine) or in an emulator to use the product. It runs natively on Windows, macOS and Linux.

Here are 3 screen-shots of MQ Visual Edit running natively on Windows, macOS and Linux, all showing the same opened queue.

MQ Visual Edit on Windows (click image to see larger picture):

MQ Visual Edit on MacOS (click image to see larger picture):

MQ Visual Edit on Linux (SUSE) (click image to see larger picture):

So, people of the MQ world, do you need MQ tools that work on Windows, macOS and Linux? Capitalware has 3 MQ tools that can fill the void:

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, Linux, macOS (Mac OS X), Windows Comments Off on MQ Visual Edit on Windows, macOS & Linux

IBM: ‘Mac users are happier and more productive’

Here’s an interesting article over at ComputerWorld called: IBM: ‘Mac users are happier and more productive’

On first look (the critic in me), I’m thinking the survey must be paid for by Apple.

At Jamf Nation User Conference, IBM CIO Fletcher Previn said that IBM employees who use Macs are more likely to stay with IBM and exceed performance expectations compared to PC users. The article says:

  • There are 22% more macOS users who exceeded expectations in performance reviews, compared to Windows users.
  • High-value sales deals tend to be 16% larger for macOS users, compared to Windows users.
  • macOS users are 17% less likely to leave IBM, compared to those who use Windows.
  • MacOS users are happier with the third-party software availability within IBM — just 5% of macOS users ask for additional software, compared to 11% of Windows users.

I have a MacBook Pro. I have tried several times to use it as my main development machine but my brain just cannot get around the user interface differences. I am a ‘keyboard guy’ and not a ‘mouse guy’. I am just much faster using Windows rather than macOS.

Fletcher Previn also said:
IBM also observed that users found it easier to migrate from a previous version of Windows to a Mac than to upgrade older Windows systems to the latest version of Windows. IBM claims 98% of its Mac users said migration from Windows to macOS was easy, compared to 86% of people shifting from Windows 7 to Windows 10 who felt the same way.

Now that, I would TOTALLY agree with. Several weeks ago, I switched both my development desktop PC and laptop PC from Windows 7 Pro to Windows 10 Pro and it has been VERY frustrating to get use to where Microsoft has put everything. To me, it looks like Windows 10 was designed for novice users and all advanced features are hidden. Come on Microsoft, not everyone is a grandma/grandpa using Windows 10!! Plus there is a Windows 10 Home release for those people.

Also, did someone at Microsoft think everyone is partially blind? I’m using the same 2 Samsung monitors each at 1920 x 1200 with my Windows 10 PC that I used with my Windows 7 PC. On Windows 7’s desktop, I had 13 icons per column but on Windows 10’s desktop, the icons are larger and it can only show 11 icons per column. When I go into the ‘Display Settings’, under ‘Scale and Layout’, it is set to ‘100% (Recommended)’. The only values are larger!!! I cannot go down to 90% or 95%. Again, it feels like Microsoft designed Windows 10 for grandma/grandpa who don’t see very well. Ugh!

I’m still in the Windows 10 learning curve. Hopefully, I’ll get through it soon! 🙁

Regards,
Roger Lacroix
Capitalware Inc.

macOS (Mac OS X), Programming, Windows Comments Off on IBM: ‘Mac users are happier and more productive’

MQ Message Encryption Critical Bug Fixed

Recently, a customer reported a bug in MQ Message Encryption (MQME) for certain types of JMS messages (aka MQRFH2) where the MCA (amqrmppa) process would crash.

After some investigation, a buffer overrun condition was discovered that would cause the MCA (amqrmppa) process to crash. The bug has been fixed.

Please contact support@capitalware.com for the latest release of MQME.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Encryption, Security, Unix, Windows Comments Off on MQ Message Encryption Critical Bug Fixed

MQ Visual Edit V3.0.0.1 Released

The other day while doing some testing, I discovered a bug in MQ Visual Edit’s auto-load of a message when editing a message. I have fixed it and uploaded MQ Visual Edit v3.0.0.1 for all platforms.

As always, this is a FREE upgrade for ALL licensed users of MQ Visual Edit V2/V3.

For those you downloaded and install MQ Visual Edit v3.0.0 over the last couple of weeks, I strongly suggest that you un-install it then download and install MQ Visual Edit v3.0.0.1. Simply use the Capitalware download information from your order email.

Note: This is also applicable to MQ Visual Browse.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, IBM MQ Appliance, Linux, macOS (Mac OS X), MQ Visual Edit, Windows Comments Off on MQ Visual Edit V3.0.0.1 Released

Fedora 31 Released

Fedora Project has just released Fedora 31.
https://fedoramagazine.org/announcing-fedora-31/

Fedora is a Linux distribution developed by the community-supported Fedora Project and sponsored by Red Hat. Fedora contains software distributed under various free and open-source licenses and aims to be on the leading edge of such technologies. Fedora is the upstream source of the commercial Red Hat Enterprise Linux distribution.

Regards,
Roger Lacroix
Capitalware Inc.

Linux, Open Source, Operating Systems Comments Off on Fedora 31 Released

IBM MQ Fix Pack 9.0.0.8 Released

IBM has just released Fix Pack 9.0.0.8 for IBM MQ V9.0 LTS
https://www.ibm.com/support/pages/node/1089052?myns=swgws&mynp=OCSSYHRD

Regards,
Roger Lacroix
Capitalware Inc.

Fix Packs for MQ, IBM i (OS/400), IBM MQ, Linux, Unix, Windows Comments Off on IBM MQ Fix Pack 9.0.0.8 Released

MQ Visual Edit V3.0.0 Released

Capitalware Inc. would like to announce the official release of MQ Visual Edit v3.0.0. This is a FREE upgrade for ALL licensed users of MQ Visual Edit V3. MQ Visual Edit allows users to view, manipulate and manage messages in a queue and/or topic of a IBM MQ queue manager and presents the data in a simplified format similar to a database utility or spreadsheet program.

For more information about MQ Visual Edit go to:
https://www.capitalware.com/mqve_overview.html

    Changes for MQ Visual Edit v3.0.0:

  • Added a new MQ tool called Visual Queue Statistics. It allows the user to visually display the queue statistic values in a chart.
  • Added a new MQ tool called Visual Subscription Rate. It allows the user to visually display the subscription’s message count in a chart.
  • Added a dialog window to retrieve the context value for ‘Put Options’ for the Restore Queue/Topic feature.
  • Added Context for ‘Put Options’ on the Open Queue/Topic window for Put Server, SIM Client, SIM Server and Publish Server.
  • Added a new viewer on the Message Edit window called: Fixed Width Viewer. Clicking it will parse the message data into a spreadsheet display.
  • Added a new viewer on the Message Edit window called: CSV Viewer. Clicking it will parse the message data into a spreadsheet display.
  • Added a new viewer on the Message Edit window called: FIX (Financial Information eXchange) Viewer. Clicking it will parse the message data into a spreadsheet display.
  • Added a wizard so that the user can quickly create and save Fixed Width format structure, CSV column names and FIX column names.
  • Added a new options on the Preferences’ Edit Window called: ‘For CSV Viewer, use first row for column name’, drop selector for field delimiter and a drop selector for record separator.
  • Added a new options on the Preferences’ Edit Window called: ‘For FIX Viewer, use first row for column name’ and drop selector for delimiter, drop selector for field delimiter and a drop selector for record separator.
  • Added the ability to associate a time zone with a queue manager. Hence, the message’s date and time will be in the specified time zone.
  • Changed Queue Manager Access Profile, Group and Favorites information to be stored in a SQLite database
  • Added code to automatically migrate CommProfileDB, CommProfileGroupDB and FavoritesDB property files to a SQLite database.
  • For a fresh install of MQ Visual Edit (Windows and Linux only), added code to automatically collect local queue manager information and display it to the user for importing.
  • Added the ability to import queue manager settings from MQ Explorer.
  • Added the ability to import queue manager settings from MO71.
  • Added the ability to import queue manager settings from MQ Visual Edit V1/V2 property file.
  • Added the ability to import queue manager settings from a CCDT file.
  • Added the ability to import local queue managers.
  • Fixed an issue with resetting the view correctly and also when changing the language.
  • Added launching a web browser from the Registration window for the user to register the license key.
  • New location for SQLite database and the MQVE logfile. They will now be stored in {home_directory}\Capitalware\ directory. i.e. For Windows: C:\Users\{UserId}\Capitalware\ and for Linux/macOS: /home/{UserId}/Capitalware/
  • The MQ Visual Edit logfile will now be called: MQVE.log (rather than MQVE_error.log).
  • Moved the MQ Server and Monitor tools to their own menu dropdown called: MQ Tools
  • Enhanced the error message regarding an expired license key
  • Updated code to better handle “https” call to capitalware.com for registration
  • Updated code to set JVM UserId because MQ client library uses it when USE_MQCSP_AUTHENTICATION_PROPERTY is set to true
  • Increased the size of the buttons on the windows, so people with Microsoft Surface tablets will find it easier to click buttons with their fingers.
  • Added icons to all of the buttons on the windows. The icons for the buttons match the icons for the menu items. Hence, it makes for a better and more consistent look and feel.
  • Fixed an issue with parsing PCF messages and displaying them.
  • Fixed an issue with Queue Monitor tool and connecting to a z/OS queue manager.
  • Fixed a bug on Message Edit window where Wrap Text menu item was not honoring user Preferences.
  • Fixed a bug on Message Edit window where Replace menu item was not being enabled/disabled correctly.
  • Switched from Apache Xerces to Java’s builtin XML parser.
  • Updated docs (English only)

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, IBM MQ Appliance, Linux, macOS (Mac OS X), MQ Visual Edit, Windows Comments Off on MQ Visual Edit V3.0.0 Released

MQ Visual Browse V3.0.0 Released

Capitalware Inc. would like to announce the official release of MQ Visual Browse v3.0.0. This is a FREE upgrade for ALL licensed users of MQ Visual Browse V3. MQ Visual Browse allows users to view messages in a queue and/or topic of a IBM MQ queue manager and presents the data in a simplified format similar to a database utility or spreadsheet program.

For more information about MQ Visual Browse go to:
https://www.capitalware.com/mqvb_overview.html

    Changes for MQ Visual Browse v3.0.0:

  • Added a new viewer on the Message Edit window called: Fixed Width Viewer. Clicking it will parse the message data into a spreadsheet display.
  • Added a new viewer on the Message Edit window called: CSV Viewer. Clicking it will parse the message data into a spreadsheet display.
  • Added a new viewer on the Message Edit window called: FIX (Financial Information eXchange) Viewer. Clicking it will parse the message data into a spreadsheet display.
  • Added a wizard so that the user can quickly create and save Fixed Width format structure, CSV column names and FIX column names.
  • Added a new options on the Preferences’ Edit Window called: ‘For CSV Viewer, use first row for column name’, drop selector for field delimiter and a drop selector for record separator.
  • Added a new options on the Preferences’ Edit Window called: ‘For FIX Viewer, use first row for column name’ and drop selector for delimiter, drop selector for field delimiter and a drop selector for record separator.
  • Added the ability to associate a time zone with a queue manager. Hence, the message’s date and time will be in the specified time zone.
  • Changed Queue Manager Access Profile, Group and Favorites information to be stored in a SQLite database
  • Added code to automatically migrate CommProfileDB, CommProfileGroupDB and FavoritesDB property files to a SQLite database.
  • For a fresh install of MQ Visual Browse (Windows and Linux only), added code to automatically collect local queue manager information and display it to the user for importing.
  • Added the ability to import queue manager settings from MQ Explorer.
  • Added the ability to import queue manager settings from MO71.
  • Added the ability to import queue manager settings from MQ Visual Browse V1/V2 property file.
  • Added the ability to import queue manager settings from a CCDT file.
  • Added the ability to import local queue managers.
  • Fixed an issue with resetting the view correctly and also when changing the language.
  • Added launching a web browser from the Registration window for the user to register the license key.
  • New location for SQLite database and the MQVB logfile. They will now be stored in {home_directory}\Capitalware\ directory. i.e. For Windows: C:\Users\{UserId}\Capitalware\ and for Linux/macOS: /home/{UserId}/Capitalware/
  • The MQ Visual Browse logfile will now be called: MQVB.log (rather than MQVB_error.log).
  • Enhanced the error message regarding an expired license key
  • Updated code to better handle “https” call to capitalware.com for registration
  • Updated code to set JVM UserId because MQ client library uses it when USE_MQCSP_AUTHENTICATION_PROPERTY is set to true
  • Increased the size of the buttons on the windows, so people with Microsoft Surface tablets will find it easier to click buttons with their fingers.
  • Added icons to all of the buttons on the windows. The icons for the buttons match the icons for the menu items. Hence, it makes for a better and more consistent look and feel.
  • Fixed an issue with parsing PCF messages and displaying them.
  • Fixed a bug on Message Edit window where Wrap Text menu item was not honoring user Preferences.
  • Fixed a bug on Message Edit window where Replace menu item was not being enabled/disabled correctly.
  • Switched from Apache Xerces to Java’s builtin XML parser.
  • Updated docs (English only)

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, IBM MQ Appliance, Linux, macOS (Mac OS X), MQ Visual Browse, Windows Comments Off on MQ Visual Browse V3.0.0 Released