Enhancement to Exporting MQ Objects into Individual Files

Yesterday, I wrote a blog item regarding Exporting MQ Objects into Individual Files.

I forgot that the dmpmqcfg program can output the MQ Object information as 1 line rather than spanning multiple lines.

Hence, if we let dmpmqcfg do that work then we can simplify the Rexx script to only output the various MQ objects to their own file.

Example of running dmpmqcfg against your queue manager called MQA1 such that it outputs each MQ object as 1 line:

dmpmqcfg -m MQA1 -a -o 1line > MQA1.all.mqsc

Then you run it against my REXX script called fmt_dmpmqcfg.rex:

/* Format output from dspmqcfg command to a single line for each entry.  */
/* Put each MQ object type into a different file.                        */
/*                                                                       */
/* Example dspmqcfg command:                                             */
/*    dmpmqcfg -m MQA1 -a -o 1line > MQA1.all.mqsc                       */

trace o
Parse Arg inFN .

if inFN = "" then
do
   Say "Invalid number of parameters."
   Say "i.e."
   Say "     fmt_dmpmqcfg.rex input_file_name"
   Exit
end

types = "QMGR CHANNEL LISTENER NAMELIST PROCESS QALIAS QLOCAL QMODEL QREMOTE SERVICE SUB TOPIC AUTHREC AUTHINFO CHLAUTH COMMINFO"


/* open file */
Call Stream inFN,'C','OPEN READ'

/* Delete previous output and then open it. */
do i=1 to Words(types)
   ptr = Word(types,i)
   outFN = inFN||"."||ptr||".mqsc"
   if (STREAM( outFN, 'C', 'QUERY EXIST' ) <> "") then
      "ERASE  "outFN
   Call Stream outFN,'C','OPEN WRITE'
   counts.ptr = 0
end

total = 0
do until Lines(inFN) = 0

   inLine = Strip(LineIn(inFN))   /* read next line */

   if (inLine = "") then
      NOP   /* blank line, forget-about-it */
   else if (SubStr(inLine,1,1) = "*") then
      NOP   /* comment line, forget-about-it */
   else
   do
      ptr = Word(inLine, 2)

      if (Pos("(", ptr) > 0) then
         ptr = SubStr(ptr, 1, Pos("(", ptr) - 1)

      nothing = LineOut(inFN||"."||ptr||".mqsc", inLine)   /* write to the file  */

      counts.ptr = counts.ptr + 1
      total = total + 1
   end
end

/* close the files  */
Call Stream inFN,'C','CLOSE'
do i=1 to Words(types)
   ptr = Word(types,i)
   outFN = inFN||"."||ptr||".mqsc"
   Call Stream outFN,'C','CLOSE'
   Say outFN || " contains " || counts.ptr || " MQSC formatted commands."
end

Say ""
Say "Total formatted MQSC commands was:" total

Exit

To run the Rexx script, the command is:

rexx fmt_dmpmqcfg.rex  MQA1.all.mqsc

The Rexx script will output:

MQA1.all.mqsc.QMGR.mqsc contains 1 MQSC formatted commands.
MQA1.all.mqsc.CHANNEL.mqsc contains 43 MQSC formatted commands.
MQA1.all.mqsc.LISTENER.mqsc contains 5 MQSC formatted commands.
MQA1.all.mqsc.NAMELIST.mqsc contains 3 MQSC formatted commands.
MQA1.all.mqsc.PROCESS.mqsc contains 2 MQSC formatted commands.
MQA1.all.mqsc.QALIAS.mqsc contains 4 MQSC formatted commands.
MQA1.all.mqsc.QLOCAL.mqsc contains 111 MQSC formatted commands.
MQA1.all.mqsc.QMODEL.mqsc contains 7 MQSC formatted commands.
MQA1.all.mqsc.QREMOTE.mqsc contains 11 MQSC formatted commands.
MQA1.all.mqsc.SERVICE.mqsc contains 6 MQSC formatted commands.
MQA1.all.mqsc.SUB.mqsc contains 1 MQSC formatted commands.
MQA1.all.mqsc.TOPIC.mqsc contains 8 MQSC formatted commands.
MQA1.all.mqsc.AUTHREC.mqsc contains 438 MQSC formatted commands.
MQA1.all.mqsc.AUTHINFO.mqsc contains 4 MQSC formatted commands.
MQA1.all.mqsc.CHLAUTH.mqsc contains 3 MQSC formatted commands.
MQA1.all.mqsc.COMMINFO.mqsc contains 1 MQSC formatted commands.

Total formatted MQSC commands was: 648

As I said yesterday, create a nightly job to run dmpmqcfg, the Rexx script and push the output to your source code repository and you will have a complete history of changes made to your queue manager.

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, IBM MQ Appliance, Linux, macOS (Mac OS X), Open Source, Programming, Rexx, Windows Comments Off on Enhancement to Exporting MQ Objects into Individual Files

MacBook Pro, External Monitor, Display Link and Java

Ok. Here’s a really weird one reported yesterday by a customer.

If you are using a MacBook Pro with an external monitor (or 2) via Display Link and you try and run a Java GUI application, it will crash because the JVM thinks that it is running on a headless computer (no graphics driver).

Therefore, if are running MQ Visual Edit or MQ Visual Browse or MQTT Message Editing Suite you may get this issue because all of these programs are GUI applications that require graphics driver.

The quick and dirty solution to the issue is to change your primary display to be the built-in display. Hence, do the following:

System Preferences –> Displays –> Arrangement –> Drag menu bar to built-in display

Note: While the menu bar on a MacBook Pro (aka macOS) appears by default on all connected displays, placement of the menu bar on a display within the system preferences determines which screen is the primary.

I’m going to continue to look for a better solution.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Java, JMS, macOS (Mac OS X), MQ Visual Browse, MQ Visual Edit, MQTT Message Editing Suite Comments Off on MacBook Pro, External Monitor, Display Link and Java

Exporting MQ Objects into Individual Files

I posted this item on mqseries.net and I thought I should do a write up here.

Back in the early 2000s, I was a consultant at a customer site and they wanted the nightly dumps of the MQ objects (not messages), 1 line per object and then check those files into a source code repository.

I used SupportPac MS03 to dump the MQ objects but it put all objects in the same file and each object was on many lines. So, I wrote a simple Rexx script to extract each object type to: (1) flatten each object to 1 line and (2) write it their own object file.

SupportPac MS03 has been replaced with the dmpmqcfg program.

I have updated the list of object types in the Rexx script for the modern IBM MQ v9.2 (because IBM has added a lot of new objects in the last 15 years).

Example of running dmpmqcfg against your queue manager called MQA1:

dmpmqcfg -m MQA1 -a > MQA1.all.mqsc

Then you run it against my REXX script called fmt_dmpmqcfg.rex:

/* Format output from dspmqcfg command to a single line for each entry.  */
/* Put each MQ object type into a different file.                        */
/*                                                                       */
/* Example dspmqcfg command:                                             */
/*    dmpmqcfg -m MQA1 -a > MQA1.all.mqsc                                */

trace o
Parse Arg inFN .

if inFN = "" then
do
   Say "Invalid number of parameters."
   Say "i.e."
   Say "     fmt_dmpmqcfg.rex input_file_name"
   Exit
end

types = "QMGR CHANNEL LISTENER NAMELIST PROCESS QALIAS QLOCAL QMODEL QREMOTE SERVICE SUB TOPIC AUTHREC AUTHINFO CHLAUTH COMMINFO"


/* open file */
Call Stream inFN,'C','OPEN READ'

/* Delete previous output and then open it. */
do i=1 to Words(types)
   ptr = Word(types,i)
   outFN = inFN||"."||ptr||".mqsc"
   if (STREAM( outFN, 'C', 'QUERY EXIST' ) <> "") then
      "ERASE  "outFN
   Call Stream outFN,'C','OPEN WRITE'
   counts.ptr = 0
end

total = 0
mqscCmd = ""
do until Lines(inFN) = 0

   inLine = Strip(LineIn(inFN))   /* read next line */

   if (inLine = "") then
      NOP   /* blank line, forget-about-it */
   else if (SubStr(inLine,1,1) = "*") then
      NOP   /* comment line, forget-about-it */
   else
   do
      /* Last attribute in MQSC command? */
      if (Length(inLine) <> LastPos('+',inLine)) then
      do
         mqscCmd = mqscCmd || " " || inLine  /* add last attribute */
         ptr = Word(mqscCmd, 2)

         if (Pos("(", ptr) > 0) then
            ptr = SubStr(ptr, 1, Pos("(", ptr) - 1)

         nothing = LineOut(inFN||"."||ptr||".mqsc", mqscCmd)   /* write to the file  */

         counts.ptr = counts.ptr + 1
         total = total + 1
         mqscCmd = ""
      end
      else
      do
         temp = Strip(SubStr(inLine,1,(LastPos('+',inLine) - 1)))  /* remove '+' */
         if (mqscCmd = "") then
            mqscCmd = temp
         else
            mqscCmd = mqscCmd || " " || temp
      end
   end
end

/* close the files  */
Call Stream inFN,'C','CLOSE'
do i=1 to Words(types)
   ptr = Word(types,i)
   outFN = inFN||"."||ptr||".mqsc"
   Call Stream outFN,'C','CLOSE'
   Say outFN || " contains " || counts.ptr || " MQSC formatted commands."
end

Say ""
Say "Total formatted MQSC commands was:" total

Exit

To run the Rexx script, the command is:

rexx fmt_dmpmqcfg.rex  MQA1.all.mqsc

The Rexx script will output:

MQA1.all.mqsc.QMGR.mqsc contains 1 MQSC formatted commands.
MQA1.all.mqsc.CHANNEL.mqsc contains 43 MQSC formatted commands.
MQA1.all.mqsc.LISTENER.mqsc contains 5 MQSC formatted commands.
MQA1.all.mqsc.NAMELIST.mqsc contains 3 MQSC formatted commands.
MQA1.all.mqsc.PROCESS.mqsc contains 2 MQSC formatted commands.
MQA1.all.mqsc.QALIAS.mqsc contains 4 MQSC formatted commands.
MQA1.all.mqsc.QLOCAL.mqsc contains 111 MQSC formatted commands.
MQA1.all.mqsc.QMODEL.mqsc contains 7 MQSC formatted commands.
MQA1.all.mqsc.QREMOTE.mqsc contains 11 MQSC formatted commands.
MQA1.all.mqsc.SERVICE.mqsc contains 6 MQSC formatted commands.
MQA1.all.mqsc.SUB.mqsc contains 1 MQSC formatted commands.
MQA1.all.mqsc.TOPIC.mqsc contains 8 MQSC formatted commands.
MQA1.all.mqsc.AUTHREC.mqsc contains 438 MQSC formatted commands.
MQA1.all.mqsc.AUTHINFO.mqsc contains 4 MQSC formatted commands.
MQA1.all.mqsc.CHLAUTH.mqsc contains 3 MQSC formatted commands.
MQA1.all.mqsc.COMMINFO.mqsc contains 1 MQSC formatted commands.

Total formatted MQSC commands was: 648

If you setup a job (script/batch file) to do this nightly and then check the output files into a source code repository then you will have a complete history of changes made to your queue manager. Because the Rexx script flattens the data to 1 line per object, your source code repository will show you exactly which objects have changed.

Its a simple hack but well worth implementing.

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, IBM MQ Appliance, Linux, macOS (Mac OS X), Open Source, Programming, Rexx, Windows Comments Off on Exporting MQ Objects into Individual Files

IBM MQ Fix Pack 8.0.0.16 Released

IBM has just released Fix Pack 8.0.0.16 for IBM MQ
https://www.ibm.com/support/pages/node/6417257

Regards,
Roger Lacroix
Capitalware Inc.

Fix Packs for MQ, IBM i (OS/400), IBM MQ, Linux, Unix, Windows Comments Off on IBM MQ Fix Pack 8.0.0.16 Released

IBM MQ Message Duplication

IBM Support has a technical note labeled Does MQ have a feature to generate automatically a duplicate message?.

The options listed on that page are to use Pub/Sub or programs qload and dmpmqmsg. You can also use MQ Visual Edit and MQ Batch Toolkit to get messages from a queue and write the message to a file. You can even use the open source programs Message Multiplexer (MMX) and/or Msg2File to do it too.

The problem with these programs is that if there are getters (applications performing MQGET API calls) then those programs will NOT capture all of the messages flowing through the queue.

The Pub/Sub solution is ok IF the user does not care about duplicating the message’s MQMD fields. The instructions on this page labeled MQ: You want to put a message into a queue and you want to generate duplicate messages into other queues has the following warning:

NOTE: Strictly speaking, the messages received by Q1 and Q2 are NOT truly identical. Even though they will have the same payload (text string ‘TEST’) and same characteristics, each message will have its own unique message-id.

Notice that the only differences in this particular example are the last 2 characters from “MsgId” and CorrelId”.

The only true solution to EXACTLY replicate a message and its MQMD fields is Capitalware’s MQ Message Replication (MQMR). MQMR will duplicate/clone messages being written (via MQPUT or MQPUT1 API calls) to an application’s output queue and MQMR will write the exact same messages to ‘n’ target queues (‘n’ can be up to 100). When MQMR replicates a message both the message data and the message’s MQMD structure will be cloned. This means that the fields of the MQMD structure (i.e. PutTime, MessageId, CorrelId, UserId, etc..) will be exactly the same as the original message’s MQMD structure.


MQMR can clone messages being written to any MQ queue type. i.e. local, alias, remote and/or cluster queues (including transmission queues).

Existing applications do not need to be modified or updated nor is it necessary to set up Publish/Subscribe to use MQMR. MQMR is deployed and configured at the queue manager level; hence, no application changes are required.

MQMR includes 2 auxiliary programs to help with the management of the cloned/replicated messages:

  • MQ Queue To SQLite DB (MQ2SDB) program will offload MQ messages to an SQLite database.
  • SQLite DB To MQ Queue (SDB2MQ) program will load SQLite database rows into messages in an MQ queue.

Capitalware offers free trials of MQMR including free support, so if anyone would like to try out MQMR then send the email to support@capitalware.com

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Replication, Raspberry Pi, Unix, Windows Comments Off on IBM MQ Message Duplication

Interesting Blog Item: IBM Knowledge Center – LATEST!

Morag Hughson of MQGem has a new blog posting called IBM Knowledge Center – LATEST! regarding a new feature in the IBM Knowledge Center.

Instead of providing a link with the MQ version number, you can swap the version number for the word “latest”. This is a cool feature for putting links in a blog posting or forums on the internet. The only “gotcha” is that you have to manually edit the link yourself which of course can lead to typos hence an invalid link.

IBM should add a “Latest Link” link on the button bar on each page next to “Print PDF Help” buttons, so that you don’t have to manually edit the link.

Regards,
Roger Lacroix
Capitalware Inc.

HPE NonStop, IBM i (OS/400), IBM MQ, IBM MQ Appliance, Linux, macOS (Mac OS X), Programming, Security, Unix, Windows, z/OS Comments Off on Interesting Blog Item: IBM Knowledge Center – LATEST!

IBM MQ End of Service Dates

So, what version of IBM MQ (aka WebSphere MQ & MQSeries) are you running? Are you behind in upgrading your release of MQ?

Everybody “should be” running IBM MQ v9.0, v9.1 or v9.2, if you want support from IBM.

In case it has slipped your mind, here is a list of IBM MQ releases and there end of service dates:

Note: IBM v9.1 and v9.2 do not yet have end of service dates.

Regards,
Roger Lacroix
Capitalware Inc.

IBM i (OS/400), IBM MQ, IBM MQ Appliance, Linux, Unix, Windows, z/OS Comments Off on IBM MQ End of Service Dates

Reasons MQSC Commands for an AMQP Channel May Not Work

IBM has posted a support document describing the various reasons why an MQSC command may not work for an AMQP channel.
https://www.ibm.com/support/pages/node/6411487

Regards,
Roger Lacroix
Capitalware Inc.

Education, IBM MQ, IBM MQ Appliance, Linux, Unix, Windows Comments Off on Reasons MQSC Commands for an AMQP Channel May Not Work

Using Loki and Grafana with MQ Logs

Mark Taylor of IBM has posted a blog item on using Loki and Grafana software to parse and display information from MQ logs.
https://marketaylor.synology.me/?p=838

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, IBM MQ Appliance, Linux, Unix, Windows Comments Off on Using Loki and Grafana with MQ Logs

Local Transactions Using IBM MQ and the JMS API

Olja Rastic-Dulborough has written an article that introduces the user to the concepts of local transactions using IBM MQ and JMS.
https://developer.ibm.com/components/ibm-mq/articles/an-introduction-to-local-transactions-using-mq-and-jms/

Regards,
Roger Lacroix
Capitalware Inc.

Education, HPE NonStop, IBM i (OS/400), IBM MQ, IBM MQ Appliance, Java, JMS, Linux, macOS (Mac OS X), Programming, Raspberry Pi, Unix, Windows, z/OS Comments Off on Local Transactions Using IBM MQ and the JMS API