IBM MQ Fix Pack 9.0.0.6 Released

March 28, 2019 Roger Lacroix

IBM has just released Fix Pack 9.0.0.6 for IBM MQ V9.0 LTS
https://www.ibm.com/support/docview.wss?uid=ibm10876306

Regards,
Roger Lacroix
Capitalware Inc.

Fix Packs for MQ, IBM i (OS/400), IBM MQ, Linux, Unix, Windows Comments Off on IBM MQ Fix Pack 9.0.0.6 Released

Swift 5 Released

March 26, 2019 Roger Lacroix

Ted Kremenek of Apple has announced release of Swift 5:
https://swift.org/blog/swift-5-released/

Swift 5 is a major milestone in the evolution of the language. Thanks to ABI stability, the Swift runtime is now included in current and future versions of Apple’s platform operating systems: macOS, iOS, tvOS and watchOS. Swift 5 also introduces new capabilities that are building blocks for future versions, including a reimplementation of String, enforcement of exclusive access to memory during runtime, new data types, and support for dynamically callable types.

Regards,
Roger Lacroix
Capitalware Inc.

iPhone, iPad & iPod Touch, Linux, macOS (Mac OS X), Open Source, Programming, Swift Comments Off on Swift 5 Released

Oh Hardware, how I hate thee!

March 25, 2019 Roger Lacroix

Well, there went 2 days of my life I won’t get back. 🙂

About 2 weeks ago, when I booted up my SLES 64-bit Linux server, it complained about issues with the hard drive. So, I ran “fsck” to fix the file system issues. There was only a couple of problems then everything was fine. I had booted the server a couple of more times then it happened again (Note: I don’t leave the server running because I don’t use it every day). Again, I ran fsck and then the OS was happy. I have a backup of everything on it, but I figured I would backup important files again. Last Wednesday, it happened again that the OS complained. I fixed it but on Thursday, everything went sideways. While using it, the OS started complaining that it could not find executables like “ls” command. I tried to “sudo” to root but command not found. I tried to shut it down from the console but nothing. I did a hard reset of the server and let cool off. I figured I was SOL but you never know. A hour later, I powered it on but the OS just hung.

So, I pulled the box out of the rack, opened it up to see what hard drive it had. Now, this server is probably 10-11 years old (I know, nothing last forever). The hard drive was a Seagate Barracuda 500GB SATA2. I copied the motherboard model (ASUS P5E-VM DO) then downloaded the manual to see what it supported which was SATA2 (and SATA1). I got curious if I could use a SSD drive and after a few searches it appeared so. I checked my local computer store to see what they had but all they had were SATA3. Again, a quick internet search says that SATA3 is backward compatible.

I purchased a Samsung 860 EVO. I installed it, screwed the case closed and put it back in the rack. Nope, didn’t check it because how dumb can you be to screw 4 screws and plug in 2 cables. Booted it up and went into the BIOS and no hard drive. Did a refresh, nothing, rebooted it, still nothing. Started to swear. Pulled it out of the rack, opened the case up, unplug the 2 cables, plugged them back in and triple checked that they were actually plugged in. I put the case on the floor, plugged in the video, keyboard and power cables and turned it on. When I go into the BIOS, bingo the hard drive is there. So, Murphy’s Law just proved after all these years of playing around with hardware, I can still do dumb things. D’oh!

I checked the specs for IBM MQ v9.1 and it is supported on RHEL 7.3 (or higher) and SLES 12 SP3 (or higher). In the 90’s and 2000’s, I only used Red Hat (Fedora) as the Linux OS for building and testing my applications, exits, etc. Around 7-8 years ago, I switched to SUSE as my primary Linux distribution for x86, x64, POWER and System Z because it seemed a lot of customers were using SUSE and IBM had it as their default Linux distribution for POWER and System Z VMs.. On Friday, I decided I would continue with SLES since I like using the system tool called YaST (Note: I’m not a SysAdmin).

Installing SLES was easy, the only odd part is that they seemed to have removed package selection after picking the type of usage you wanted (i.e. server, workstation, etc.). After the install, of course, a bunch of stuff was missing: gcc, ldap & pam development, plus a few other things. YaST to the rescue.

Created the mqm group, mqm UserId plus my UserId (add mqm group as additional group) and my testing UserIds.

I installed IBM MQ v9.1, Quest Authentication Services (aka Vintela Authentication Services), Centrify’s DirectControl and all of my tools. My fingers got tired of doing “rpm -ivh …”.

Next, I created my test queue managers and started them. Before I ever play around with my (Capitalware) security stuff, I make sure everything is working perfectly first. I start by feeding runmqsc the following MQSC commands:

ALTER QMGR CHLAUTH(DISABLED)
ALTER QMGR CONNAUTH(' ')
DEFINE CHANNEL ('SYSTEM.ADMIN.SVRCONN') CHLTYPE(SVRCONN) TRPTYPE(TCP) REPLACE
DEFINE LISTENER('LISTENER.TCP') TRPTYPE(TCP) CONTROL(QMGR) PORT(1414)
START LISTENER('LISTENER.TCP')

I started MQ Explorer, added the remote queue manager and when I connected, I got reason code 2035 (not authorized). I checked the /etc/group file and yes, my UserId was in the mqm group. The interesting thing I did not know, clearing/blanking CONNAUTH does not disable it. You have to either restart the queue manager or issue the following MQSC command: 

REFRESH SECURITY TYPE(CONNAUTH)

Note: Before someone posts, “hey, why are you turning off MQ security”, it is because I going to be using MQAUSX which of is better than what comes with MQ. 🙂

Now that MQ Explorer and client tools like MQ Visual Edit can successfully connect to the new queue manager, I can move on to applying MQAUSX security solution.

Time to do some builds. GCC was chewing its way through the makefile then I get the following error:

/usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: /tmp/cc13IfK2.o: undefined reference to symbol 'ber_free'
/usr/lib64/gcc/x86_64-suse-linux/4.8/../../../../x86_64-suse-linux/bin/ld: /usr/lib64/liblber-2.4.so.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status

WTF! I just used the very same makefile on the now dead SLES 11 SP2 64-bit server. After a bunch of searches, I found an explanation on StackOverflow about implicit vs explicit linking. What the above error message “DSO missing from command line” is trying to say (very badly) is that I need “-llber” on my GCC link step. Once I added to the makefile then GCC successfully compiled and linked everything in the makefile.

I have done some basic testing of my tools and security stuff and everything is looking good. Too bad I wasted 2 days because of a hard drive failure. 🙁

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM MQ, Linux, Operating Systems Comments Off on Oh Hardware, how I hate thee!

New: Universal File Mover v2.0.0

March 20, 2019 Roger Lacroix

Capitalware is pleased to announce the release of Universal File Mover v2.0.0. It is a free open source project.

Universal File Mover is more than a simple tool to manage the transfer of files. It allows the user to combine business processes into a workflow. The user combines a series of Action commands to create the UFM Workflow XML file. UFM is supported on AIX, HP-UX, IBM i (OS/400), Linux, macOS (Mac OS X), Solaris and Windows.

    Changes:

  • Added RemoveItems action – it removes items in a file from a source file.
  • For Global onerrorfail=”Y”, if the workflow with a Watch Action encounters an error with the delete on MQSend Action then the entire workflow will stop and the OnError will run.
  • For MQReceive Action, perform tokenizing of MQ_RECEIVE_FILE before calling ‘Default Execute’ (if used)
  • Renamed package from “biz.capitalware” to “com.capitalware”
  • Added support for the single combined MQ JAR file called: “com.ibm.mq.allclient.jar” (i.e. IBM MQ v8 & higher).
  • Fixed a bug with the installer putting scp1.xml and sftp1.xml in the wrong directory.
  • Updated to the latest release of JSch.
  • There are now 2 installers for Windows: 32-bit and 64-bit installers

For more information on Universal File Mover, please go to:
https://www.capitalware.com/ufm_overview.html

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, IBM MQ Appliance, Java, Linux, macOS (Mac OS X), Open Source, Universal File Mover, Unix, Windows Comments Off on New: Universal File Mover v2.0.0

Offline C Runtime Library Reference

March 14, 2019 Roger Lacroix

Google is a great search engine. When I search for “ftime linux”, the first search hit will be the man page for ftime which is exactly what I wanted. But sometimes I just want the whole manual in PDF format so that I can simply read about functions and scroll through it while I’m trying to decide how I want to implement a particular feature. Hence, having the particular OS’s (Operating System’s) C Runtime Library in a PDF is really great (no knocks on Google).

So, here are a bunch of downloadable C Runtime Libraries in PDF format:

OS Landing Page Direct link to PDF
AIX PDF format documentation for XL C/C++ for AIX, V16.1 PDF direct link for XL C/C++ Language Reference
Linux The GNU C Library PDF direct link for GNU C Library Reference
IBM i (OS/400) PDF file for ILE C/C++ Runtime Library Functions PDF direct link for ILE C/C++ Runtime Library Functions
Windows MSDN C Run-Time Library Reference PDF direct link for C Runtime Library Reference
z/OS z/OS XL C/C++ Runtime Library Reference PDF direct link for z/OS XL C/C++ Runtime Library Reference

For some strange reason, for AIX, IBM produces the a “Language Reference” but not a “Runtime Library Reference” but for IBM i (OS/400) and z/OS they have a proper “C/C++ Runtime Library Reference”. The same is true for the Linux. It is a “Language Reference” but not a “Runtime Library Reference”.

There you go. Enjoy your offline “C/C++ Runtime Library Reference” PDFs.

Note: If anyone has the real/correct link for either AIX or Linux’s “C/C++ Runtime Library Reference”, please send it to me and I will update the posting. Also, if you have a link to HP-UX, Solaris and/or macOS’s “C Runtime Library Reference”, let me know and I’ll post it.

Regards,
Roger Lacroix
Capitalware Inc.

C, Education, IBM i (OS/400), Linux, Programming, Unix, Windows, z/OS Comments Off on Offline C Runtime Library Reference

More RSS Sidebar MQ Content on my Blog

March 7, 2019 Roger Lacroix

You may have noticed for a couple of weeks that the “IBM MQ” content sidebar was not showing anything. That is because IBM again changed the link. IBM doesn’t publish the information and they love to change it every so often, so for me it is like a game of whack-a-mole.

Someone sent me the RSS feed link for both IBM MQ and WebSphere MQ. I didn’t know that IBM had 2 different ones. So, I posted both RSS sidebars on my blog.

I also decided to add the IBM Developer Messaging Blog RSS feed, Colin Paice Blog RSS feed, Leif Davidsen Blog RSS feed and Lyns Random Thoughts Blog RSS feed.

Hence, enjoy the MQ content.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, HPE NonStop, IBM i (OS/400), IBM MQ, Linux, Unix, Windows, z/OS Comments Off on More RSS Sidebar MQ Content on my Blog

Weird Problem related to a Channel Security Exit

March 1, 2019 Roger Lacroix

I posted the following weird channel security exit problem on the MQSeries ListServer.

I received a lot of emails (both public and private) offering help to the problem. Thanks. It was not an SELinux issue nor was it a corrupt shared library issue. It was an issue related to permissions for a mount point.

Here is the description to the problem followed by the solution:

I’ve got a really weird problem, that the internet seems to say that it is a permission issue with “/tmp” but I’m at a loss.

– Brand new: Red Hat Enterprise Linux Server release 7.6 (Maipo)
– Fresh install of MQ v9.1.0.1 64-bit

Installed MQAUSX into /var/mqm/exits64/ and yes it is a working version that works perfectly on other Linux distributions.

Here is the MQ error message:

08/02/19 12:46:45 - Process(51014.254) User(roger) Program(amqrmppa)
                    Host(someserver) Installation(Installation1)
                    VRMF(9.1.0.1) QMgr(MQA1)
                    Time(2019-02-08T11:46:45.514Z)
                    ArithInsert1(536895861)
                    CommentInsert1(/var/mqm/exits64/mqausx)
                    CommentInsert2(/var/mqm/exits64/mqausx: failed to map segment from shared object: Operation not permitted)
                    CommentInsert3(64)

AMQ6175E: The system could not dynamically load the shared library
'/var/mqm/exits64/mqausx'. The system returned error message
'/var/mqm/exits64/mqausx: failed to map segment from shared object: Operation
not permitted'.

EXPLANATION:
This message applies to UNIX systems. The shared library '/var/mqm/exits64/mqausx' 
failed to load correctly due to a problem with the library.

ACTION:
Check the file access permissions and that the file has not been corrupted.

“mqausx” is a standard Unix/Linux shared library (It is running on tons of Linux servers without issue). Even the “ldd” command gives a weird result:

$ ldd /var/mqm/exits64/mqausx
ldd: warning: you do not have execution permission for `/var/mqm/exits64/mqausx'
        not a dynamic executable

The permissions are set as follows:

chown mqm:mqm mqausx
chmod 750 mqausx

I even tried 777 for permissions. i.e.

-rwxrwxrwx  1 mqm  mqm  247087 Feb  7 15:11 mqausx

It still failed. Its a shared library, what is or how is a shared library “dynamic executable”?

Here is the solution that Josh suggested that worked:

First, issue the following command against your shared library:

$ df /var/mqm/exits64
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/aaa/bbb           5107712   3967288   1140424  78% /var/mqm

Use whatever is under “Mounted on” (i.e. /var/mqm – you may have something else like /var ) in the following command:

$ mount | grep /var/mqm

In my customer’s particular case, their mount point was on /var and the results of the command was:

/dev/aaa/bbb on /var type xfs (rw,nosuid,nodev,noexec,relatime,attr2,inode64,logbsize=256k,sunit=512,swidth=512,noquota)

The source of the issue was that the mount point had “noexec”permission. Hence, once that option was removed then everything went back to normal.

I hope this helps someone else in the future.

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, Linux, MQ Auditor, MQ Authenticate User Security Exit, MQ Channel Connection Inspector, MQ Channel Encryption, MQ Channel Throttler, MQ Enterprise Security Suite, MQ Message Encryption, MQ Message Replication, MQ Standard Security Exit, Unix Comments Off on Weird Problem related to a Channel Security Exit

Best Practices for an IBM MQ Channel’s SHARECNV Parameter

March 1, 2019 Roger Lacroix

I was chatting with Josh the other day and he asked me if I knew that IBM now does not recommend uses the default value of 10 for a channel’s SHARECNV (shared conversation) value. He said that the MQ Knowledge Center recommends for distributed queue managers a value of 1 for a channel’s SHARECNV.

Of course, I said WTF and where did you read that. He emailed me the link and sure enough, that is what it says.

The default settings for client and server connection channels changed in Version 7.0 to use shared conversations. Performance enhancements for distributed severs were then introduced in Version 8.0. To benefit from the new features that were introduced alongside shared conversations, without the performance impact on the distributed server, set SHARECNV to 1 on your Version 8.0 or later server connection channels.

And further down it says:

However, for distributed servers, processing messages on channels that use the default configuration of 10 shared conversations is on average 15% slower than on channels that do not use shared conversations.

A 15% performance hit for using the default value for SHARCNV on distributed queue managers!!! OMG!

From a guy who writes MQ exits, I have always hated the SHARECNV parameter when it has a value greater than 1 because in my opinion, IBM did a horrible job in their implementation of it as it relates to a channel security exit.

When SHARECNV parameter is set to 1, a channel security exit is invoked with the following ExitReason values:

  • MQXR_INIT
  • MQXR_INIT_SEC
  • MQXR_SEC_MSG <- optional
  • MQXR_SEC_PARMS
  • MQXR_TERM

    • When SHARECNV parameter is set to a value greater than 1, i.e. 10, a channel security exit is invoked with the following ExitReason values:

  • MQXR_INIT
  • MQXR_INIT_SEC
  • MQXR_SEC_MSG <- optional
  • MQXR_SEC_PARMS – for session # 1
  • MQXR_SEC_PARMS – for session # 2
  • MQXR_SEC_PARMS – for session # 3
  • MQXR_SEC_PARMS – for session # 4
  • MQXR_SEC_PARMS – for session # 5
  • MQXR_SEC_PARMS – for session # 6
  • MQXR_SEC_PARMS – for session # 7
  • MQXR_SEC_PARMS – for session # 8
  • MQXR_SEC_PARMS – for session # 9
  • MQXR_SEC_PARMS – for session # 10
  • MQXR_TERM

    • Everything looks EXACTLY the same for the 1st session but for session #2 to session #10, the calling sequence is totally different.

    Secondly, did you happen to notice that when all but the last session ends, the channel security exit is NOT made aware of this fact!!! The channel security exit is only invoked with MQXR_TERM when the last session, whichever number that is, terminates (never for the other sessions ending). I have no idea who dreamt this up, but man it is a bad design.

    I am so happy that Josh told me about this little tidbit. Now I get to tell my customers that IBM’s best practices for a channel’s SHARECNV parameter on distributed platforms, is to set it to 1 rather than use the default value of 10.

    It would be interesting to know the performance for z/OS queue managers whose channel’s SHARECNV is set to 10 vs 1.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    HPE NonStop, IBM i (OS/400), IBM MQ, IBM MQ Appliance, Linux, Unix, Windows Comments Off on Best Practices for an IBM MQ Channel’s SHARECNV Parameter

    RIP: MQ Technical Conference

    February 19, 2019 Roger Lacroix

    Capitalware Inc. is a small software company specializing in IBM MQ. Capitalware used to be a vendor at IBM’s Transaction and Messaging Technical Conferences (T&M Conference). IBM merged several conferences including the T&M Conference into Impact 2007. I went as a vendor to Impact 2007 but found that most attendees were not involved in MQ. So, I decided that it would be the last conference I would attend as a vendor. I starting bugging IBM PartnerWorld, marketing and education people to bring back IBM’s T&M conference. This went on for 6 years with IBM people always saying “that is not the direction we are going in”.

    So, in earlier 2013, after many years of frustration, I decided to stick my neck out and create MQ Technical Conference (MQTC). I wanted a technical conference to be focused on IBM MQ (plus IBM products that use MQ) and thought 3 days was a good length. Since, my wife and I have 5 kids, I decided the conference should be at a family-friendly hotel/resort. I had already been to many conferences at Kalahari Resorts in Sandusky, Ohio, so decided it was a good location for MQTC. When I signed the contract with Kalahari Resorts, I had to personally guarantee to pay them $24,000 USD (for food and facilities) regardless if anyone showed up or not (plus penalties for minimum room guarantee). I was pretty nervous about it and hoped attendees would show up.

    It is a lot of work to create a conference, coordinate with vendors, food, audio/video, scheduling of sessions, etc. but I always enjoyed the conference and I believe most attendees found it valuable.

    Last year, Ray Daniel of IBM came to MQTC v2.0.1.8 as part of IBM sponsorship team. He so enjoyed the conference that he decided IBM needed to create its own conference modeled after MQTC. He created a 3-day technical conference called Integration Technical Conference (ITC). Like MQTC, it will have 5 session rooms, like MQTC, it will be heavy into IBM MQ and like MQTC, it will be at a family-friendly venue. The big difference, between ITC and MQTC is that ITC will be FREE to attend. Last year, MQTC v2.0.1.8 costs more than $65,000 USD and I cannot possibly offer a free conference.

    Therefore, I have decided to suspend MQTC as IBM now offers a conference for MQ users which is free.

    I do hope that Ray Daniel continues the tradition of having the cheesecake bar and ice-cream social at ITC. 🙂

    I would like to thank:

  • All of the attendees who came to the 6 MQTC conferences
  • All of the various speakers from a wide variety of industries who spoke at the 6 MQTC conferences
  • All the sponsors who supported the 6 MQTC conferences.

    • Thank you.

    Finally, I would like to especially thank T.Rob Wyatt for his support and help with MQTC.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    P.S. I would like to hear feedback from ITC attendees on core and non-code MQ sessions.

    Capitalware, Education, IBM MQ, MQ Technical Conference 10 Comments

    How to Compile and Link MQ Programs on Linux

    February 13, 2019 Roger Lacroix

    Someone on StackOverflow asked “How to compile a C program without knowing the include files”.

    JasonE gave a really good answer, so I thought I would repeat it here:

  • To compile and link 64-bit MQ applications on Linux, follow the instructions here
  • To compile and link 32-bit MQ applications on Linux, follow the instructions here

    • In summary:

  • -I is for the product includes, which are (For Linux) usually in /opt/mqm/inc
  • -L is the path to the libraries in your example which are (For Linux) usually in /opt/mqm/lib (for 32 bit applications) and /opt/mqm/lib64 (for 64 bit applications)
  • -l (lower case L) is for the required library/libraries

    • and the actual library you need is either:

  • mqm – server bound C applications (ie -lmqm, which links with libmqm.so)
  • mqic – client bound C applications (ie -lmqic, which links with libmqic.so)

    • and a suffix of _r if you are building as a threaded application (ie you are linking with -lpthread as well, ie providing -lmqm_r or -lmqic_r which in effect links with libmqm_r.so or libmqic.so)

    cmqc.h is the name of the main header file, and there are other cmq*.h headers you can optionally include as well.

    Regards,
    Roger Lacroix
    Capitalware Inc.

    C, C++, IBM MQ, Linux, Programming Comments Off on How to Compile and Link MQ Programs on Linux