Levels of JRE and GSkit bundled with IBM MQ

March 9, 2018 Roger Lacroix

IBM has posted a support note regarding the levels of JRE and GSkit bundled with IBM MQ
http://www.ibm.com/support/docview.wss?uid=swg27038676

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, Java, JMS, Linux, Security, Unix, Windows Comments Off on Levels of JRE and GSkit bundled with IBM MQ

Creating a Standard for MQ Messages to/from SQLite DB

March 8, 2018 Roger Lacroix

Yesterday, I wrote about MQ Messages to/from SQLite DB which are a pair of utilities for offloading and loading messages to/from SQLite database.

I thought it would be a good idea to publish what I designed so that other vendors, companies or users can easy use the standard.

First, the SQL to create the table is:

CREATE TABLE IBM_MQ_MESSAGES(Version            INT,
                             Report             INT,
                             MsgType            INT,
                             Expiry             INT,
                             Feedback           INT,
                             Encoding           INT,
                             CodedCharSetId     INT,
                             Priority           INT,
                             Persistence        INT,
                             BackoutCount       INT,
                             PutApplType        INT,
                             MsgSeqNumber       INT,
                             Offset             INT,
                             MsgFlags           INT,
                             OriginalLength     INT,
                             Format             CHAR(8),
                             ReplyToQ           CHAR(48),
                             ReplyToQMgr        CHAR(48),
                             UserIdentifier     CHAR(12),
                             ApplIdentityData   CHAR(32),
                             PutApplName        CHAR(28),
                             PutDate            CHAR(8),
                             PutTime            CHAR(8),
                             ApplOriginData     CHAR(4),
                             MsgId              BLOB,
                             CorrelId           BLOB,
                             AccountingToken    BLOB,
                             GroupId            BLOB,
                             Data               BLOB );

I like the KISS principle, so the name of the table is “IBM_MQ_MESSAGES”. Clean and obvious. The column names are directly from the MQMD structure in cmqc.h file.

    Column layout for the table:

  • The MQMD MQLONG fields are all INT (32-bit) columns for SQLite.
  • The MQMD MQCHAR fields are all CHAR(*) columns for SQLite.
  • The MQMD MQBYTE fields are all BLOB columns for SQLite.
  • The message data (payload) is stored as BLOB column in SQLite.

The naming standard for the SQLite database file is based on the queue manager’s name, the queue name and the current day’s date.
I.e.

    QMgrName–QueueName–YYYY_MM_DD.mqsdb

  • QMgrName is the name of the source MQ queue manager
  • QueueName is the name of the source MQ queue
  • YYYY_MM_DD is the current year, month and day.

For a file extension, I decided to create a uniquely identifyible file extension called ‘mqsdb’. Now, I could have gone with a more standard file extension like ‘db’ or ‘sqlite’ but I wanted something more descriptive that would clearly identify the file’s purpose.

Next, reducing the database size or attempting to keep it as small as possible. Hence, any MQMD field that was either all blank or null, I set the table column to null for that record. This is done for all 9 CHAR fields and 5 BLOB fields.
i.e.

/* Is the Format field all blank? */
if (memcmp(pMD->Format, MQFMT_NONE, MQ_FORMAT_LENGTH) == 0)
   sqlite3_bind_null(hStmt, 16);
else
   sqlite3_bind_text(hStmt,  16, pMD->Format, MQ_FORMAT_LENGTH, SQLITE_STATIC);

/* Is the CorrelId field all null? */
if (memcmp(pMD->CorrelId, MQCI_NONE, MQ_CORREL_ID_LENGTH) == 0)
   sqlite3_bind_null(hStmt, 26);
else
   sqlite3_bind_blob(hStmt, 26, (char *)&(pMD->CorrelId), MQ_CORREL_ID_LENGTH, SQLITE_STATIC);

So, in theory, if the Format field is blank or the CorrelId field is null, then by setting the column field to null, it should save disk space.

Finally, reading the SQLite database where certain columns may or may not be null.

/* Retrieve the Format field, is it null? */
p = (char *)sqlite3_column_text(hStmt,  15);
if (p != NULL)
   memcpy(md.Format, p, MQ_FORMAT_LENGTH);

/* Retrieve the CorrelId field, is it null? */
p = (char *)sqlite3_column_blob(hStmt, 25);
if (p != NULL)
   memcpy(md.CorrelId, p, MQ_CORREL_ID_LENGTH);

So there you have it, a new MQ Messages to SQLite database standard that I hope everyone will adopt. 🙂

Now, for those people who REALLY like to read code, please don’t get your shorts in a knot because the column numbers are different between writing and reading the database. It is one of those quirks in SQLite. For sqlite3_bind_* API calls, the column number begin with ‘1’ whereas for sqlite3_column_* API calls, the column number begin with ‘0’. Hence, that is why they differ by 1 and it is something you just need to remember. 🙂

Maybe some time in the near future, I’ll write another blog posting on how to handle this new MQ Messages to SQLite database standard from Java.

Regards,
Roger Lacroix
Capitalware Inc.

C, Capitalware, Database, IBM i (OS/400), IBM MQ, Linux, MQ Message Replication, Open Source, Programming, Unix, Windows Comments Off on Creating a Standard for MQ Messages to/from SQLite DB

MQ Messages to/from SQLite DB

March 7, 2018 Roger Lacroix

Capitalware has an MQ solution called MQ Message Replication (MQMR).

MQ Message Replication will clone messages being written (via MQPUT or MQPUT1 API calls) to an application’s output queue and MQMR will write the exact same messages to ‘n’ target queues (‘n’ can be up to 100). When MQMR replicates a message both the message data and the message’s MQMD structure will be cloned. This means that the fields of the MQMD structure (i.e. PutTime, MessageId, CorrelId, UserId, etc..) will be exactly the same as the original message’s MQMD structure.

I have always left it up to the customer to decide what to do with and/or how to offload replicated messages in the target queue(s). There are probably 100 solutions for extracting messages in a queue and writing the information to a file: Capitalware has solutions, other vendors have products, SupprtPacs and even the new dmpmqmsg program included with IBM MQ. These are all fine tools to get the job done and of course, I prefer my tools which use VEQ format that I created a very long time ago (and is published here).

Over this past weekend, I got the bright idea to create the 101st solution for offloading and loading MQ messages to and from an SQLite database. I will include the solution with MQMR for free – an extra little bonus for customers. 🙂

Why bother? Well, it is nice to have MQ messages as records in a database table. It allows the user to do what ever they want to the MQMD header and/or the message data. There are lots of SQLite tools to manage/interact with an SQLite database. Plus, an SQLite database is a single file. So, a user can quickly compress/zip the file and move it another server without any issues.

I like the KISS (Keep It Simple Stupid) principle. So rather than have 1 program do both loading and unloading of messages, I broke it up into 2 light-weight programs. Ok, light-weight may be a bit of an understatement since they both have MQ and database calls. 🙂

  • MQ Queue To SQLite DB (MQ2SDB) program will offload MQ messages to an SQLite database.
  • SQLite DB To MQ Queue (SDB2MQ) program will load SQLite database rows into messages in an MQ queue.

I decided to write these 2 programs in C rather than Java. I like to mix things up. 🙂 I’ll do builds of them on AIX, HP-UX, IBM i, Linux, Solaris and Windows.

Both programs can be run from the command line or as an MQ Service. I would strongly suggest that people run MQ2SDB program as an MQ Service on the same queue manager where MQMR is configured as an MQ API Exit. Hence, when the queue manager is started, MQ2SDB will write MQ messages to the SQLite database right away.

MQ2SDB program will create the database file based on the queue manager’s name, the queue name and the current day’s date.
I.e.
{QMgrName}-{QueueName}-YYYY_MM_DD.mqsdb

When retrieving MQ messages and writing them to an SQLite database, MQ2SDB program will automatically roll to the next file at midnight (or when the next message arrives after midnight).

Here’s a screen-shot of DB Browser for SQLite with an SQLite database table that contains 100 records (messages). Click to see a larger image of it.


By default, when the SDB2MQ program is run, it will load all of the records in the database table as messages into the specified queue. The SDB2MQ program has 2 optional parameters (StartPosition and RowCount) to control where to start the load and how many records to load as messages into the queue.

If you interesting in trying it out, please send an email to support@capitalware.com to request a trial of MQ Message Replication with these new extra tools.

Note: If people are interested in offload/load messages to/from an SQLite database then I will extend this feature into MQ Visual Edit, MQ Visual Browse and MQ Batch Toolkit.

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, Database, IBM i (OS/400), IBM MQ, Linux, MQ Message Replication, Unix, Windows Comments Off on MQ Messages to/from SQLite DB

MQ Messages and GDPR – What are you doing about it?

March 2, 2018 Roger Lacroix

What is GDPR? From Wikipedia:

The General Data Protection Regulation (GDPR) (Regulation (EU) 2016/679) is a regulation by which the European Parliament, the Council of the European Union and the European Commission intend to strengthen and unify data protection for all individuals within the European Union (EU). It also addresses the export of personal data outside the EU. The GDPR aims primarily to give control back to citizens and residents over their personal data and to simplify the regulatory environment for international business by unifying the regulation within the EU.[1] When the GDPR takes effect, it will replace the data protection directive (officially Directive 95/46/EC)[2] of 1995. The regulation was adopted on 27 April 2016. It becomes enforceable from 25 May 2018 after a two-year transition period and, unlike a directive, it does not require national governments to pass any enabling legislation, and is thus directly binding and applicable.

May 25th, 2018 is only 3 months away. For those companies in the EU and any company that interacts with EU residences, what have you done to protect IBM MQ messages that contain personal data?

  • Is the message data encrypted?
  • Are the channels between queue managers using encryption?
  • Are the channels between client applications and the queue managers using encryption?
  • When client applications connect to a remote queue manager, are the client connections authenticated?

Capitalware has created and sells a variety of MQ solutions that solve the above listed questions:

* MQME does have support for Pub/Sub encryption. Several customers have been testing and using it for about 9 months. I just haven’t made the official announcement.

And if your company’s auditors want to know ‘who is doing what in MQ’, we also have created and sell the following MQ solutions:

Regards,
Roger Lacroix
Capitalware Inc.

Capitalware, IBM i (OS/400), IBM MQ, Linux, MQ Message Encryption, Security, Unix, Windows Comments Off on MQ Messages and GDPR – What are you doing about it?

Installing IBM MQ on Windows

March 1, 2018 Roger Lacroix

Over on IBM developerWorks, there are a couple of recent blog posting on how to install IBM MQ on Windows using MSIEXEC and a response file.

Installing MQ using MSIEXEC on Windows by Alamelu Nagarajan:
https://developer.ibm.com/messaging/2018/03/01/installing-mq-using-msiexec-windows/

Installation of IBM MQ components on Windows using Command Line by Prema Laxmanachar:
https://developer.ibm.com/messaging/2018/02/26/installation-ibm-mq-components-windows-using-command-line/

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ, Windows Comments Off on Installing IBM MQ on Windows

American Express and Me

March 1, 2018 Roger Lacroix

A long time ago, I mean a LONG time ago when I was 20, back in 1981 (like I said, a long time ago), I was working full time at a Chrysler/Dodge dealership and also working part-time at McDonald’s on the weekends.

I already had a Visa credit card for about 8-12 months (I don’t really remember) and saw the pamphlet (credit application) to get an American Express credit card. I decided to apply for it. I filled everything out, I had good credit history, I was working 2 jobs, over 60 hours a week and figured I was a shoe-in to get another credit card (Plus I was still living at home).

I mailed the credit application to American Express and waited – no online instant gratification in those days!! A few weeks later, I received a letter from American Express declining my application. I was shocked!! So I decided to call the number on the letter. We went over everything and person said that “I didn’t make enough money”. I reiterated that I was working 2 jobs and that as time goes on, my salary would increase. They said no. I even grovelled/begged for them to look past the “hard line about income” and give a hardworking person a chance. And they absolutely refused. Cold as ice!!

When I hung up the phone, I was upset and mad. And the more I thought about it, the madder I got. As my wife would say (I didn’t know her back then), “my French blood came out” and I swore I would never EVER have an American Express credit card.

Over the years, I have had various Visa and Mastercard credit cards and even a variety of store credit cards (i.e. Sears) but I have never ever had an American Express credit card.

My wife has asked me to get an American Express credit card over the years and I said no. The kids have bugged me to get one because of the “American Express Front of the Line” for purchasing concert tickets and I said no. That French blood of mine never forgets!!

Even when I started Capitalware, I refused to get an American Express for Business credit card. Nope. Not going to happen!

One of my favorite credit cards is the Marriott Rewards Visa. Years ago, I used to be a traveling consultant doing MQ work for various customers. I would always try to stay at a Marriott property, so that I could get/keep my status to/at “Platinum”. There is absolutely nothing better than having Platinum status and staying at an upscale Marriott property. They treat you like you are royalty. 🙂 Its awesome. 🙂

A month ago, I received a letter from Marriott Rewards Visa saying that the credit card was being discontinued at the end of March 2018. I was disappointed but I know reward companies do change credit cards every once in a while. I figured I would use it exclusively until it was no longer valid (rack up those points).

A few days ago, I received a letter from Marriott Rewards saying that they had signed up with American Express. Ahhhhh. No. No. No. Why them – the evil empire!!! Why couldn’t they have picked Mastercard!!!


I thought about it for a while and thought about it and thought about it some more. Yin and Yang. My love for Marriott against my hatred of American Express.

I showed my wife the letter from Marriott Rewards, she smiled and said “so, are you going to apply for it” and I said yes. That’s right, I said yes. The love of Marriott won out over my hatred of American Express (I’ve been pulled to the dark-sided!). So, I applied online for the SPG American Express Rewards credit card and bingo, I was approved. (Of course, it would have been funny as hell, if they had declined me. It would have justified my hatred of them.)

So, what’s my point to the story? Companies need to look beyond simple hard lines in the sand and sometimes say that a hard working individual does make up that gap between what they have and what is required. I’m a loyal guy – a company treats me well and I will stick with them. Over 37 years, look at all the business American Express lost out on by alienating me.

Regards,
Roger Lacroix
Capitalware Inc.

General Comments Off on American Express and Me

Now That’s a Trailer!!

February 23, 2018 Roger Lacroix

Early this week, the wife, kids and I went to see Black Panther (great movie). But before the Black Panther movie started, there was the usual assortment of trailers including the next trailer for Solo Star Wars. It was a fine trailer nothing that would make me say “I gotta see that movie”.

Last night, I stumble across someone’s recut of the Solo Star Wars trailer using the Beastie Boys Sabotage song. OMG! Now that’s how you do a trailer!!! If you haven’t see it, check this out – make it full-screen and turn UP the volume:

Lucusfilm (aka Disney) hire this person ASAP to do all of your trailers!!!

Regards,
Roger Lacroix
Capitalware Inc.

General, Video Comments Off on Now That’s a Trailer!!

IBM MQ on IBM Cloud

February 22, 2018 Roger Lacroix

Last month, IBM announced an experimental service for MQ on IBM Cloud:
https://www.ibm.com/blogs/bluemix/2018/01/experimental-service-mq-ibm-cloud/

Last week, IBM announced that MQ on IBM Cloud has moved into the “beta phase”:
https://www.ibm.com/blogs/bluemix/2018/02/mq-on-ibm-cloud-weve-hit-beta/

So, if you like your MQ in the SAAS (Software-As-A-Service) format then you should give the beta a try. If I can find some time this week, I’m going to give it a whirl. 🙂

Regards,
Roger Lacroix
Capitalware Inc.

IBM MQ Comments Off on IBM MQ on IBM Cloud

My Saga in Getting New Speakers for my PC

February 20, 2018 Roger Lacroix

A couple of weeks ago, my old Logitech 2.1 speaker system starting to crackle and pop in the right speaker, even with the PC off. After a few hours, it was driving me nuts, so it got pulled and trashed. It was over 10 years old, so I got good use out of the speaker system. 🙂

I did a bunch of searches and narrowed it down to either Audioengine A2+ 2.0 speaker system (no sub-woofer) for $329 CDN or Logitech z623 2.1 speaker system for $190 CDN.

The Audioengine A2+ speaker system has awesome reviews but no sub-woofer and most people suggest that you should also purchase the Audioengine DS1 Desktop Speaker Stands for another $55 CDN. So, it was getting a little pricey for the Audioengine A2+ plus I really do like having a deep and rich bass via a sub-woofer. Anyway, I wasn’t feeling well that week, so I put off the purchase.

Last week, I got an email from Amazon showing me all the speaker systems that they had on sale. I guess they were worried that I wasn’t going to buy anything. Note: I never put anything in my shopping-cart. So, they were definitely tracking what I was looking at. 🙂

The email had 12 speaker systems listed and of course both Audioengine A2+ and Logitech z623 were on the list but they had Logitech z623 listed for $149.98 CDN, $44 CDN off. 🙂 Well, that became a very easy purchase, click ‘Add to Cart’ and paid for the Logitech z623 on Thursday and it said they would be delivered on Tuesday (today) – yesterday was a holiday here.

Amazon showed that Purolator would be delivering the package. Purolator is owned by Canada Post and if you have read my previous blog postings, you know I hate Canada Post. It is toxic crown (government) organization that think everyone should worship the ground they walk on.

So, around 2:30PM today, I decided to check the status of my package. It showed that it was delivered at 1:04PM. I shook my head and looked again and said WTF!! I work at home and my office is next to the front door. I have a big window in front of my desk that I can see anyone driving up to my house. Plus, as many of you know, my wife home-schools our 5 kid, so there is ALWAYS someone at home.

I went outside and checked around the house, nothing. I checked the neighbors on both sides of our house, nothing. I checked the houses across the street from my house and nothing. (Of course, I looked like a thief casing the neighborhood). So I called Purolator and see what the @#$%# was going on with my package. The customer service rep was an arrogant ass. He blew me off, said there was nothing he could do and told me to complain to Amazon. I kept saying ‘why can’t he ask the deliver what actual house did he deliver the package too’. He said that there was nothing he could do, can’t contact the driver and I had to lodge a complaint with Amazon. And you wonder why I hate Canada Post and any company related to Canada Post.

I had a bitch session with my wife then went back to work, mad as hell. Around 4:30PM, the new neighbor (many houses down the street) showed up at my house. When I opened front door, the father and daughter were holding a VERY large box. I started to smile and the father said that they found this box at their front door and believe it belongs to me. I immediately thanked them (at least 5 times) and explained Purolator’s non-sense. I’m happy there are still honest people in this world because they could have easily kept it and nobody would be the wiser. Note: The shipping label on the box had my name, the correct address and was perfectly readable!!!

My wife said I should call Purolator again and tell them that our neighbor was doing their job. This time when I called, I actually got a useful customer service rep. who was (1) shocked at the incorrect delivery and (2) opened a case/ticket as to why it went to the wrong address.

So, I unboxed (no unboxing video) my new toy and hooked everything up. I fired up WinAmp, scrolled down to “The Glorious Sons” and then double clicked “The Union” album and turned up the volume. If you haven’t heard of The Glorious Sons, they are an excellent Southern-Rock (with a hint of Alt-Rock) band from Kingston, Ontario Canada. So, I played through that album and then moved on to “Royal Blood” and played their self-titled album. The boys and I saw Royal Blood a couple of summer’s ago as an opening act for Foo Fighters. They were awesome. I’ve never heard 2 guys make such complete and awesome sounding music in a live performance. Next up, was the “Foo Fighters” and their “Concrete and Gold” album. Then for some Euro content (ok – Icelandic), I played “Kaleo” and their “A+B” album (another Southern-Rock band with a hint of Alt-Rock). The last 2 albums, I played for the test was “Wintersleep’s” “The Great Detachment” album and “The Strumbellas’” “Hope” album.

So, after listening to 6 albums, I’ll give the Logitech z623 speaker system 2 great big thumbs up. 🙂 Ah music, it makes the world go around. 🙂 And reading application dumps so much easier!!! 🙂

Regards,
Roger Lacroix
Capitalware Inc.

General, Music Comments Off on My Saga in Getting New Speakers for my PC

Suganya Rane will be Speaking at MQTC v2.0.1.8

February 16, 2018 Roger Lacroix

Suganya Rane of Sabre will be speaking at MQ Technical Conference v2.0.1.8 (MQTC).

    Suganya Rane’s Sessions:

  • MQ Monitoring on the Cloud
  • MQ on AWS

For more information about MQTC, please go to:
http://www.mqtechconference.com

Regards,
Roger Lacroix
Capitalware Inc.

Education, IBM MQ, MQ Technical Conference Comments Off on Suganya Rane will be Speaking at MQTC v2.0.1.8