bvstone
MAILTOOL, JOBWATCH, G4SLK, G4MS and G4G Team Up to Monitor and Answer MSGW Errors
Posted: 09/16/2017, 11:59:52
UPDATE: The processing program has been updated to include not only a reply using Gmail, but also Office 365 and Slack!
Recently I was asked if there was any way that any BVSTools software could be used to answer MSGW messages. This issue was another piece of software was being used to answer MSGW via email, but that was all done via the IBM SMTP server. So finding replies in the IFS and parsing them wasn't too hard. But in this case, the customer was using a cloud email service (such as GMail or Office 365). That makes retrieving the replies to the emails a little more difficult as they are no longer just in folders in the IFS.
What I came with was a solution that involves the following:
- Use JOBWATCH and MAILTOOL and/or GreenTools for Slack (G4SLK) to send a specifically formatted email or message when a job enters MSGW status. (See this article for more info on that).
- Replying to the email (using any email client), or replying to the Slack message that will include the reply you want to send to the MSGW job.
- Use one or more of the following to read the reply and parse out the job information along with the reply so that we can call the Send Reply Message (QMHSNDRM) API and pass along the reply.
GreenTools for G Suite (G4G), specifically the G4GGMAIL addon
GreenTools for Microsoft Apps (G4MS), specifically the G4MSRMAIL addon
GreenTools for Slack (G4SLK), specifically the G4SLKCH addon
There was an issue, though. We can just read ALL the emails in an inbox and look for replies to MSGW emails. But, with a properly formatted email subject for the original email that is sent out when MSGW job goes out, along with creating a new email label and filter rule, it was easy enough to set up so we only will be reading emails/replies from a specific label. This process should work with Gmail and Outlook 365 accounts, although this demonstration will focus on a GMail account.
Adding an Entry to JOBWATCH to Send an Email
The first step is to use the Job Watch Configuration (JOBWATCHCF) command and add an entry to call our email program. The information we will need is the job name (&J), user id (&U) and job number (&N). We're also passing along the job status (&S), job function (&F) and text from the error (&E) simply for information to add to the email.
CALL PGM(BVSTONES/MSGWEMAILR) PARM('&J' '&U' '&N' '&S' '&F' '&E')
Setting Up the MSGW Email/Message So it is Unique
The next step was to format an email message so it would be unique, and we could use GMail filters to put them in a specific folder/label. What I decided was to use, in the subject of the email, my machine's serial number plus the text MSGW. So when the email was sent, the subject would start with SxxxxxxMSGW. I did try using special characters but the GMail Filters seemed to ignore them, so I instead use the machine serial number to make it unique.
Here is a the code use for sending the email (the program that is called from JOBWATCH):
H DFTACTGRP(*NO) BNDDIR('BVSTOOLS')
****************************************************************
* Prototypes *
****************************************************************
/COPY QCOPYSRC,P.MAILTOOL
/COPY QCOPYSRC,P.G4SLKCH
****************************************************************
D MsgTS S 256 INZ
D ErrMsg S 256 INZ
D rc S 10i 0
D msgID S 13s 0
D CRLF S 2 INZ(X'0D25')
D Subject S 32000
D Message S 32000
****************************************************************
C *ENTRY PLIST
C PARM WPJob 10
C PARM WPUser 10
C PARM WPJobNbr 6
C PARM WPJobStatus 4
C PARM WPFunction 10
C PARM WPErrorText 4096
/free
EXSR $EMail;
EXSR $Slack;
*INLR = *ON;
//***************************************************************
//* Send Email
//***************************************************************
begsr $EMail;
Subject = 'SxxxxxxxMSGW: Job ' + %trim(WPJob) + ' is in ' +
%trim(WPJobStatus) + ' status.';
Message = 'If you wish to answer this message, click the reply ' +
'button and replace <<REPLY>> with your reply.\n\n' +
'Do NOT alter anything else in the message.\n\n' +
'{reply:<<REPLY>>}\n' +
'{job:' + WPJob + WPUser + WPJobNbr + '}\n\n' +
'JOB: ' + %trim(WPJob) +
'\nJOB NUMBER: ' + %trim(WPJobNbr) +
'\nSTATUS: ' + %trim(WPJobStatus) +
'\nFUNCTION: ' + %trim(WPFunction) +
'\n\nMESSAGE:\n ' + %trim(WPErrorText);
if (#mailtool_init() >= 0);
rc = #mailtool_setValue('configuration_file':
'/bvstools/bvstone_mailtool.json');
rc = #mailtool_loadDefaults();
rc = #mailtool_addTORecipient('bvstone@bvstools.com');
rc = #mailtool_setValue('subject':Subject);
rc = #mailtool_setValue('message':Message);
rc = #mailtool_sendMail(errMsg:msgID);
endif;
if (#mailtool_init() >= 0);
rc = #mailtool_setValue('configuration_file':
'/bvstools/bvstone_office365.json');
rc = #mailtool_loadDefaults();
rc = #mailtool_addTORecipient('bvstone@bvstools.com');
rc = #mailtool_setValue('subject':Subject);
rc = #mailtool_setValue('message':Message);
rc = #mailtool_sendMail(errMsg:msgID);
endif;
endsr;
//***************************************************************
//* Send a Message to Slack
//***************************************************************
begsr $Slack;
Message = '{"job":"' + WPJob + WPUser + WPJobNbr + '"}' + CRLF +
'JOB: ' + %trim(WPJob) + CRLF +
'USER: ' + %trim(WPUser) + CRLF +
'JOB NUMBER: ' + %trim(WPJobNbr) + CRLF +
'STATUS: ' + %trim(WPJobStatus) + CRLF +
'FUNCTION: ' + %trim(WPFunction) + CRLF +
'MESSAGE: ' + CRLF + %trim(WPErrorText);
rc = #g4slkch_setValue('id':'bvstools');
rc = #g4slkch_setValue('team':'bvstools');
rc = #g4slkch_setValue('channel':'alerts');
rc = #g4slkch_setValue('as_who':'*BOT');
rc = #g4slkch_setValue('user_name':'JobWatch');
rc = #g4slkch_setValue('message':Message);
//rc = #g4slkch_setValue('debug':'*YES');
msgTS = #g4slkch_sendMessage(errMsg);
Message = 'To send a reply copy and paste this string, changing ' +
'<<reply>> to the reply you wish to send:' + CRLF + CRLF +
'{reply:<<REPLY>>}' +
'{job:' + WPJob + WPUser + WPJobNbr + '}';
rc = #g4slkch_setValue('id':'bvstools');
rc = #g4slkch_setValue('team':'bvstools');
rc = #g4slkch_setValue('channel':'alerts');
rc = #g4slkch_setValue('as_who':'*BOT');
rc = #g4slkch_setValue('user_name':'JobWatch');
rc = #g4slkch_setValue('message':Message);
//rc = #g4slkch_setValue('debug':'*YES');
msgTS = #g4slkch_sendMessage(errMsg);
endsr;
/end-free
When JOBWATCH finds a job in the MSGW status this program sends 2 emails, one from a gmail account and the other from an Office 365 account. Following is an example:
|
|
When the user receives this email, they click Reply and replace <<REPLY>> with the answer they wish to send. In this case, we'll use C for cancel. When the email is returned to the sender the top portion will look like this:
{reply:C}
{job:BVSTONE BVSTONE 124289}
So now all we need to do is parse out the reply and job information.
The user will also receive a slack message in a special private channel created named "alerts".
The user is instructed to copy and paste the last line and replace <<REPLY>> with the reply. Tested on both the web and android app for slack, the copying is actually super easy!
Reading The EMail/Slack Reply Using G4G and G4SLK
When looking for replies to these MSGW emails you'll recall that we want them to go to a specific folder or label.
First you will want to create a new label in the GMail account that sends the MSGW email (and also will receive the replies).
Next we want to set up a filter to send our replied to emails to this label. To do this in GMail enter the Settings for the account and click on the Filters and Blocked Addresses option. I then set up a filter that "Has the Words" SxxxxxxxMSGW (remember, that is our unique ID) and have it skip the Inbox and move them into my new MSGW label.
Next, because we will only be processing emails from our MSGW label we need to find the Label ID for this folder. This is a unique ID used by Google. This is done using the List Gmail Labels (G4GLSTML) command included in the G4G software package:
G4GLSTML ID(bvstone@gmail.com)
This will populate a file in our G4G library named G4GMLPF. The file looks like this:
File Name . . . . G4GMLPF
Library . . . . G4G
Format Descr . .
Format Name . . . RG4GML
File Type . . . . PF Unique Keys - N
Field Name FMT Start Lngth Dec Key Field Description
GGMLID A 1 256 Google ID
GGMLLID A 257 256 Label ID
GGMLLNM A 513 256 Label Name
GGMLTYPE A 769 64 Label Type
GGMLTMSG P 833 13 00 Total Messages
GGMLUMSG P 840 13 00 Unread Messages
We then use any method you want to find the Label ID (GGMLLID) for our label of MSGW. I chose to use SQL:
select GGMLLID from G4GMLPF where GGMLID = 'bvstone@gmail.com'
and GGMLLNM = 'MSGW'
The value returned is "Label_15" in my case. This should always remain the same unless you delete and recreate the label. Then you will want to retrieve the new label ID.
For the new additions using Microsoft Office 365 and Slack the instructions are similar.
Finally, we set up a program to read messages from this label and process the reply.
This next program also includes what is needed to open and read the email or reply to the slack message (which are retrieved into the IFS), retrieve the message key information required to send the reply (using the QUSRJOBI API, which since V6R1 now returns the message key!), and finally send the reply itself using the QMHSNDRM API.
H DFTACTGRP(*NO) ACTGRP('G4G') BNDDIR('BVSTOOLS')
****************************************************************
* prototypes
****************************************************************
/copy qcopysrc,p.g4gmail
/copy qcopysrc,p.g4msmail
/copy qcopysrc,p.g4slkch
*
D QmhSndRm PR ExtPgm('QMHSNDRM')
D MsgKey Like(MsgKey)
D MsgQ Like(MsgQ)
D MsgRpy Like(MsgRpy)
D MsgRpyLen Like(MsgRpyLen)
D RmvMsg Like(RmvMsg)
D WPError Like(WPError)
*
D QUsrJobI PR ExtPgm('QUSRJOBI')
D JOBI0200 Like(JOBI0200)
D RcvLen Like(RcvLen)
D FmtName Like(FmtName)
D QJobName Like(JobInfo)
D WPIntJobID Like(WPIntJobID)
D WPError Like(WPError)
*
D openStmf PR 10I 0 ExtProc('open')
D path * Value options(*String)
D oflag 10I 0 Value
D mode 10U 0 Value Options(*NOPASS)
D ccsid 10U 0 Value Options(*NOPASS)
*
D readStmf PR 10I 0 EXTPROC('read')
D fd 10I 0 Value
D buffer * Value
D bufferLen 10U 0 Value
*
D closeStmf PR 10I 0 Extproc('close')
D fd 10I 0 Value
****************************************************************
D O_CCSID C 32
D O_RDONLY C 1
D O_TEXTDATA C 16777216
*
D JOBI0200 Ds
D BytRtn 10i 0
D BytAvl 10i 0
D JobName 10
D UserName 10
D JobNumber 6
D IntJobID 16
D JobStatus 10
D JobType 1
D JobSubType 1
D SbsDescName 10
D RunPty 10i 0
D SysPoolId 10i 0
D PrcTime 10i 0
D NbrIO 10i 0
D NbrIntSess 10i 0
D RspTimeTot 10i 0
D FuncType 1
D FuncName 10
D ActJobSts 4
D NbrDBLW 10i 0
D NbrIntMLW 10i 0
D NbrnonDBLW 10i 0
D DBLWTime 10i 0
D IntMLWTime 10i 0
D nonDBLWTime 10i 0
D Res1 1
D CurSysPID 10i 0
D ThreadCount 10i 0
D PrcTimeTotal 20i 0
D NbrAuxIOR 20i 0
D PrcUTforDB 20i 0
D PageFaults 20i 0
D ActJobSts4End 4
D MemPoolName 10
D MsgReply 1
D MsgKey 4
D MsgQName 10
D MsgQLib 10
D MsgQLibASPDN 10
*
D WPError DS
D EBytesP 1 4B 0 INZ(60)
D EBytesA 5 8B 0 INZ
D EMsgID 9 15 INZ
D EReserverd 16 16 INZ
D EData 17 76 INZ
*
D RcvLen S 10i 0 INZ(%size(JOBI0200))
D FmtName S 8 INZ('JOBI0200')
D QJobName S 26 INZ('*INT')
D WPIntJobID S 16
*
D slackID S 256
D slackUser S 256 INZ('bvstools')
D msgFile S 256
D message S 65535
D scanText S 65535 Varying
D tempString S 65535 Varying
D jobInfo S 26
D messageCount S 10i 0
D fd S 10i 0
D rc S 10i 0
D i S 10i 0
D j S 10i 0
*
D MsgQ S 20
D MsgRpy S 10
D MsgRpyLen S 10i 0
D RmvMsg S 10 INZ('*NO')
****************************************************************
/free
// get GMail Email
#g4gmail_setValue('id':'bvstone@gmail.com');
#g4gmail_setValue('label_id':'Label_15');
#g4gmail_setValue('get_new_only':'*YES');
#g4gmail_setValue('mark_as_read':'*YES');
//#g4gmail_setValue('debug':'*YES');
messageCount = #g4gmail_listMessages();
if (messageCount > 0);
EXSR $Process;
endif;
// get Office 365 Email
#g4msmail_setValue('id':'bvstone@bvstools.onmicrosoft.com');
#g4msmail_setValue('label_id':'AAMkADlhYzk2NzE1LWEyMjYtNGVjYS1iZWY4L' +
'WViNzU1M2E4NGM5NAAuAAAAAACJhuGQKonEQrabmZaor0-' +
'4AQDrO2ySXXeDSb7Y9VeUhICtAAE7DNt0AAA=');
#g4msmail_setValue('get_new_only':'*YES');
#g4gmail_setValue('mark_as_read':'*YES');
//#g4msmail_setValue('debug':'*YES');
messageCount = #g4msmail_listMessages();
if (messageCount > 0);
EXSR $Process2;
endif;
#g4slkch_setValue('id':slackUser);
#g4slkch_setValue('team':'bvstools');
#g4slkch_setValue('user_name':slackUser);
slackID = #g4slkch_getUserIDFromName();
// get Slack Responses
#g4slkch_setValue('id':slackUser);
#g4slkch_setValue('team':'bvstools');
#g4slkch_setValue('channel':'alerts');
#g4slkch_setValue('clear_history':'*YES');
//#g4slkch_setValue('debug':wpDebug);
messageCount = #g4slkch_getGroupHistory();
if (messageCount > 0);
EXSR $Process3;
endif;
EXSR $Return;
//***************************************************************
//* Process
//***************************************************************
begsr $Process;
exec sql
declare C1 cursor for
select GGMPBODYL from G4GMPPF
where lower(GGMPID) = 'bvstone@gmail.com'
limit 1;
exec sql open C1;
exec sql fetch from C1
into :MsgFile;
dow (SQLCOD = 0);
fd = openStmf(%trimr(MsgFile):
O_RDONLY + O_TEXTDATA + O_CCSID:
0:0);
if (fd >= 0);
rc = readStmf(fd:%addr(message):%size(message));
closeStmf(fd);
if (rc > 0);
EXSR $DoReply;
endif;
endif;
exec sql fetch from C1
into :MsgFile;
enddo;
exec sql
close C1;
endsr;
//***************************************************************
//* Process2
//***************************************************************
begsr $Process2;
exec sql
declare C2 cursor for
select GMMBLOC from G4MSMBPF
where lower(GMMBID) = 'bvstone@bvstools.onmicrosoft.com'
limit 1;
exec sql open C2;
exec sql fetch from C2
into :MsgFile;
dow (SQLCOD = 0);
fd = openStmf(%trimr(MsgFile):
O_RDONLY + O_TEXTDATA + O_CCSID:
0:0);
if (fd >= 0);
rc = readStmf(fd:%addr(message):%size(message));
closeStmf(fd);
if (rc > 0);
EXSR $DoReply;
endif;
endif;
exec sql fetch from C2
into :MsgFile;
enddo;
exec sql
close C2;
endsr;
//***************************************************************
//* Process3
//***************************************************************
begsr $Process3;
exec sql
declare C3 cursor for
select GGHMSGLNK from G4SLKGHPF
where lower(GGHID) = :slackUser and
lower(GGHTEAM) = 'bvstools' and
GGHGROUP = 'G5L485K6H' and
GGHUSER = :slackID;
exec sql open C3;
exec sql fetch from C3
into :MsgFile;
dow (SQLCOD = 0);
fd = openStmf(%trimr(MsgFile):
O_RDONLY + O_TEXTDATA + O_CCSID:
0:0);
if (fd >= 0);
rc = readStmf(fd:%addr(message):%size(message));
closeStmf(fd);
if (rc > 0);
EXSR $DoReply;
endif;
endif;
exec sql fetch from C3
into :MsgFile;
enddo;
exec sql
close C3;
endsr;
//***************************************************************
//* Find and process the reply
//***************************************************************
begsr $DoReply;
// Find Reply
scanText = '{reply:';
i = %scan(scanText:Message);
if (i <= 0);
LEAVESR;
endif;
i += %len(scanText);
scanText = '}';
j = %scan(scanText:Message:i);
if (j <= i);
LEAVESR;
endif;
tempString = %trim(%subst(Message:i:j-i));
MsgRpy = %scanrpl(' ':' ':tempString);
MsgRpyLen = %len(%trim(MsgRpy));
// Find Job Info
scanText = '{job:';
i = %scan(scanText:Message:i);
if (i <= 0);
LEAVESR;
endif;
i += %len(scanText);
scanText = '}';
j = %scan(scanText:Message:i);
if (j <= i);
LEAVESR;
endif;
tempString = %trim(%subst(Message:i:j-i));
JobInfo = %scanrpl(' ':' ':tempString);
QUsrJobI(JOBI0200:RcvLen:FmtName:JobInfo:WPIntJobID:WPError);
if (MsgKey <> ' ');
MsgQ = MsgQName + MsgQLib;
QmhSndRm(MsgKey:MsgQ:MsgRpy:MsgRpyLen:RmvMsg:WPError);
endif;
endsr;
//***************************************************************
//* Return
//***************************************************************
begsr $return;
*INLR = *ON;
return;
endsr;
/end-free
So, the first step is to retrieve all unread messages that have our MSGW label (or are in the alerts channel in Slack).
For G4G, when that is done, we read through the G4GMPPF file. This file contains information for "parts" of our email messages as well as the location(s) for the bod of the message. Parts can be attachments, images, etc. What we are concerned about is the actual message, or body, itself. The G4GMPPF file looks like this:
File Name . . . . G4GMPPF
Library . . . . G4G
Format Descr . .
Format Name . . . RG4GMP
File Type . . . . PF Unique Keys - N
Field Name FMT Start Lngth Dec Key Field Description
GGMPID A 1 256 Google ID
GGMPMID A 257 256 Message ID
GGMPPID A 513 256 Part ID
GGMPMIME A 769 256 Mime Type
GGMPBODYL A 1025 256 Body Location
GGMPFILEN A 1281 1024 Filename
GGMPATTID A 2305 1024 Attachment ID
GGMPATTL A 3329 256 Attachment Location
GGMPERR A 3585 10 Error Code
For Office 365 and Slack, again, the process is similar.
Our SQL statement we are using reads through this file using our GMail ID, and it also only retrieves 1 record. Because emails can be send as text and/or HTML, this file could contain more than 1 entry for the body of the message. In our case we don't care if it's the text or HTML version, we just want one of them to process.
For each record we find we use the GGMPBODYL field to get the fully qualified path to the message body which is stored in the IFS. We then open that file, read it's contents, and close the file.
We then scan through the message for the reply the user sent as well as the job information that was also in the email. If you recall, it looked like this:
{reply:C}
{job:BVSTONE BVSTONE 124289}
UPDATE: After doing some testing with Microsoft Office 365 I saw that they were returning only HTML. The job info (and possibly the reply) was getting some spaces replaced with . Because of this we added a couple %scanrpl() BIF calls to replace with a space.
Once we have this, from G4G, G4MS or G4SLK, we call the QUSRJOBI API to retrieve the message key and message queue name and library which are then finally used, along with the reply, to call the QMHSNDRM and answer the message.
Final Thoughts
Things to consider:
- The program that processes the MSGW label in our email account needs to be on a loop (say, running every 5-10 minutes). Either that or set up a web hook to call your program using a web service when a new MSGW email arrives.
- Another option, instead of replying to an email, would be creating a hyperlink that connects back to your IBM i passing the reply and job information that way. This way it will be MUCH easier to get the data itself.
- Formatting the subject so that it is something unique may take a few tries, but it's not so bad. Just remember that, at least for GMail, special characters are ignored. So if you set up a filter looking only for something like "MSGW" this will affect ALL emails with MSGW in them. I was surprised to find out just how many of those I had in my account when I first set this up (using MSGW only).
- Formatting the data in the email, the reply information and job information, also could be tricky. I chose to use a psudo-JSON format. The important part to remember is that if the email reply is text or HTML (which you will have no control over) that any HTML markups will not corrupt your data when you are searching for it.
- Using more APIs to retrieve the actual allowable replies could be an option.
- If this process will be used in more than one application (for example letting them reply OR click on a link) a lot of the processing could be cleaned up into more easy to use ILE subprocedures so you're not copying code from program to program. (Yes! ILE is awesome!)
As always, feel free to contact me with any questions, or reply here and ask questions, make comments, etc. Thanks!
Last edited 09/17/2017 at 12:10:52
Reply
© Copyright 1983-2024 BVSTools
GreenBoard(v3) Powered by the eRPG SDK, MAILTOOL Plus!, GreenTools for Google Apps, jQuery, jQuery UI, BlockUI, CKEditor and running on the IBM i (AKA AS/400, iSeries, System i).