bvstone

Reading more than 32k or 64k using QtmhRdStin

Posted:

Reading more than 32k or 64k using QtmhRdStin

Recently there was a discussion on one of the Midrange-L mailing lists where someone wanted to be able to read in a very large amount of data from a web service.

This brought about the discussion of the largest string an RPG program could use was 32k, then 64k (approximately), and now 16mb (with V6R1 and up).

But what if you want to read more than that?  Well, this is something I always wanted to try so I did it.  Instead of using a string as the receiver variable on the QtmhRdStin API I decided to try using a pointer.  The following is what I came up with:

     H DFTACTGRP(*NO) BNDDIR('ERPGSDK')
      ****************************************************************
      * Prototypes                                                   *
      ****************************************************************
      /COPY QCOPYSRC,P.ERPGSDK
      /COPY QCOPYSRC,P.HTTPSTD
      /COPY QCOPYSRC,P.IFSSTD
      /COPY QCOPYSRC,P.STRINGS
      *
     D #RdStinPtr      PR                  ExtProc('QtmhRdStin')
     D  RcvRec                         *   Value
     D  RcvLen                       10I 0 Const
     D  RcvRecLen                    10I 0
     D  WPError                      56
      ****************************************************************
      * Work Variables
      ****************************************************************
     D WPError         DS
     D  EBytesP                      10i 0 INZ(%size(WPError))
     D  EBytesA                      10i 0
     D  EMsgID                        7
     D  EReserverd                    1
     D  EData                        40
      *
      * Data read in from page
     D inContLen       S             10i 0
      *
     D data@           S               *
     D rspLen          S             10i 0
     D fd              S             10i 0
      ****************************************************************
      /free
       #startup();
       inContLen = #st_CtoN(#GetEnv('CONTENT_LENGTH'));
       #writeTemplate('stdhtmlheader.erpg');
       #loadTemplate('readbig.erpg':'top');
       #replaceData('/%length%/':%char(inContLen));
       #writeSection();

       if (inContLen > 0);
         data@ = %alloc(inContLen);

         if (data@ <> *null);
           exsr $WriteToStmf;
           dealloc data@;
         endif;

       endif;
       #cleanup();

       *INLR = *on;
       //-------------------------------------------------------------/
       // Write Standard Input to Stream File                         /
       //-------------------------------------------------------------/
       begsr $WriteToStmf;

         #RdStinPtr(data@:inContLen:rspLen:WPError);

         if (rspLen > 0);

           fd = #openStmf('/tmp/bigposttest.txt':1);

           if (fd >= 0);
             #writeStmf(fd:data@:rspLen);
             #closeStmf(fd);
           endif;

         endif;

       endsr;
      /end-free

Now, I'm using our eRPG SDK here to make things a little easier.  But the process is quite simple...

  1. Get the Content Length of the request.  This should be available as an Environment Variable named CONTENT_LENGTH and represent the size of the data in the POST made to the server.
  2. If this is larger than zero, use the %alloc BIF to allocate the space needed using a pointer in your program. 
  3. Use the pointer from step 2 as the receiver for the QtmhRdStin API.
  4. Now that you have a pointer to your data, you can manipulate it or, as I did, write it all to a stream file

This application also does return a response to the requesting application with the length of the data received.  The very simple template for that is as follows:

/$top
Content-Length: /%length%/

How I made the request use using our GETURI application to perform at HTTP request as follows:

GETURI URI('myserver/cgi-bin/readbig') DATA(*STMF) DSTMF('/tmp/test.txt') METHOD(*POST)

The file used was about 2mb just for the test.  This was also running on a V5R4 machine so the maximum size of a string is 64k.

This is nothing new, and with newer OS versions isn't really that necessary unless the data you plan on reading from a web application is larger than 16mb, but it was a fun exercise that I've wanted to try for quite some time, and finally made the time for.


Last edited 02/22/2015 at 07:13:18



Latest Posts:

MAILTOOL Updated to Allow G4GSMAIL as Option in Routers File MAILTOOL Updated to Allow G4GSMAIL as Option in Routers File
Posted by June 28, 2020
BVSTools >> BVSTools Announcements >> eMail Tool (MAILTOOL) Specific Announcements
BVSTools Now Offers Interface with Infor's ION APIs BVSTools Now Offers Interface with Infor's ION APIs
Posted by May 15, 2020
BVSTools >> BVSTools Announcements
More V7R4 IFS File CCSID Issues and The Fix More V7R4 IFS File CCSID Issues and The Fix
Posted by March 4, 2020
IBM Power Systems >> (QGPL) IBM i
Error Retrieving IP Address by Name Error Retrieving IP Address by Name
Posted by February 25, 2020
BVSTools >> BVSTools Software Discussion
Logging jobs that hit an outq Logging jobs that hit an outq
Posted by February 13, 2020
Programming >> CL Programming
GreenTools for Google Apps (G4G) v12.60 Released with Shared Drive Features and More... GreenTools for Google Apps (G4G) v12.60 Released with Shared Drive Features and More...
Posted by February 4, 2020
BVSTools >> BVSTools Announcements >> GreenTools for G Suite (Google Apps) (G4G) Specific Announcements
Allowing Requests over Port 80 For SSL Validation (ie, Namecheap, etc) Allowing Requests over Port 80 For SSL Validation (ie, Namecheap, etc)
Posted by January 31, 2020
Programming >> Web Programming
GreenTools for Slack (G4SLK) v3.00 Released GreenTools for Slack (G4SLK) v3.00 Released
Posted by January 17, 2020
BVSTools >> BVSTools Announcements >> GreenTools for Slack (G4SLK) Specific Announcements
Calling a QSH Command from RPG Calling a QSH Command from RPG
Posted by December 26, 2019
Programming >> RPG Programming
SPLTOOL Print Range (PRTRNG) Function Updated to Handle Spooled Files up to 999,999,999 Pages SPLTOOL Print Range (PRTRNG) Function Updated to Handle Spooled Files up to 999,999,999 Pages
Posted by December 14, 2019
BVSTools >> BVSTools Announcements >> Spooled File Tools (SPLTOOL) Specific Announcements
GreenTools for Microsoft Apps (G4MS) Updated to v6.00 - Now Uses Microsoft Graph APIs GreenTools for Microsoft Apps (G4MS) Updated to v6.00 - Now Uses Microsoft Graph APIs
Posted by November 24, 2019
BVSTools >> BVSTools Announcements >> GreenTools for Microsoft Apps (G4MS) Specific Announcements
V7R4 Changes CCSID of Compressed File Using PASE JAR Command - Here's The Fix V7R4 Changes CCSID of Compressed File Using PASE JAR Command - Here's The Fix
Posted by November 21, 2019
IBM Power Systems >> (QGPL) IBM i
Using GETURI to Make OAuth 2.0 Requests - Custom Headers for Access Tokens Using GETURI to Make OAuth 2.0 Requests - Custom Headers for Access Tokens
Posted by November 11, 2019
BVSTools >> BVSTools Software Discussion >> Get URI (GETURI) Specific Discussion
GreenTools for Microsoft Apps (G4MS) v5.00 Released with Updated OneDrive Support and 3rd Party Functionality GreenTools for Microsoft Apps (G4MS) v5.00 Released with Updated OneDrive Support and 3rd Party Functionality
Posted by October 20, 2019
BVSTools >> BVSTools Announcements >> GreenTools for Microsoft Apps (G4MS) Specific Announcements
BVSTools is Now Running V7R4M0 BVSTools is Now Running V7R4M0
Posted by September 28, 2019
BVSTools >> BVSTools Announcements

Reply




Copyright 1983-2020 BVSTools
GreenBoard(v3) Powered by the eRPG SDK, MAILTOOL Plus!, GreenTools for Google Apps, jQuery, jQuery UI, BlockUI, CKEditor and running on the IBM i (AKA AS/400, iSeries, System i).