bvstone

Reading more than 32k or 64k using QtmhRdStin

Posted:

Reading more than 32k or 64k using QtmhRdStin

Recently there was a discussion on one of the Midrange-L mailing lists where someone wanted to be able to read in a very large amount of data from a web service.

This brought about the discussion of the largest string an RPG program could use was 32k, then 64k (approximately), and now 16mb (with V6R1 and up).

But what if you want to read more than that?  Well, this is something I always wanted to try so I did it.  Instead of using a string as the receiver variable on the QtmhRdStin API I decided to try using a pointer.  The following is what I came up with:

     H DFTACTGRP(*NO) BNDDIR('ERPGSDK')
      ****************************************************************
      * Prototypes                                                   *
      ****************************************************************
      /COPY QCOPYSRC,P.ERPGSDK
      /COPY QCOPYSRC,P.HTTPSTD
      /COPY QCOPYSRC,P.IFSSTD
      /COPY QCOPYSRC,P.STRINGS
      *
     D #RdStinPtr      PR                  ExtProc('QtmhRdStin')
     D  RcvRec                         *   Value
     D  RcvLen                       10I 0 Const
     D  RcvRecLen                    10I 0
     D  WPError                      56
      ****************************************************************
      * Work Variables
      ****************************************************************
     D WPError         DS
     D  EBytesP                      10i 0 INZ(%size(WPError))
     D  EBytesA                      10i 0
     D  EMsgID                        7
     D  EReserverd                    1
     D  EData                        40
      *
      * Data read in from page
     D inContLen       S             10i 0
      *
     D data@           S               *
     D rspLen          S             10i 0
     D fd              S             10i 0
      ****************************************************************
      /free
       #startup();
       inContLen = #st_CtoN(#GetEnv('CONTENT_LENGTH'));
       #writeTemplate('stdhtmlheader.erpg');
       #loadTemplate('readbig.erpg':'top');
       #replaceData('/%length%/':%char(inContLen));
       #writeSection();

       if (inContLen > 0);
         data@ = %alloc(inContLen);

         if (data@ <> *null);
           exsr $WriteToStmf;
           dealloc data@;
         endif;

       endif;
       #cleanup();

       *INLR = *on;
       //-------------------------------------------------------------/
       // Write Standard Input to Stream File                         /
       //-------------------------------------------------------------/
       begsr $WriteToStmf;

         #RdStinPtr(data@:inContLen:rspLen:WPError);

         if (rspLen > 0);

           fd = #openStmf('/tmp/bigposttest.txt':1);

           if (fd >= 0);
             #writeStmf(fd:data@:rspLen);
             #closeStmf(fd);
           endif;

         endif;

       endsr;
      /end-free

Now, I'm using our eRPG SDK here to make things a little easier.  But the process is quite simple...

  1. Get the Content Length of the request.  This should be available as an Environment Variable named CONTENT_LENGTH and represent the size of the data in the POST made to the server.
  2. If this is larger than zero, use the %alloc BIF to allocate the space needed using a pointer in your program. 
  3. Use the pointer from step 2 as the receiver for the QtmhRdStin API.
  4. Now that you have a pointer to your data, you can manipulate it or, as I did, write it all to a stream file

This application also does return a response to the requesting application with the length of the data received.  The very simple template for that is as follows:

/$top
Content-Length: /%length%/

How I made the request use using our GETURI application to perform at HTTP request as follows:

GETURI URI('myserver/cgi-bin/readbig') DATA(*STMF) DSTMF('/tmp/test.txt') METHOD(*POST)

The file used was about 2mb just for the test.  This was also running on a V5R4 machine so the maximum size of a string is 64k.

This is nothing new, and with newer OS versions isn't really that necessary unless the data you plan on reading from a web application is larger than 16mb, but it was a fun exercise that I've wanted to try for quite some time, and finally made the time for.


Last edited 02/22/2015 at 07:13:18



Latest Posts:

GreenTools For G Suite (G4G) v12.00 Released With Base OAuth 2.0 Functionality GreenTools For G Suite (G4G) v12.00 Released With Base OAuth 2.0 Functionality
Posted by July 28, 2019
BVSTools >> BVSTools Announcements >> GreenTools for G Suite (Google Apps) (G4G) Specific Announcements
BVSTools Small Price Increase in 2020 BVSTools Small Price Increase in 2020
Posted by July 26, 2019
BVSTools >> BVSTools Announcements
GreenTools for Vertex Cloud (VTXCLOUD) Now Available GreenTools for Vertex Cloud (VTXCLOUD) Now Available
Posted by July 22, 2019
BVSTools >> BVSTools Announcements >> GreenTools for Vertex Cloud (VTXCLOUD) Specific Announcements
GreenTools for Google Apps (G4G) - Drive Addon Successfully Verified by Google GreenTools for Google Apps (G4G) - Drive Addon Successfully Verified by Google
Posted by July 22, 2019
BVSTools >> BVSTools Announcements >> GreenTools for G Suite (Google Apps) (G4G) Specific Announcements
Why I Cancelled my DynDNS Service and How I Replaced It with an IBM i Application Why I Cancelled my DynDNS Service and How I Replaced It with an IBM i Application
Posted by July 17, 2019
IBM Power Systems >> (QGPL) IBM i
Green Tools for G Suite (G4G) Product Updates (Licensing, Functionality, Base Product) Green Tools for G Suite (G4G) Product Updates (Licensing, Functionality, Base Product)
Posted by July 13, 2019
BVSTools >> BVSTools Announcements >> GreenTools for G Suite (Google Apps) (G4G) Specific Announcements
Reading JSON Data from Standard Input With YAJL and RPG Reading JSON Data from Standard Input With YAJL and RPG
Posted by July 12, 2019
Programming >> Proof of Concept (POC)
MAILTOOL Updated to Allow Use of IBM Global Security Kit (GSKIT) for SSL/TLS Communications MAILTOOL Updated to Allow Use of IBM Global Security Kit (GSKIT) for SSL/TLS Communications
Posted by June 19, 2019
BVSTools >> BVSTools Announcements >> eMail Tool (MAILTOOL) Specific Announcements
GETURI v10.00 Released Supporting IBM Global Security Kit (GSKIT) and Server Name Indication (SNI) GETURI v10.00 Released Supporting IBM Global Security Kit (GSKIT) and Server Name Indication (SNI)
Posted by June 11, 2019
BVSTools >> BVSTools Announcements >> Get URI (GETURI) Specific Announcements
BVSTools Now Offers Vertex Cloud Interface BVSTools Now Offers Vertex Cloud Interface
Posted by April 15, 2019
BVSTools >> BVSTools Announcements
Token Has an Invalid Signature Error for Office 365 Email Token Has an Invalid Signature Error for Office 365 Email
Posted by March 22, 2019
BVSTools >> BVSTools Software Discussion >> GreenTools for Microsoft Apps (G4MS) Specific Discussion
Resending Emails that have Errored Out with Updated Router or Authentication Information Resending Emails that have Errored Out with Updated Router or Authentication Information
Posted by March 1, 2019
BVSTools >> BVSTools Software Discussion >> Email Tools (MAILTOOL) Specific Discussion
BVSTools Offers Toolset to Work With HubSpot OAuth 2.0 APIs On Your IBM i BVSTools Offers Toolset to Work With HubSpot OAuth 2.0 APIs On Your IBM i
Posted by January 27, 2019
BVSTools >> BVSTools Announcements
G4MSDRV Currently Not Supported G4MSDRV Currently Not Supported
Posted by January 17, 2019
BVSTools >> BVSTools Announcements >> GreenTools for Microsoft Apps (G4MS) Specific Announcements
Removing Trailing Carriage Returns and/or Line Feeds from a String with RPG Removing Trailing Carriage Returns and/or Line Feeds from a String with RPG
Posted by December 26, 2018
Programming >> RPG Programming

Reply




Copyright 1983-2019 BVSTools
GreenBoard(v3) Powered by the eRPG SDK, MAILTOOL Plus!, GreenTools for Google Apps, jQuery, jQuery UI, BlockUI, CKEditor and running on the IBM i (AKA AS/400, iSeries, System i).