Best way to compress streams/files in OE.

Posted by pliscki on 26-Jan-2016 06:22

Hi

I'm working on a project where I need to compress XML, convert to Base64 and send it to a SOAP web server.

I couldn't find a better way to do this, without having to export the XML to some temp directory, execute a external lib to compress the file and then import the compressed file so I could convert it to Base64 and send it.

I was wondering if there was a better solution for this.

Any suggestions?

Thanks

OpenEdge Development - Forum

All Replies

Posted by Brian K. Maher on 26-Jan-2016 06:44

What version of OpenEdge?
 
What operating system?
 
If using .NET classes is an option take a look at the System.IO.Compression namespace.
 
Brian

Posted by James Palmer on 26-Jan-2016 06:47

Brian got there first. But if you have access to the .NET classes then this is a good way to go. I set up compression of all the blobs in a database and it didn't take long to sort out at all.

Posted by pliscki on 26-Jan-2016 07:00

Hi Brian,

Thanks for replying.

Current OE version is 10.2b, although we've already started working with 11.6.

OS is windows server.

We're not using .net classes, but I'll take a look either way.

Posted by Brian K. Maher on 26-Jan-2016 07:10

For 10.2B, you should google for zlib, winzip and pkzip.  I remember that one or more of them provide a dll interface which you can use.  I expect it will be more complex than the .NET classes.

Posted by AdrianJones on 26-Jan-2016 07:44

there is a "standard library" on eo hive with some zlib content...

www.oehive.org/.../lib

NB. haven't used it myself.

Posted by Mark Bartscherer on 26-Jan-2016 07:44

If you use the WRITE-XML() Method you can use a MEMPTR as target-type. To encode just use the base64-encode function passing the memptr. You will get a longchar with base64 encoded text Is this what you are looking for?

Posted by gus on 26-Jan-2016 08:16

> On Jan 26, 2016, at 8:11 AM, Brian K. Maher wrote:

>

> For 10.2B, you should google for zlib,

zlib is part of OpenEdge. You should have it in the DLC/bin or lib directory.

--

regards,

gus (gus@progress.com)

"If I set here and stare at nothing long enough, people might think I'm an engineer working on something."

-- S.R. McElroy

Posted by Peter Judge on 26-Jan-2016 08:24

While true, there are some caveats, not least of which is that it doesn't ship with all  products (only with the dev products, IIRC).
 
Working with this library is also ... fun ... since you end up doing Ye Olde C-style Coding (nothing wrong with that, if you like that sort of thing, but it's less pleasant in ABL than you might imagine). There's a MS tool called dependency walker that's really helpful when working with DLLs etc.
 
If you can use the .NET classes, that will probably be much simpler.
 

"If I set here and stare at nothing long enough, people might think I'm an engineer working on something."

https://images.duckduckgo.com/iu/?u=http%3A%2F%2Fecx.images-amazon.com%2Fimages%2FI%2F41vxxkR86xL.jpg&f=1
 
;)
 
 

Posted by gus on 26-Jan-2016 08:48

> On Jan 26, 2016, at 9:24 AM, Peter Judge wrote:

>

> While true, there are some caveats, not least of which is that it doesn't ship with all products (only with the dev products, IIRC).

it was put in for the 4GL client / appserver stream compression

it is actually very simple to use but i do not have any 4gl examples. but there are on oehive

--

regards,

gus (gus@progress.com)

"If I set here and stare at nothing long enough, people might think I'm an engineer working on something."

-- S.R. McElroy

Posted by pliscki on 26-Jan-2016 09:41

Is it possible to use .net class DeflateStream  in 10.2b?

I'm trying to create an example but I'm unable to define the enumerator CompressionLevel, which is the second parameter of the DeflateStream Constructor.

I'm not used to work with .net within ABL.

Posted by pliscki on 26-Jan-2016 09:48

Thanks,
But my problem is with the XML compression. Just then I can convert it to Base64.



Posted by Peter Judge on 26-Jan-2016 11:16

Here is a rough transcription from one of the MSDN examples. The syntax is quite close: the big changes are the line terminators (. Instead of ;) and the method/function indicators (: intead of .).
 
using  System.*.
using  System.IO.*.
using  System.IO.Compression.*.
 
def var originalFileStream  as FileStream .
def var compressedFileStream  as FileStream.
def var compressionStream  as DeflateStream.
 
file-info:file-name = 'TestURI.log'.
 
originalFileStream = File:OpenRead(file-info:full-pathname).
 
message  originalFileStream:length.
 
compressedFileStream = File:Create(session:temp-dir + "testuri.cmp").
 
compressionStream = new DeflateStream(compressedFileStream, CompressionMode:Compress).
 
originalFileStream:CopyTo(compressionStream).
 
compressionStream:Dispose().
 
message 'done '  .
 
 
 
 

Posted by pliscki on 27-Jan-2016 05:20

Thanks Peter,

On 11.6 it seems to work, but on 10.2b it doesn't compile due to the following error:

Could not locate element 'CopyTo' in class 'System.IO.FileStream'. (12927)

Anyway, I wonder if I could use a MemoryStream instead of a FileStream in order to work just with just memptrs.

I was able to convert a memptr into a MemoryStream, though I'm not sure if it works as I couldn't convert it back to memptr.

def var mXML as memptr no-undo.
def var mResult as memptr no-undo.
def var streamSize as int no-undo.
def var memStream as MemoryStream no-undo.
def var i as int no-undo.

/* load memptr var with some value */

/* memptr to MemoryStream */
streamSize = get-size(mXML).   
memStream = new MemoryStream(streamSize).

do i = 1 to streamSize:
 memStream:WriteByte(get-byte(mXML, i)).
end.

/* MemoryStream to memptr*/
set-size(mResult) = memStream:Length.
do i = 1 to memStream:Length:
   put-byte(mResult, i) = memStream:ReadByte().
end.

Posted by pliscki on 27-Jan-2016 07:13

Answering my own question.

CopyTo doesn't work on 10.2b because it is available just for .NET version 4.0 and above.

According to progress, 10.2.b only supports .NET 3.0.

knowledgebase.progress.com/.../000054406

Posted by tbergman on 31-Jan-2016 06:39

There are some compression examples included in the download of my PugChallenge presentation on .Net.

I 'm not sure if they will work on 10.2B.

http://pugchallenge.org/downloads2015/211_DotNet.zip

Look in PugUtils.cls

Tom

Posted by Alon Blich on 31-Jan-2016 07:43

in case you'd like to try the zlib library in the standard libraries project at the oehive.org.

it will work on your version and unix/linux.

<code>
{slib/slibz.i}

define var cIn as longchar no-undo.
define var cOut as longchar no-undo.
define var iSize as int no-undo. 

cIn = " lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum lorem ipsum".

run z_compressStr64( cIn, output cOut, output iSize ).
message string( cOut ) view-as alert-box.

run z_uncompressStr64( cOut, iSize, output cIn ).
message string( cIn ) view-as alert-box.
</code>

hth

This thread is closed