Best way to persistently store LONGCHAR data

Posted by jmls on 19-Apr-2011 14:14

I have a need to have a persistent longchar variable, and was just wondering what the best way to go about this is. There is no DB attached.

1) define a temp-table with a field as CLOB and copy-lob the longchar to the file. export the temp-table. This creates the .txt file for the other fields, and a .blb file for the CLOB

2) define a temp-table with a field as CLOB and xml-write the temp-table.

3) COPY-LOB the longchar to a file and back, exporting the temp-table for the other fields.

4) define a temp-table with a field as CLOB. Base64-Encode the longchar, store it in the CLOB and write-xml the temp-table to a file.

5) send the data to some sort of messaging server to store, and fire a retrieve message to get it back

6)

1) and 3) creates two files, which I don't really want to have to manage

2) seemed to cause some problems reading back in. Maybe my code (probably). However, the contents of the xml file can be modified outside the application's control

4) works well, and obscufates the data (trivial exercise to decode, but stops silly alterations)

5) seems way OTT

Anyone got any other ideas ?

All Replies

Posted by Admin on 19-Apr-2011 14:40

I have a need to have a persistent longchar variable, and was just wondering what the best way to go about this is. There is no DB attached.

Low long are you going to store it? How often do you need to retrieve it? How important is it that nobody modifies the data out of the applications control? How large are the files?

You've missed the most obvious one, I guess: Store it in the OpenEdge DB as a CLOB. Via AppServer, when there is no DB attached.

Posted by jmls on 19-Apr-2011 14:45

1) Storing indefinitely. The file will be put into a version control system

2) Not very often

3) very

4) files can be 1-100k+

5) Can't use appserver as there may be a distributed scenario where

appserver connections are simply just not available

Thanks !

Posted by Matt Baker on 19-Apr-2011 14:53

Why involve a temp-table at all?

If you are worried about modifications, encrypt the data with a simple password.

define variable data as memptr no-undo.

define variable mykey as raw no-undo.

mykey = generate-pbe-key("simplepassword").

define variable utfdata as longchar no-undo init "some chunk of data".

copy-lob utfdata to data convert target codepage "utf-8". 

data = ENCRYPT (data, mykey).

output to value("backup.enc").

export data.

output close.

input from value("backup.enc").

import data.

input close.

data = decrypt(data, mykey).

copy-lob from data to utfdata convert target codepage "utf-8".

set-size(data) = 0.

MESSAGE string(utfdata)

    VIEW-AS ALERT-BOX.

If you need other fields, you could probably borrow the OERA code for serializing objects.

Posted by jmls on 19-Apr-2011 15:05

I'm using a temp-table because internally I have an Object which needs

(de)serializing , and the longchar data is just one of the properties

of this object. I've got auto-generated code that moves the object to

a temp-table and back, but now I have a need for a longchar property

However, the password is a useful tip. Thanks.

Posted by Peter Judge on 19-Apr-2011 20:52

jmls wrote:

I have a need to have a persistent longchar variable, and was just wondering what the best way to go about this is. There is no DB attached.

1) define a temp-table with a field as CLOB and copy-lob the longchar to the file. export the temp-table. This creates the .txt file for the other fields, and a .blb file for the CLOB

2) define a temp-table with a field as CLOB and xml-write the temp-table.

3) COPY-LOB the longchar to a file and back, exporting the temp-table for the other fields.

4) define a temp-table with a field as CLOB. Base64-Encode the longchar, store it in the CLOB and write-xml the temp-table to a file.

5) send the data to some sort of messaging server to store, and fire a retrieve message to get it back

6)

1) and 3) creates two files, which I don't really want to have to manage

2) seemed to cause some problems reading back in. Maybe my code (probably). However, the contents of the xml file can be modified outside the application's control

4) works well, and obscufates the data (trivial exercise to decode, but stops silly alterations)

5) seems way OTT

Anyone got any other ideas ?

"Persistent" seems to imply writing to either disk or DB. Since the DB is out, you're left with disk. Messaging doesn't sound very persistent to me, unless you can guarantee that the messaging server lives forever (and in reality, it'll probably persist the message to disk or DB itself).

I would serialise the whole object to a single file and as Matt points out, there is an AutoEdge example on this javascript:;  for binary serialisation*. If you want to prevent modifications, encrypt the file before writing to disk. This is closest to your #4.

Another consideration as to the format of the file (XML or JSON or a binary protocol or what-have-you) is whether you reasonably can expect the format of that object to change. If so, then the XML route may be best.

-- peter

* it's my intent to have more serialisation options (XML, JSON etc) in AETF than just the simple binary protocol that's there now. All contributions gladly accepted

Posted by stevenseagal008 on 21-Apr-2011 17:49

Use git.  You can use it like a file storage DB if you want, accessing blobs by SHA-1.  Also, if the content ever changes, the SHA-1 would change as well (i.e. you can detect tampering).

Since it's a distributed version control system, you don't need a central server.  Just do 'git init' and you have yourself a fresh repo.

If you do want to sync with a remote server (you say you have intermittent remote access) - it's easy: git push / git pull.

Posted by gus on 22-Apr-2011 09:43

you could do something like this instead of messing about with temp-tables.

define variable data as longchar no-undo init "hello Julian".

def var filename as char no-undo init "zzz.dat".

output to value (filename).

export data.

output close.

Posted by jmls on 22-Apr-2011 10:55

The reason I use temp-tables is that I am trying to persist an object

and it's properties, one of which is a longchar.

This thread is closed