Hi all,
In a folder on disk I have a series of CSV files, each containing 1 row of data. I want to read all of the rows from all of these files into a single stream.
Thus far the only way I can figure to do this is via two separate streams, one for the list of files and the other for each individual file... the contents of each I would then read into a single temp-table for future reference.
For example, something like...
/* assume variables and temp table are already defined */
DEFINE STREAM instr_files.
DEFINE STREAM instr_recs.
INPUT STREAM instr_files THROUGH VALUE("ls data/*.csv").
REPEAT:
IMPORT STREAM instr_files DELIMITER "?" vc-filename NO-ERROR.
INPUT STREAM instr_recs FROM vc-filename.
REPEAT:
IMPORT STREAM instr_recs DELIMITER "," vc-message-raw NO-ERROR.
CREATE tt-messages.
ASSIGN
tt-messages.tc-field1 = vc-message-raw[1]
tt-messages.tc-field2 = vc-message-raw[2]
tt-messages.tc-field3 = vc-message-raw[3]
tt-messages.tc-field4 = vc-message-raw[4]
.
END. /* end read messages */
END. /* end read files */
My question is, is there a way to do this all in a single stream loop?
As a stab in the dark, I tried...
INPUT STREAM instr_files THROUGH VALUE("cat data/*.csv").
...but that doesn't work :)
Thanks
Regards,
Chris
- Check INPUT FROM OS-DIR to get the list of files
- To open the files, that's where you need to use INPUT FROM VALUE(<expression>). <expression> should resolve to a relative or absolute pathname -> easiest is to use the absolute path as returned by INPUT FROM OS-DIR.
Alternatively you can read the files into a longchar using copy-lob. You can handle the longchar as you would a char. That way you only have one stream. For small files I find it a lot more easy to use than streaming.
- Check INPUT FROM OS-DIR to get the list of files
- To open the files, that's where you need to use INPUT FROM VALUE(<expression>). <expression> should resolve to a relative or absolute pathname -> easiest is to use the absolute path as returned by INPUT FROM OS-DIR.
Alternatively you can read the files into a longchar using copy-lob. You can handle the longchar as you would a char. That way you only have one stream. For small files I find it a lot more easy to use than streaming.
Thanks Frank and James. Two good solutions!
On my env INPUT STREAM instr_files THROUGH VALUE("cat data/*.csv") works fine.
10.1C04
AIX 6.1.0.0