Adaltas

Node CSV version 0.2 with streaming API

Announced in august, the Node CSV parser in its version 0.2 has just been released. This version is a major enhancement as it aligned the parser with the best Node.js practice in respect of streams. The CSV parser behave both as a Stream Writer and a Stream Reader.

Be carefull, to achieve this goal, a few changes in the API were required which make the compatibility slightly broker.

Migration

I’m trying to remember all the changes in the API. I will keep this section updated with your suggestions in case I forget.

The functions ‘from’ and ‘to’ are now rewritten as ‘from.’ and ‘to.’. The ‘data’ event is now the ‘record’ event. The ‘data’ now recieved a stringified version of the ‘record’ event.

The new Stream API

This is the most important enhancement which was announced in my last post. This little schema illustrate the structure of the stream architecture from Node.js applied to the CSV parser:

1
2
3
4
5
6
7
8
9
|-----------|      |---------|---------|       |---------|
|           |      |         |         |       |         |
|           |      |        CSV        |       |         |
|           |      |         |         |       |         |
|  Stream   |      |  Writer |  Reader |       |  Stream |
|  Reader   |.pipe(|   API   |   API   |).pipe(|  Writer |)
|           |      |         |         |       |         |
|           |      |         |         |       |         |
|-----------|      |---------|---------|       |---------|

As you can see, this new version is fully cmpliant with the stream API. It is both an Stream Writer to send input data and a Stream Reader to access output data.

Example:

1
2
3
fs.createReadStream( './in' )
.pipe( csv() )
.pipe( fs.createWriteStream('./out') )

Convenient functionnalities

Alternatively, it comes with convenient functions accessible by the from and to properties. Some of those functions were already present in the 0.1 release and are simply renamed. For exemple, the csv.fromPath() function is now csv.from.path(). New functions have been added such a csv.to.string.

Example:

1
2
3
csv()
.from.path( './in' )
.to.string( function(data){ console.log(data) } )

Documentation

Like I have done in the past in many projects like mecano, the readme content has been reduced to a minimum and the documentation is generated directly from the source code. A small script was written specifically from that purpose. The idea is to document each function with comments written in a markdown syntax. A simple regexp parser read each files, extract the comment and write markdown file inside a “./doc” folder. The doc folder is finally copied into the Jekyll directory of this website before being uploaded here.

Note, at the time of this writing, the script need some improvements and the API documentation need to be reviewed and enhanced (check the markdown syntax, typos). Not being a native english speaker doesn’t help as well. As always, your contributions are appreciated.

Conclusion

Please try the new version and let me know how you feel with it.

Comments

Fork me on GitHub