Adaltas

Node CSV version 0.1 and future developments

The Node CSV parser has just reach version 0.1 which close the 0.0.x releases. Started almost 2 years ago, the project has received a tremendous amount of participation in the form of bug reports, pull request and emails. It is used by a large population of the NodeJs community and could now be considered as stable.

Well, this isn’t the best news. The really awesome news is that I have now some time to start the development of the v0.2 branch. This paves the way to some exciting developments which I am going to present in more details.

Backward compatibility

We say in french that we can’t prepare scramble eggs without breaking some eggs. So yes, the API will change, for the best. Since the beginning of the project, it felt wrong and the incomplete support of the stream API makes it non standard at best and, to be honest, hard to understand and use properly.

If your project rely on the 0.0.x or 0.1.x versions, don’t feel nervous. We will maintain support and any bug that we found will be ported to those versions.

The new Stream API

This little schema illustrate the structure of the future code:

1
2
3
4
5
6
7
8
9
 |------------|        |------------|------------|         |------------|
 |            |        |            |            |         |            |
 |  csv.from  |        |           CSV           |         |   csv.to   |
 |            |        |            |            |         |            |
 |   Stream   |        |   Writer   |   Reader   |         |   Stream   |
 |   Reader   | .pipe( |    API     |    API     | ).pipe( |   Writer   | )
 |            |        |            |            |         |            |
 |            |        |            |            |         |            |
 |------------|        |------------|------------|         |------------|

A new CSV instance will be fully cmpliant with the stream API. It will be both an Stream Writer to send input data and a Stream Reader to access output data.

Example:

1
2
3
fs.createReadStream( './in' )
.pipe( csv() )
.pipe( fs.createWriteStream('./out') )

Convenient functionnalities

Alternatively, it will comes with convenient functions accessible by the from and to properties. Some of those functions were already present in the 0.1 release and are simply renamed. For exemple, the csv.fromPath() function is now csv.from.path(). New functions are planned such a the csv.to.string and the csv.to.array.

Example:

1
2
3
csv()
.from.path( './in' )
.to.path( './out' )

Code isolation

Until now, all the source code present inside a single file. From now on, the code layout will be:

  • ”./lib/from.js” Input convenient functions
  • ”./lib/to.js” Output convenient functions
  • ”./lib/csv.js” Core function implement the Stream API
  • ”./lib/options.js” Default input and output options
  • ”./lib/state.js” Default state object

This refactoring shall simplify the understanding of the source code and help new comers to prepare pull request which are particurly welcome.

Documentation

Like I have done in the past in many projects like mecano, the readme content will be reduced to a minimum and the documentation will be generated directly from the source code. A small script will be written specifically from that purpose. The idea is to document each function with comments written in a markdown syntax. A simple regexp parser read each files, extract the comment and write markdown file inside a “./doc” folder. The doc folder is finally copied into the Jekyll directory of this website before being uploaded here.

I find it much easier to document a project on a per function basis than by maintaining coherent a doc forlder or a large readme file.

CoffeeScript

This project was probably the last one I could claim to be written in JavaScript. I’m not here to start a war. I didn’t choose CoffeeScript to be cool, I use it for a year and a half now, and I find writting old JS to be a pain. The test have be translated some time ago but the production code was still in JavaScript. Starting from version 0.2.0, the code will be maitain in CoffeeScript. The source file will be located inside the “./src” folder and the transpiled JavaScript will be located inside the “./lib” folder.

Conclusion

It is an excellent time to watch the development of this new version and come with your own feedback. I’ll be happy to listen to your suggestions.

Comments

Fork me on GitHub