skip the boring parts
After fixing a pretty serious bug in the combination of svenson and jcouchdb and releasing svenson 1.3.4 and jcouchdb 0.9.1-4 and jcouchdb 0.10.0-2, I got around to release a new, improved version of Hood, too. It’s called “Beta 2” and still lacks some features but the documentation is a lot better and the application design is basically what I think will be the final layout of things.
Head over to google code and grab yourself a copy.
Links:
On the occasion of presenting CouchDB and jcouchdb at my place of work, I got around to finally create a small example application that is now downloadable as sneak preview. There need to be bugs fixed, features implemented and lots of documentation to be added, but it kind of works.
It’s called “Hood” for neighbourhood and allows you to mark places or people around a place of activity of yours, called hood. it is meant to foster collaboration / tips on local places etc.
It’s Spring Web Application demonstrating some techniques of working with jcouchdb. It’s an eclipse WTP/Spring IDE project with all dependencies you need besides couchdb and tomcat or another servlet container.
Stay tuned for hood to grow into a fullblown app.
Links:
jcouch db logo
This blog has been relatively stale for a while now — mostly due to increasing work pressure and the limited amount of time I had left for it. What little time I had was mostly wasted on coding on it. The previous version was written in turbogears which seemd like a good idea back in 2006. Over time I kind of regretted that decision, which was not really Turbogears fault, but just caused by my laziness to really update it etc.
I’ve been frustrated with the old blog code and the old design for some time now and wanted to redo everything and already started multiple attempts in various technological reincarnations — the latest being an implementation in java/spring using my jcouchdb library. The design got more and more ambitious over time; so ambitious that it would basically require me to work full-time on it. Continue reading
Implementing a streaming attachment feature for jcouchdb, I started to wonder whether it would be a good idea for svenson to support JSON parsing from a stream, too, as I don’t really need the complete stream to start constructing the java object graph.
Implementing stream parsing was really nice and easy thanks to the units test present in svenson. After that, I came upon two ways to generally cut down on memory use. All tokens with fixed values could just have a single instance. The recording of tokens to provide token based look ahead was not really needed in all cases. But how much does that save?
As a test case I wrote a small tool class to generate random, nested JSON datasets, generated two test files of 65kb and 4.5mb size and parsed these with svenson 1.2.8 and what now is svenson 1.3.
Measuring the actual memory usage for these two test files proved to be difficult. Somehow none of the programs I tried seemed to give me the data I wanted. Eclipse TPTP just ignored Strings that were no member of any class but just parameters, making stream and string parsing look exactly the same memory-wise. tijmp and others did not provide the data I wanted at all.
So in the end I wrote a little python script that parses a hprof ASCII output to
This provided meaningful data and also showed some points for further improvement. There was a huge number of java.lang.reflect.Method allocations which turned out to be caused by svenson inspecting the target classes for annotations and appropriate methods which was done on a per target basis instead of the better per target class basis.
All in all the memory usage went down quite a bit:
memory usage for different svenson versions, with and without streaming
45% less memory for the small file and 62% for the large file for all allocations. I think that is really good..
Below are some links to the files needed to repeat the benchmarking. The transform hprof script might also prove to be useful for other projects if changed appropriately.
The new jcouchdb release will also use stream parsing.
Links:
edit:
The command to generate the hprof file was something like
java -agentlib:hprof=heap=sites,depth=100,cutoff=0 -cp .. svensonperf.ReadJSONOld big.json
