ned Productions Consulting


Technology musings by Niall Douglas
ned Productions Consulting
(an expert advice and services company based in Ireland)


Friday 24th February 2017 10.56am

Link shared: http://www.dabeaz.com/ply/

Still feeling poorly, aches, sweats and no appetite, and little energy. You have to fight yourself to do anything. Been like this for quite a few days now, and I'm looking forward to it lifting because it's highly non-helpful for productivity with the ACCU conference just a few weeks away now.

Speaking of productivity, I am still (slowly) banging away on my Python C preprocessor during Clara naptimes. Last week I realised that I cannot avoid tokenisation and colouring thereof to correctly implement a C preprocessor, so I started looking into PLY, a Python Lex-Yacc library (http://www.dabeaz.com/ply/). Turns out, to my slight annoyance, that it ships with a C preprocessor as a use case example. So that's two weeks of free time effectively wasted :(

PLY's example preprocessor is therefore now my starting base. It has quite a few bugs in it, it was never intended as anything past a demo, so I've been fixing those bugs and getting it closer to standards conformance. It's a full tokenising preprocessor, so performance will not be stellar, but it'll be good enough and it handles stuff like expanding f(f(z)) where f(x) expands to f(2 * (x)) etc just fine. Its single biggest limitation is a total lack of rescanning support, so you're supposed to retokenise after expansion and do more expanding. That requires colouring the tokens, something PLY's implementation doesn't bother with.

At this slow pace of development, I may need to pause this preprocessor and simply get Outcome into the peer review queue in time for ACCU - or rather, immediately after it. The preprocessed includes were always a nice to have rather than anything important.